Process UHDAS data using Python ============================================= In order to use all-python processing, we will generate the configuration file necessary from modern ship's settings and edit as needed. This file will be (a) consistent with all-python processing (b) guarantees newest file format (c) is more flexible (1a) get some information about this cruise: uhdas_info.py --overview --logfile uhdas_info.txt ../../uhdas_data/km1001c We see: - positions from ashtech, simrad, and posmv - headings from gyro, posmv and ashtech - 38 files; all present (1b)# in km1001c_fullproc make a directory called 'config' # and run: mkdir config cd config uhdas_proc_gen.py -s km # This creates a file called "km_proc.py" # copy to a new version for our cruise (use the cruisename "km1001c"): cp km_proc.py km1001c_proc.py #Edit the file and add these five (5) # lines to the top of km1001c_proc.py # full path to uhdas data directory (do not use relative path) uhdas_dir = '/home/currents/programs/qpy_demos/uhdas_data/km1001c' yearbase = 2010 # usually year of first data logged shipname = 'Kilo Moana' # for documentation cruiseid = 'km1001c' # for titles # go back one level cd .. (2) now we can use that file. Set up the processing directory by typing: adcptree.py os38nb --datatype uhdas --cruisename km1001c NOTE: Now there is no error message about missing configuration files (3) (a) change directories to ADCP processing directory just created cd os38nb NOTE: there is a "config" directory and it has a copy of the file km1001c_proc.py that we created in #2 (b) create a quick_adcp.py control file unix: this example is written as a bash "heredoc" for cut-and-past ease, but you can create q_py.cnt with an editor to have the parts between (or including) the comments windows: - you can create q_py.cnt with an editor to have the parts between (or including) the comments - it may be easier to call it "q_py.txt" to avoid fighting with your editor, which may try to set the suffix cat << EOF > q_py.cnt ####----- begin q_py.cnt------------ ## all lines after the first "#" sign are ignored ## python processing --yearbase 2010 --cruisename km1001c # used to identify configuration files # *must* match prefix of files in config dir --update_gbin ## NOTE: You should generally remake gbins ## - you are not sure ## - if parameters for averaging changed ## - various other reasons. ## ==> MAKE SURE you move the original gbin directory ## to another name first!! --py_gbindirbase gbin # (will put them adjacent to q_py.cnt) # The other option is to move the # original gbins out of the way, # to a new directory (eg. # gbin.orgimat) --configtype python ## <=== USE THE NEW FILE WE CREATED --sonar os38nb --dbname aship --datatype uhdas --ens_len 300 --ping_headcorr ## applies heading correction. ## settings found in config files --max_search_depth 3000 ## use topography for editing? ## 0 = "always use amplitude to guess the bottom; ## flag data below the bottom as bad ## -1 = "never search for the bottom" ## positive integer: use ADCP amp to autodetect ## the bottom. Only do this in "deep water", ## i.e. topo says bottom is deeper than this EOF (c) run quick_adcp.py: quick_adcp.py --cntfile q_py.cnt --------- (4) review the data (a) check calibration:heading correction device: figview.py cal/rotate/*png conclude: no action needed (b) check calibration: **watertrack** ------------ Number of edited points: 34 out of 35 amp = 1.0074 + -0.0033 (t - 30.3) phase = 0.05 + 0.1635 (t - 30.3) median mean std amplitude 1.0080 1.0074 0.0064 phase 0.0900 0.0511 0.4541 ------------------- (c) look at the data: dataviewer.py (5) edit "gautoedit" cd edit gautoedit.py -n5 # apply editing: cd .. quick_adcp.py --steps2rerun apply_edit:navsteps:calib --auto # check editing -- looks OK dataviewer.py (6) - check calibrations again **watertrack** ------------ Number of edited points: 34 out of 35 amp = 1.0071 + -0.0020 (t - 30.3) phase = 0.05 + 0.1809 (t - 30.3) median mean std amplitude 1.0080 1.0071 0.0058 phase 0.0900 0.0494 0.4431 ------------ still need scale factor: quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_amplitude 1.007 --auto **watertrack** ------------ Number of edited points: 34 out of 35 amp = 1.0001 + -0.0018 (t - 30.3) phase = 0.05 + 0.1840 (t - 30.3) median mean std amplitude 1.0010 1.0001 0.0058 phase 0.0935 0.0496 0.4439 ------------ phase correction is very small: quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_angle 0.06 --auto **watertrack** ------------ Number of edited points: 34 out of 35 amp = 1.0001 + -0.0018 (t - 30.3) phase = -0.01 + 0.1828 (t - 30.3) median mean std amplitude 1.0010 1.0001 0.0057 phase 0.0365 -0.0095 0.4437 ------------ OK, done (7) make plots: quick_web.py --interactive - or, to use the same sections as were used in the postprocessing demo mkdir webpy cp ../../km1001c_postproc/os38nb/webpy/sectinfo.txt webpy quick_web.py --redo - view with a browser, look at webpy/index.html (8) extract data adcp_nc.py adcpdb contour/os38nb km1001c_demo os38nb quick_adcp.py --steps2rerun matfiles --auto ---- done with processing. Add os38nb/cruise_info.txt to the top of this file. Edit with correct info