Process ENR data with Python: recast as if it were UHDAS data =============================================================== More detail exists in the documentation. ============================================ The directory structure here matches the directory structure in qpy_demos. - adcp_pyproc - vmdas_data - ps0918 - uhdas_data - fake_uhdas_data We start with a command line (terminal) in adcp_pyproc where we will be working. (1) Make needed directories: mkdir ps0918_enrproc mkdir ps0918_enrproc/config mkdir ../fake_uhdas_data (2) stage for singleping processing. (i) get info about data files cd ps0918_enrproc vmdas_info.py --logfile lta_info.txt os ../../vmdas_data/ps0918/*LTA summary: - these are 'bb' pings - transducer angle was 1.18 - N1R is a gyro (primary heading) - N2R is an Ashtech (heading correction) - N3R is a GPS (positions) (ii) convert VmDAS data to UHDAS data (a) cd config reform_vmdas%s.py ../../.. - vmdas: browse for the ps0918 vmdas directory - uhdas: browse for the *root* uhdas_data directory (python will make the actual cruise directory) - shipID (use 'zzz' -- the rest are already assigned) - instrument name -- "os75" in this case - cruise name: specify something like 'ps0918_os75' to distinguish cruise and instrument CLICK: "Make config files" ==> writes two output files reform_defs.py (will be used in step 4) vmdas2uhdas.py (transform vmdas data into uhdas dasta) (b) Transform Vmdas data into UHDAS data now: python vmdas2uhdas.py This creates N1R, N2R, N3R directories in the vmdas_data directory, and then recasts the OS75 and all these as UHDAS data in the uhdas_data/ps0918_os75 directory. (iii) Finally, run the following program, which will create the processing configuration file for your new 'fake' UHDAS data. You need to specify the file with variables for this: proc_starter%s.py reform_defs.py Fill the variables with information from vmdas_info.py clockwise transducer angle : 1.18 (type this) transducer depth (m) : 3 (type this) position (feed, message) : ('N3R','gps') dropdown heading (feed, message) : ('N1R','hdg') dropdown heading correction : ENABLE (feed, message) : ('N2R','adu') dropdown CLICK: "Make config files" ==> writes an output file ps0918_os75_proc.py which will be used by quick_adcp.py for processing. ==> You *must* follow the instructions: Your quick_adcp.py control file should start with this: --yearbase 2009 --cruisename ps0918_os75 (3) setup and run processing (a) Now go back one directory to "ps0918_enrproc" and make the OS75 ENR processing directory: cd .. adcptree.py os75bb --datatype uhdas --cruisename ps0918_os75 cd os75bb NOTE: You *must* use the same "cruise name" here that is specified above. This is the prefix for the file just written in the "config" directory (ps0918_os75_proc.py). Adcptree identifies this file by the prefix contained in the "cruisename". The CODAS processing dirctory is the first argument seen by adcptree.py ("os75bb"). This name is not used by any other programs. (b) change directories to ADCP processing directory just created and make the quick_adcp.py control file (ii) create a quick_adcp.py control file windows: - you can create q_py.cnt with an editor to have the parts between (or including) the comments - it may be easier to call it "q_py.txt" to avoid fighting with your editor, which may try to set the suffix unix: this example is written as a bash "heredoc" for cut-and-past ease, but you can create q_py.cnt with an editor to have the parts between (or including) the comments cat << EOF > q_py.cnt ### q_py.cnt is ## python processing --yearbase 2009 ## required, for decimal day conversion ## (year of first data) --cruisename ps0918_os75 ## *must* match prefix in config dir --dbname aship ## database name; in adcpdb. eg. a0918 --datatype uhdas ## datafile type --sonar os75bb ## specify instrument letters, frequency, ## (and ping type for ocean surveyors) --ens_len 300 ## averages of 300sec duration --update_gbin ## required for this kind of processing --configtype python ## file used in config/ dirctory is python --ping_headcorr ## ps0918_proc.py says use HDT first, ## correct to ashtech --max_search_depth 1500 ## use topography for editing? ## 0 = "always use amplitude to guess the bottom; ## flag data below the bottom as bad ## -1 = "never search for the bottom" ## positive integer: use ADCP amp to autodetect ## the bottom. Only do this in "deep water", ## i.e. topo says bottom is deeper than this EOF (c) run quick_adcp.py: quick_adcp.py --cntfile q_py.cnt ---------- (7) (a) check calibration: **watertrack** (cal/watertrk/adcpcal.out) ----------- Number of edited points: 2 out of 2 amp = 1.0030 + 0.1228 (t - 264.8) phase = -0.01 + 3.2807 (t - 264.8) median mean std amplitude 1.0030 1.0030 0.0099 phase -0.0150 -0.0150 0.2645 ------------ **bottomtrack** ------------ unedited: 21 points edited: 18 points, 2.0 min speed, 2.5 max dev median mean std amplitude 0.9996 1.0004 0.0044 phase 0.3090 0.3392 0.1713 ------------ (b) look at heading-correction plots figview.py cal/rotate/*png ---------- (8) apply rotation correction (no scale factor necessary) (all in one line) quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_angle 0.32 --auto **bottomtrack** ------------ unedited: 21 points edited: 18 points, 2.0 min speed, 2.5 max dev median mean std amplitude 0.9995 1.0004 0.0045 phase -0.0232 0.0223 0.1739 ------------ (9) now edit cd edit gautoedit.py -n5 # apply editing: cd .. quick_adcp.py --steps2rerun apply_edit:navsteps:calib --auto # check editing (use "dataviewer.py") # -- looks OK (10) make web plots (copy sections from LTA processing) mkdir webpy cp ../../ps0918_ltaproc/os75bb_lta/webpy/sectinfo.txt webpy quick_web.py --redo # or make manually: quick_web.py --interactive - view with a browser, look at webpy/index.html (11) extract data quick_adcp.py --steps2rerun matfiles --auto adcp_nc.py adcpdb contour/os75bb ps0918_demo os75bb -----------