Process UHDAS data using Python
=============================================
(1) directory setup: create a working directory for this cruise
mkdir km1001c_fullproc
cd km1001c_fullproc
(1a) get some information about this cruise:
uhdas_info.py --logfile uhdas_info.txt ../../uhdas_data/km1001c
cat uhdas_info.txt
We see:
- positions from ashtech and posmv
- headings from gyro, posmv and ashtech
- 138 files; all present
(1b) configuration file for processing:
In order to use CODAS Python processing, we need to generate a
suitable configuration file, containing instructions for processing
such as transducer depth, transducer angle and ancillary data sources
(heading and attitude). This new file contains modern settings and
modern syntax for the specified ship: create a new
file, with modern syntax and variable names, and then edit as needed.
# In km1001c_fullproc make a directory called 'config'
mkdir config
cd config
# and run:
uhdas_proc_gen.py -s km
# This creates a file called "km_proc.py"
# copy to a new version for our cruise (use the cruisename "km1001c"):
cp km_proc.py km1001c_proc.py
#Edit the file and add these four (4)
# lines to the top of km1001c_proc.py
# full path to uhdas data directory (do not use relative path)
uhdas_dir = '/home/currents/programs/adcp_py3demos/uhdas_data/km1001c'
yearbase = 2010 # usually year of first data logged
shipname = 'Kilo Moana' # for documentation
cruiseid = 'km1001c' # for titles
# go back one level
cd ..
(2) now we can use that file.
Set up the processing directory by typing:
adcptree.py os38nb --datatype uhdas --cruisename km1001c
NOTE: Now there is no error message about missing configuration files
(3) (a) change directories to ADCP processing directory just created
cd os38nb
NOTE: there is a "config" directory and it has a copy of the file
km1001c_proc.py that we created in #2
(b) create a quick_adcp.py control file
unix: this example is written as a bash "heredoc" for
cut-and-past ease, but you can create q_py.cnt with
an editor to have the parts between (or including)
the comments
windows:
- you can create q_py.cnt with an editor to have the
parts between (or including) the comments
- it may be easier to call it "q_py.txt" to avoid
fighting with your editor, which may try to set the suffix
cat << EOF > q_py.cnt
####----- begin q_py.cnt------------
## all lines after the first "#" sign are ignored
## python processing
--yearbase 2010
--cruisename km1001c # used to identify configuration files
# *must* match prefix of files in config dir
--update_gbin ## NOTE: You should generally remake gbins
## - you are not sure
## - if parameters for averaging changed
## - various other reasons.
## ==> MAKE SURE you move the original gbin directory
## to another name first!!
# --py_gbindirbase gbin # (will put them adjacent to q_py.cnt)
--configtype python ## <=== USE THE NEW FILE WE CREATED
--sonar os38nb
--dbname aship
--datatype uhdas
--ens_len 300
--ping_headcorr ## applies heading correction.
## settings found in config files
--max_search_depth 3000 ## use topography for editing?
## 0 = "always use amplitude to guess the bottom;
## flag data below the bottom as bad
## -1 = "never search for the bottom"
## positive integer: use ADCP amp to autodetect
## the bottom. Only do this in "deep water",
## i.e. topo says bottom is deeper than this
EOF
(c) run quick_adcp.py:
quick_adcp.py --cntfile q_py.cnt
---------
(4) review the data
(a) check calibration:heading correction device:
figview.py cal/rotate/*png
conclude: no action needed
(b) check calibration:
**watertrack**
------------
Number of edited points: 34 out of 35
amp = 1.0074 + -0.0033 (t - 30.3)
phase = 0.05 + 0.1635 (t - 30.3)
median mean std
amplitude 1.0080 1.0074 0.0064
phase 0.0900 0.0511 0.4541
-------------------
(c) look at the data:
dataviewer.py
(5) edit "gautoedit"
cd edit
gautoedit.py
# apply editing:
cd ..
quick_adcp.py --steps2rerun apply_edit:navsteps:calib --auto
# check editing -- looks OK
dataviewer.py
(6)
- check calibrations again
**watertrack**
------------
Number of edited points: 34 out of 35
amp = 1.0071 + -0.0020 (t - 30.3)
phase = 0.05 + 0.1809 (t - 30.3)
median mean std
amplitude 1.0080 1.0071 0.0058
phase 0.0900 0.0494 0.4431
------------
still need scale factor:
quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_amplitude 1.007 --auto
**watertrack**
------------
Number of edited points: 34 out of 35
amp = 1.0001 + -0.0018 (t - 30.3)
phase = 0.05 + 0.1840 (t - 30.3)
median mean std
amplitude 1.0010 1.0001 0.0058
phase 0.0935 0.0496 0.4439
------------
phase correction is very small:
quick_adcp.py --steps2rerun rotate:navsteps:calib --rotate_angle 0.06 --auto
**watertrack**
------------
Number of edited points: 34 out of 35
amp = 1.0001 + -0.0018 (t - 30.3)
phase = -0.01 + 0.1828 (t - 30.3)
median mean std
amplitude 1.0010 1.0001 0.0057
phase 0.0365 -0.0095 0.4437
------------
OK, done
(7)
make plots:
quick_web.py --interactive
- or, to use the same sections as were used in the postprocessing demo
mkdir webpy
cp ../../km1001c_postproc/os38nb/webpy/sectinfo.txt webpy
quick_web.py --redo
- view with a browser, look at webpy/index.html
(8) extract data
adcp_nc.py adcpdb contour/os38nb km1001c_demo os38nb
quick_adcp.py --steps2rerun matfiles --auto
----
done with processing.
Add os38nb/cruise_info.txt to the top of this file.
Edit with correct info
