7.1. Changes and Updates

(2021/05/15)

Changes:

  • flattened these sections: Installation, Processing, and Operations

  • added more troubleshooting, added uvship

  • revised for 20.04

  • more Sphinx-y

(2019/10/15)

Changes:

  • rearranged slightly

  • added more troubleshooting

(2018/10/01)

Changes:

Now working completely in Python3.

  • replaced (and improved) dataviewer.py – only runs under Python3

  • added adcp_database_maker.py (to take UHDAS and VmDAS data to the poseprocessing stage)

  • revamped patch_hcorr.py

(2015/05/01)

Changes:

  • Python2/Python3 compatibility added. Now requires python-future

  • Changed installation instructions to support running the code using Python2 or Python3, with VirtualEnv or Anaconda. This makes for a more standard installation process and greatly reduces the $PATH (and eliminates adjustments to $PYTHONPATH). The old mechanism, where executables and libraries and source were all mixed together, presently works, but is no longer supported.

(2014/02/14)

Changes:

  • Changed the way reference layer smoothing was called:

    • These options are no longer available:

      • --use_smoothr

      • --use_refsm

    • Instead, use:

      • --ref_method smoothr (for smoothr; no choice for pingdata)

      • --ref_method refsm (default; permits options below)

  • Can also now specify offset between GPS and ADCP, eg:

    • --xducer_dx 2 --xducer_dy 30

  • Can also now specify the half-width of the reflayer smoothing window:

    • --refuv_smoothwin 3 (the default) or use 2 for less smoothing, or 0 (or 1) for “no smoothing”

(2013/10/31)

Changes:

  • gautoedit.py: add “Reset Some Editing”; change how ascii files are treated

  • quick_adcp.py: more rationalization of options, more stripping of matlab

  • redo processing demos for CODAS Python processing

(2013/05/01)

Changes:

  • officially retire Matlab processing

  • redo processing demos for CODAS Python processing

  • redo installation to highlight CODAS Python virtual computer

(2010/08/08)

Changes:

  • revise all components for a more linear feel

  • change content to reflect new Python processing added Jan 2011

(2010/10/30)

Changes:

  • Removing any further mention of HDSS. Giving up.

(2009/05/01)

Improvements:

  • corrected but in gautoedit and set_lgb that had bottom masking (from side-lobe interference) hardwired at 30deg.

  • added documentation about this and more about editing in general

(2008/06/31)

Changes:

  • additional reorganization of python files

  • work continues with python tools for access and plotting codas data

Improvements:

  • continued efforts to improvement documentation

  • added more variables to “getmat”

  • added 64bit linux support

  • more improvements to single-ping editing

(2007/05/31)

Improvements:

  • improvements to documentation organization (including this page)

  • added “persistence” of heading correction over small gaps (UHDAS data)

  • added a distinction between yearbase (for date calculation) and proc_yearbase (in case it was specified badly, or to join two UHDAS cruises that span a year boundary)

  • More improvements to single-ping editing

Bug Fixes:

  • rewrote sonar variable specifications for HDSS so they work with Matlab R2007a

(2006/10/25)

Changes:

  • The entire directory tree is being reorganized to allow for better version control. This slightly changes the installation instructions (mostly the user’s executable and python paths).

  • We are embarking on an effort to move away from the use of Matlab in processing and display software. Over time, routines presently performed in Matlab will be ported to Python. We will attempt to keep the Matlab code running as Mathworks changes Matlab, but some things will falter, and eventually we will not maintain them. The present examples of this include the placement of a colorbar with gautoedit, and some aspects of color mapping.

Improvements:

  • Gautoedit polygon zapping and bottom editing tool was improved for use with Matlab R14

  • Codas database now stores cleaned averaged amplitude under ‘amp’ and stores individual beam amplitudes under ‘rawamp’; both are accessible with “getmat”

  • A new build system is being tested that uses python-based scons. If you wish to compile the binaries for your system, download codas3_declspec.zip and contact us for more information about gotchas and what to expect. If you embark on this route you will be a beta tester.

Bug fixes:

  • Fixed: a bug in treatment of bottom depth resulted in databases with noisy bottom depths were sometimes too large.

  • Fixed: reference layers depths behaved badly if chosen to be shallow

  • Fixed: inconsistent treatment of ensemble length caused problems for some long LTA averages

(2006/03/30) New Additions:

  • UHDAS documentation (acquisition, processing) added

(2005/10/17) New Additions:

  • improved heading-correction code

  • split python processing scripts and executables into two groups for cleaner updates

(2005/05/16)

  • experimental HDSS tools

  • UHDAS processing demo (in q_demos.zip)

  • improving documentation for single ping data

(2004/12/30)

Major upgrades:

  • Support for Matlab R14

  • Support for Macintosh OSX binaries (download binosx.zip)

Minor changes:

  • fixed a recent bug in quick_adcp.py that made ‘steps2rerun’ fail with VmDAS data

  • gautoedit buttons are more visible for 1024x768 screens

Features introduced in earlier distributions:

(2004/05/18)

  • Online documentation of Codas3 and ADCP processing. This documentation is also present in the zip file distribution “doc.zip”, so you will have it on your disk once you download all the appropriate zip files. When creating a processing directory, there will also be an html file “adcp_processing.html” containing a link to the local processing documentation.

  • A new variable-depth raference layer scheme “refsm” is now the official standard for data with non-dithered GPS fixes and only short gaps. It works well with cruises that have steep topography or patchy depth loss due to bubbles or hull noise. It does not do well if there are missing navigation points, as it currently interpolates missing positions. “Smoothr” is still required for watertrack calibration and reference layer plotting, and can still be used for navigation.

  • A new python program (quick_adcp.py can set up the gautoedit files and run through all the basic processing steps. It is useful for a quick look at a dataset and is sufficient for processing most shipboard ADCP datasets. It can handle averaged data from DAS2.48 (pingdata files) or VmDAS (LTA or STA files), and can perform configurable single-ping editing on UHDAS or VMDAS single-ping data (for example ENX). Quick_adcp.py can be run at sea on a unix machine at regular intervals, creating an up-to-date codas database for scientists. Editing of the averaged data in the CODAS database can be automated under these circumstances. Advantages to using quick_adcp.prl include:

    1. standardized names

    2. impliments the new refsm reference-layer calculation

    3. automatic setup of gautoedit files

    4. works with pingdata or LTA files

    5. works with ENX or UHDAS single-ping files

    6. consistency in parameters

    7. steps are recorded in a log file

    8. many parameters can be changed on the command line or in a control file

    9. makes it relatively painless to delete a database and start over

    10. runs under MS Windows

  • Improvements to the new editing package “ gautoedit” pronounced Gee! Autoedit include improved automatic bottom detection, manual bottom editing, and rectangle or polygon zap. This tool is a graphical means of editing CODAS ADCP data (i.e. averaged velocities) in user-determined time chunks (eg, 2-4 hours for messy data; 1-3 days for clean data and a high resolution monitor). This tool was designed to screen data for things like ringing, on-station wire interference, jittery navigation, and bottom interference when bottom tracking was not on. The old editing (best done 20-30 profiels at a time) can still be used for fine-tuning single-bin editing. This is also a useful tool for simply looking at a dataset, even if you don’t use it for editing.

  • Single-ping or averaged VmDAS data can be read directly into matlab, allowing scientists to access specfic time ranges to comput shears or to look at the backscatter amplitude.

  • The older perl script (quick_adcp.prl) is no longer supported. It has been ported to Python, and additional functionality has been added.

other tools:

  • A fairly comprehensive but undocumented and unsupported plotting tool called “gplot” is available in our matlab distribution. Add the “guiplots” subdirectory to your path. Default paths are configured with the expectation that you will be running it from the adcp processing directory. Specifically, it will need to load adcpsect output files (*_xy.mat) from a contour or vector subdirectory, and can make nice panel plots of percent good and other parameters assuming your database is in an adcpdb subdirectory.

Caveats:

  • Both adcptree.py and quick_adcp.py work under Unix and Windows systems. For Windows users, there are .BAT files supplied which access these two programs: adcptree_py.bat (for adcptree.py), and quickadcp_py.bat (for quick_adcp.py)

  • Refsm should only be used with 5-minute averages and high quality GPS data (i.e. not Selective Availability, and no gaps). Smoothr still works fine if you are stuck with jittery fixes.

    refsm will interpolate across gaps in navigation. This is probably OK for 1-2 points. If the gap is 3-6 points, and if the ship is not maneuvering, you can substitute the positions from smoothr. For longer gaps, substitute navigation from other source. If you have no navigation, edit the out profiles. Refsm leaves a log “refsm_interp.log” in the current directory (“nav” if processing, “edit” when editing)

Updates:

  • 2003/02/27: gautoedit now has an additional red button to allow flagging of the bottom. Output is written directly to abottom.asc

  • 2003/02/27:a new setup program, “adcptree.prl” allows specification of data type so that the directory can be created to process VmDAS LTA files as well as ENS files. (Processing of ENS files is not ready).

  • 2003/02/27:A new Python program joins the processing toolbox:

    “quick_adcp.py”. This has all the features of quick_adcp.prl as well as features such as: o keywords to rerun specific steps (eg. nav steps, or calibration) o can process LTA files as easily as pingdata files

  • 2003/03/03: gautoedit

    now has an additional gray bar which turns off smoothing of navigational prior to calculating ocean velocity. This might be useful if you think that ocean velocity artifacts are caused by navigation or navigational smoothing.

  • 2003/04/03: gautoedit now has one more manual editing

    buttons, “pzap”, bringing the number of buttons that immediately write to ascii files to four (the others being “Trange” and “rzap”, and “bottom”). Documentation for these buttons was updated.

  • 2003/05/05: fixed a bug in which

    on-station error velocity was actually being applied when underway also (i.e. there was no difference between on-station and underway error velocity: both were applied always).

  • 2003/05/05: added a field to adjust the

    tolerance when deleting low shallow PG. This editing is designed to identify probable ringing and delete the offending bins. You may want to adjust it or disable it, and now you can do that in the gautoedit window. Smaller numbers mean alot of PG are deleted. Larger numbers make it less effective.

  • quick_adcp.prl batch processing for pingdata (no longer supported)

  • using quick_adcp.prl to process ADCP data