3.6.1. UHDAS: DAS.py, timers, and data flow

UHDAS has four components at sea: acquisition, processing, monitoring, and access (to data and figures). A longer introduction to these components is in the Monitoring at Sea Section.

Some of the tasks UHDAS performs are system services, such as

  • apache (web serving)

  • samba (export directories as Windows shares)

  • NFS (export directories using Network File System protocol)

  • cron (allows us to schedule a once-per-day email

All the rest are driven by programs ultimately spawned by the UHDAS program, DAS.py. These come in three categories:

  • programs that run for a long time (eg. the duration of serial logging)

  • programs that run sporadically, expected to take a few seconds (2-20) each, and should die at the end. These programs run on timers

A typical life cycle of DAS.py would be as follows… (click on images for a larger view)

typical life cycle of DAS.py

3.6.1.1. Start DAS.py

  • reads /home/adcp/config/sensor_cfg.py and /home/adcp/config/procsetup_onship.py and pops up a graphical tool (see the UHDAS Gui tutorial for more detail).

starting UHDAS

3.6.1.2. Start cruise

  • uses the cruise name typed by the ADCP Operator

  • creates the directory structure for a new cruise in /home/data

  • creates two symbolic links to this new cruise directory

    • /home/data/current_cruise

    • /home/adcp/cruise (signals the cruise is active)

  • starts 2-hour timer for backups:

    • rsync /home/data to any specified disks

starting UHDAS

3.6.1.3. Start recording

3.6.1.3.1. Data acquisition: ascii files (“raw”)

  • communicates with ADCP(s) to start pinging

  • passively acquires strings from other serial ports

    • starts ser_asc and ser_bin processes for acquisition and rbin generation

start recording raw data raw data file format

3.6.1.3.2. Data acquisition: binary files, parsed from ascii (“rbin”)

  • makes intermediate versions of serially-logged ascii data:

    • rbin (parse ascii files, save as binary)

translation from raw (ascii) to rbin rbin directory structure

3.6.1.3.3. Data processing: 5-minute timer (gbins)

  • starts 5-minute timers for averaging (preliminary processing)

  • one 5-minute timer per ADCP+pingpype

    • makes gbin files (align serial data and pings to same UTC time)

gbin directory structure

3.6.1.3.4. Data processing: 5-minute timer (create averaged profile)

  • using the same 5-minute timer

    • compute and store one 5-minute average ocean velocity

    • make 5-minute profile plots; update web page

staging 5-minute profile

3.6.1.3.5. Data processing: 30-minute timer (CODAS processing)

  • starts 30-min timers for CODAS processing and figures

  • one 30-min timer per ADCP+pingtype

    • updates CODAS files

    • runs postprocessing (editing, etc) steps

    • makes contour and vector plots from CODAS database (1hr)

    • stores averaged data for science

updating CODAS files, figures on web site

3.6.1.4. Stop recording

  • wake up ADCPs (quit pinging)

  • terminates ser_asc and ser_bin processes (no more logging)

  • termintes all timers

3.6.1.5. End cruise

  • copy archived figures from /home/adcp/www to relevant processing directory

  • break symbolic link in /home/adcp (the one in /home/data is persistent)

  • run last rsync, kill the rsync timer

additional notes about DAS.py

  • If the UHDAS graphical tool is killed while logging is active, the serial acquisition processes and the timers continue to run. Restarting the gui will reconnect it to the processes that are already running.

  • If a change is made to sensor_cfg.py, proc_cfg.py, or uhdas_cfg.py

    • you need to start a new cruise segment, since this is a change in metadata and/or logging parameters

    • you must kill the gui and restart it, because DAS.py reads the two files when it starts

    • if (after a change) the gui refuses to start, you probably typed something wrong. Look in /home/adcp/log for recent *.warn files and try to find the typo.