Status: Monitoring from Shore

On land, there are two basic mechanisms for monitoring.

  • daily text email: Once per day, an email is sent from each ship to parties on shore containing a summary of information about the status of the processing and data quality. A similar email goes out to ship’s tech email account.
  • daily email attachment to Univ Hawaii: An separate email is sent with a collection of diagnostic information and heavily averaged sample of the last 3 days of processed ADCP data from each of the ADCP+pingtype data. The figures generated from the data are a potent diagnostic tool. The text email (above) is stored as one of the files in the collection of diagnostic files.

The entire collection of diagnostic files is available for troubleshooting for anyone with a WWW connection

Daily text email:

The text email is designed to provide enough information to determine at a glance whether everything is working or not. If there is a problem, the next step is to look at the files sent in the diagnostic collection. These files are supposed to provide sufficient information to decide what action should be taken.

The daily text email contains the following information:

  • time (when the email was generated)

  • cruise status (active? no cruise set?)

  • processing status (is the CODAS database recent?)

  • attitude devices (statistics of accurate heading devices)

  • computer information
    • how long has the computer been running? recently rebooted?
    • NTP time server: found?
  • a link to the figures generated from the data

  • a summary of warnings and file ages

Diagnostic Files

The diagnostic files attempt to provide sufficient information to tell if something is going wrong and what the problem is (or where to look for it).

This table show the file names and the categories of various files.

The most useful files are:

  • status_str.txt : This is the text email summary, described above.

  • DAS_main.txt : recent status (stop logging, start logging, etc)

  • tails.txt : Contains the last 12
    • timestamps and serial messages for each NMEA instrument (and ADCP log)
    • times and sizes of raw logging files
    • times and sizes of rbin files
    • times and sizes of gbin files
  • commands_*.txt : present settings (ADCP commands)

  • cals.txt : ongoing output of watertrack and bottomtrack calibration calculation.

  • ashtech_gyro_pystats.txt : quality of ashtech (similar names for other devices; these files are contained in the text email)


Daily text email: Tutorial

The first page of a text email looks like this:

UHDAS daily email: page 1

The next set of images will step through the parts of the email and how to read them


time (when the email was generated)

UHDAS daily email: page 1, "timestamp"

cruise status (active? no cruise set?)

UHDAS daily email: page 1, "cruise status"

processing status (is the CODAS database recent?)

UHDAS daily email: page 1, "processing status"

attitude devices (statistics of accurate heading devices)

UHDAS daily email: page 1, "accurate heading device statistics"

bottom track

Newer installations say whether bottom track is on or off. Bottom track should be OFF if the bottom is out of range. Keep this in mind when you look at the figures. Are they in deep water with bottom track on?

UHDAS daily email: "bottom track"
computer information
  • how long has the computer been running? recently rebooted?
  • NTP server: found?
UHDAS daily email: page 1, "computer status"
check the link
  • go look at the figure
UHDAS daily email: page 1, "computer status"

checking the WWW figure

UHDAS daily email: example of figures on www

Daily text email: Indications of Trouble

Reset Ashtech

All too common. Everything is fine with the UHDAS system, but the Ashtech has performed badly over the last day. Check the messages to see if it is down. Reset the Ashtech.

UHDAS daily email: Ashtech down.  reset.

Processing stopped: cause unknown

Troubleshooting is required to understand the cause. It is most likely to be a problem with the timestamps. Look in the diagnostic files to see whether all the files are updating as expected. The most likely solution is to start another named cruise segment.

UHDAS daily email: processing stopped; cause unknown

Kernel bug: “restart logging”

In this case, a simple “stop logging”, “start logging” is sufficient

UHDAS daily email: kernel bug