2.7.3.8. Regenerating UHDAS data components

Depending on the reason for single-ping processing, one may need to regenerate some of UHDAS intermediate files. UHDAS data have a specific directory structure. A description of the data directories is available in the CODAS+UHDAS At Sea section, on the Cruise directory contents page.

2.7.3.8.1. UHDAS Serial Inputs

UHDAS data files have a specific directory and filename convention. During UHDAS data acquisition the serial data are timestamped and written to disk, and translated into a simple binary format. VmDAS ascii serial fiels (N1R, N2R, N3R) can also be translated into this binary format.

  1. RAW

Serial data are logged into the raw directory

  • each serial port gets its own subdirectory

  • the file suffix is an indicator of the contents

  • every line that is logged is timestamped by the computer

Example (for a POSMV): raw/posmv/km2011_202_07200.pmv

These data are timestamped and recorded. They are the building blocks; they cannot be “remade”.

  1. RBIN

Ascii serial data are translated into binary, in the rbin directory. Each ascii serial stream (eg POSMV, GPS,…) and each configured message inside it, are translated into a binary files which can be read (or written) by Python or Matlab.

For UHDAS data, it is usually unnecessary to remake rbins. Try processing the data without remaking rbins and if there are unexplained gaps, start over at the rbin stage. This is the most efficient strategy (if you save all your notes!). For VmDAS data, rbin generation is a normal part of the procedure and instructions include this at the right stage.

Reasons to remake rbins

  • to be thorough

    • modern, batch mode rbin generation might fill gaps in a very old cruise

  • cruise was broken into several legs (segments)

    • advisable: sometimes multiple segments means missing rbins

  • rbins do not exist, eg:

    • VmDAS ENR processing

    • a serial message was logged but not parsed

  • there was a power outage during the cruise that interrupted translation

In order to remake rbins, the parsing program needs to be told what kind of NMEA message is present. The program which does this is asc2bin.py. Read the help for this progran by running asc2bin.py --hel or ask the UHDAS team if you need help.

Files are stored in a directory structure parallel to raw:

  • each serial port gets its own subdirectory (parallel to raw)

  • the NMEA message determines the file suffix

Example (for a POSMV):

  • rbin/posmv/km2011_202_07200.pmv.rbin (for $PASHR message)

  • rbin/posmv/km2011_202_07200.gps.rbin (for $GPGGA message)

UHDAS file format

km = ship letters
2011 = year of data collection
202 = zero-based integer day of year
07200 = zero-based seconds of day
pmv 'message' (see below)
rbin = suffix for rbin data

RBIN message name

There is only one rbin message name for a given NMA string. Typical UHDAS directory names are used below. Given the NMEA message (left) and the presence or absence of a checksum, the message name is fixed.

NMEA string

typical UHDAS directory name

checksum?

message name

$GPGGA

gpsnav

yes

gps

$INGGA

gpsnav

yes

gps

$GPGGA

gpsnav

no

ggn

$INGGA

gpsnav

no

ggn

$HEHDT

gyro

yes

hdg

$HEHDG

gyro

yes

hdg

$HEHDT

gyro

no

hnc

$HEHDG

gyro

no

hnc

$PASHR,ATT

ashtech

yes

adu

$PASHR,AT2

ashtech

yes

at2

$PASHR,PAT

ashtech

yes

pat

$GPPAT

ashtech

yes

paq

$PASHR

posmv

yes

pmv

$PRDID

heading

yes

rdi

$PSXN,20+$PSXN,23

seapath

yes

sea

$GPGGA+$PSXN,20

seapath

yes

gps_sea

$PSAT,HPR

csi

yes

psathpr

: (tss1)

mahrs

yes

tss1

: (tss1)

mahrs

no

hnc_tss1

CODAS Python processing must be configured so it can find appropriate instrument+message pairs for position, reliable heading, and a heading correction device (if present).

2.7.3.8.1.1. GBIN

For UHDAS data, it is prudent to remake the gbins. It doesn’t take that long, it is often necessary, and unless you know your gbins are exactly what you want, it is better to go ahead and remake them.

Reasons for remaking gbins

  • gbins were produced by Matlab processing: they do not work with Python processing

  • the cruise was broken into several legs (segments); rejoin the raw and rbin segments and make new gbins

  • you are reprocessing the data with a different serial device (eg. different GPS feed

  • gbins do not exist (eg. processing ‘reformed’ VmDAS data)