Regenerating UHDAS data components +++++++++++++++++++++++++++++++++++ .. _regenerate_uhdas: Depending on the reason for single-ping processing, one may need to regenerate some of UHDAS intermediate files. UHDAS data have a specific directory structure. A description of the data directories is available in the **CODAS+UHDAS At Sea** section, on the :ref:`Cruise directory contents ` page. UHDAS Serial Inputs ~~~~~~~~~~~~~~~~~~~~~ UHDAS data files have a specific directory and filename convention. During UHDAS data acquisition the serial data are timestamped and written to disk, and translated into a simple binary format. VmDAS ascii serial fiels (``N1R``, ``N2R``, ``N3R``) can also be translated into this binary format. (1) **RAW** Serial data are logged into the ``raw`` directory - each serial port gets its own subdirectory - the file suffix is an indicator of the contents - every line that is logged is timestamped by the computer Example (for a POSMV): ``raw/posmv/km2011_202_07200.pmv`` These data are timestamped and recorded. They are the building blocks; they cannot be "remade". (2) **RBIN** Ascii serial data are translated into binary, in the ``rbin`` directory. Each ascii serial stream (eg POSMV, GPS,...) and each configured message inside it, are translated into a binary files which can be read (or written) by Python or Matlab. For UHDAS data, it is usually unnecessary to remake rbins. Try processing the data without remaking rbins and if there are unexplained gaps, start over at the rbin stage. This is the most efficient strategy (if you save all your notes!). For VmDAS data, rbin generation is a normal part of the procedure and instructions include this at the right stage. **Reasons to remake rbins** - to be thorough + modern, batch mode rbin generation might fill gaps in a very old cruise - cruise was broken into several legs (segments) + advisable: sometimes multiple segments means missing rbins - rbins do not exist, eg: + VmDAS ENR processing + a serial message was logged but not parsed - there was a power outage during the cruise that interrupted translation In order to remake rbins, the parsing program needs to be told what kind of NMEA message is present. The program which does this is ``asc2bin.py``. Read the help for this progran by running ``asc2bin.py --hel`` or ask the UHDAS team if you need help. Files are stored in a directory structure parallel to ``raw``: - each serial port gets its own subdirectory (parallel to ``raw``) - the NMEA message determines the file suffix Example (for a POSMV): - ``rbin/posmv/km2011_202_07200.pmv.rbin`` (for $PASHR message) - ``rbin/posmv/km2011_202_07200.gps.rbin`` (for $GPGGA message) UHDAS file format :: km = ship letters 2011 = year of data collection 202 = zero-based integer day of year 07200 = zero-based seconds of day pmv 'message' (see below) rbin = suffix for rbin data **RBIN message name** There is only one rbin message name for a given NMA string. Typical UHDAS directory names are used below. Given the NMEA message (left) and the presence or absence of a checksum, the message name is fixed. +--------------------+-----------+-----------+------------+ | NMEA string | typical | checksum? | message | | | UHDAS | | name | | | directory | | | | | name | | | +====================+===========+===========+============+ | $GPGGA | gpsnav | yes | gps | +--------------------+-----------+-----------+------------+ | $INGGA | gpsnav | yes | gps | +--------------------+-----------+-----------+------------+ | $GPGGA | gpsnav | no | ggn | +--------------------+-----------+-----------+------------+ | $INGGA | gpsnav | no | ggn | +--------------------+-----------+-----------+------------+ | $HEHDT | gyro | yes | hdg | +--------------------+-----------+-----------+------------+ | $HEHDG | gyro | yes | hdg | +--------------------+-----------+-----------+------------+ | $HEHDT | gyro | no | hnc | +--------------------+-----------+-----------+------------+ | $HEHDG | gyro | no | hnc | +--------------------+-----------+-----------+------------+ | $PASHR,ATT | ashtech | yes | adu | +--------------------+-----------+-----------+------------+ | $PASHR,AT2 | ashtech | yes | at2 | +--------------------+-----------+-----------+------------+ | $PASHR,PAT | ashtech | yes | pat | +--------------------+-----------+-----------+------------+ | $GPPAT | ashtech | yes | paq | +--------------------+-----------+-----------+------------+ | $PASHR | posmv | yes | pmv | +--------------------+-----------+-----------+------------+ | $PRDID | heading | yes | rdi | +--------------------+-----------+-----------+------------+ | $PSXN,20+$PSXN,23 | seapath | yes | sea | +--------------------+-----------+-----------+------------+ | $GPGGA+$PSXN,20 | seapath | yes | gps_sea | +--------------------+-----------+-----------+------------+ | $PSAT,HPR | csi | yes | psathpr | +--------------------+-----------+-----------+------------+ | : (tss1) | mahrs | yes | tss1 | +--------------------+-----------+-----------+------------+ | : (tss1) | mahrs | no | hnc_tss1 | +--------------------+-----------+-----------+------------+ CODAS Python processing must be configured so it can find appropriate instrument+message pairs for position, reliable heading, and a heading correction device (if present). **GBIN** ^^^^^^^^^^^^ For UHDAS data, it is prudent to remake the gbins. It doesn't take that long, it is often necessary, and unless you *know* your gbins are exactly what you want, it is better to go ahead and remake them. **Reasons for remaking gbins** - gbins were produced by Matlab processing: they do not work with Python processing - the cruise was broken into several legs (segments); rejoin the raw and rbin segments and make new gbins - you are reprocessing the data with a different serial device (eg. different GPS feed - gbins do not exist (eg. processing 'reformed' VmDAS data) .. toctree:: :hidden: