Matlab requirements
You must have UH Currents group matlab code installed and
the matlab path configured. Read the relevant parts of
this page to find the
and download the matlab code ("matlab.zip"), and set up the path.
Reading raw data
Programs exist to read raw data from data files collected by various
logging programs, sometimes more than one program can log data for a
given instrument. The binary data format differs between instruments;
the filenames vary between logging programs. The "class" (below)
indicates the class of instrument (specifically the kind of binary
data stored). There is one read program for each class of
instruments: specifying the instrument class is necessary during the
read process so matlab will know which "read" program to run. These
read programs only work with instruments that have 4 beams (regardless
of how many are working):
class instrument self-contained underway
name (lowered, (requires a data
moored) acquisition program)
----- ---------- ---------- ------------------------------
bb bb adcp yes yes: VmDAS (ENR, ENS, ENX, STA, LTA
yes: Transect (*.r files only)
bb workhorse yes yes: VmDAS (ENR, ENS, ENX, STA, LTA
os ocean surveyor NA yes: VmDAS (ENR, ENS, ENX, STA, LTA
UHDAS (*.raw)
hr HDSS NA yes: HDSS (ugly file name)
son Sontek yes NA
The "read" function returns a data structure with fields present for
each beam in a three-dimensional array. Type "help nb/read", "help
bb/read", (etc) to see the options for the program. In addition to
required options (instrument class, file name, and yearbase) you can
specify the data variables to output (eg. vel is velocity, ts is
time series (heading, temperature,…)) and the indices to extract.
Note that not all fields are available in all data sets.
There are two types of single-ping data:
-
from the instrument: (eg. moored, lowered) or for shipboard
ADCP, the ENR files (from VmDAS) or "raw" files (from UHDAS).
These data have no positions, the instrument clock sets the time
stamp, and if heading is present it probably came from an internal
flux gate compass and/or tilt sensors.
-
"fully navigated": These have passed through a data acqusition system
and additional information has been added (position, attitude,
and corrected time stamp)
NOTE that the data collected may be in earth or beam coordinates, depending
on the way the instrument was set up. Fully navigated shiboard data are
in earth coordinatees.
Data present in the table above have the following fields:
"RAW" (from the
instrument) lon coord time pings per heading
lat system base ensemble (pitch,roll)
------------------+-----+---------+-----------+-----------+-------------
moored or lowered no note(1) instrument note(1) internal sensor
shipboard:
'raw'
VmDAS ENR no beam instrument 1 see note(2)
UHDAS raw no beam instrument 1 see note(2)
shipboard:
partially
navigated
VmDAS ENS yes beam UTC 1 see note(2)
shipboard:
fully
navigated
VmDAS ENX yes earth UTC 1 as specified
VmDAS STA yes earth UTC note(2) as specified
VmDAS LTA yes earth UTC note(2) as specified
UHDAS raw yes earth UTC 1 as specified
(see note(3))
HDSS uglyname yes earth UTC 1 as specified
(see note(4))
NOTES:
(1) Moored or lowered are usually collected in
earthcoordinates, but the transformation matrices are
available in this suite of programs to switch from
beam to earth coordinates given heading.
(2) Raw RDI (un-navigated) shipboard ADCP data will only
have a heading if it is put into the deck unit using
the synchro input. You cannot assume this is being
used: heading usually comes in a serial input and is
not available until the "fully navigated" stage.
(3) UHDAS raw data files are un-navigated, and are stored
along with the raw serial inputs (gzipped files with
GGA messages, etc). Two steps are needed to create
ancillary navigation files, at which point a simple
tool (in this sofware suite) can be used to attach
the navigation UTC time, and attitude to the UHDAS
files.
(4) HDSS data routines are experimental: newly written,
and unsupported. HDSS files are fully navigated, but
badly documented. If you must deal with HDSS files,
check the "HDSS blog" (currents.soest.hawaii.edu) for
more information, caveats, warnings, pitfalls, and
experience. It will probably be painful.
You must run "adcppath" and then "radcppath" to have access
to the programs that read raw data from RDI instruments
-
DO NOT add any subdirectories under 'rawadcp" other than those
provided by this mechanism, or you will be running obsolete code.
-
NEVER name a variable nb, bb, or os, since this will
override the class definition and you will be unable to use that read
function until you clear the variable.
-
Hint: start by reading a subsample of the data so you know how much
memory you will be using, and how long it will take to read the file.
-
Hint: see also cutadcp and catadcp for crude tools to subsample and
concatenate data structures. These work best with data structures
obtained using read before using restruct. See below for details
and use matlab command-line help for these commands
Examples:
(1a) UHDAS Ocean surveyor — Read all variables, first 300 values
(1b) UHDAS Ocean surveyor
(1c) UHDAS Ocean Surveyor
(1d) UHDAS Ocean surveyor
-
Read all variables, every 40th point
-
FULLY NAVIGATED data (requires the three config files used during
data acquisition and processing)
-------------------------
>> cfg = get_uhdascfg('cfgbase','lmg0501', ...
'cfgpath', 'lmg0501/raw/config',...
'instname', 'os38',...
'pingtype', 'bb',...
'logfpath', '.');
>> osdata = getsome_adcp(cfg, 'ddrange', [364 364.1])
-------------------------
(2a) VmDAS data (this scheme works for all VmDAS data files)
-------------------------
>> filename = 'KMADCP093_000000.LTA'; % this is fully navigated, averaged data
>> osdata = read(os, filename, 'vars', 'all');
-------------------------
(2b) VmDAS data — concatenating files:
-------------------------
>> filename1 = 'KMADCP093_000000.LTA';
>> filename2 = 'KMADCP093_000001.LTA';
>> filename3 = 'KMADCP093_000002.LTA';
>> osdata1 = read(os, filename1, 'vars', 'all');
>> osdata2 = read(os, filename2, 'vars', 'all');
>> osdata = catadcp(osdata1, osdata2);
>> osdata = catadcp(osdata, read(os, filename3, 'vars', 'all');
>> dd = restruct(os, osdata); % make the variables 2-dimensional; add some
-------------------------
(3) HDSS data (see HDSS blog for caveats).
-
There are two HDSS instruments, and you have to specify which one
(to get the right transformation matrices) and which beams to
use for the transformation (you never know which ones might
work for your dataset: defaults are :
beams [1 2 3 4] for HR140
beams [1 2 3 4] for HR50
-
This example is from a time when only two beams of the 50kHz
instrument were working
-------------------------
>> filename = '50K_ZHNG02RR[12_01_05.185500]';
>> hrdata = read(hr, filename','yearbase',2005,...
'log_yearbase', 2003,...
'lonsign', -1,...
'beams2do', [3 4],...
'instname', 'hr50);
-------------------------
To get raw data with U, V, W, and error velocity you need to transform
the velocities from beam coordinates to earth coordinates. For this,
you must know the angle of the transducer relative to the hull. This
usually comes as a rough guess of the angle, which is used during
aquisition, and a calibration value, which is determined after
processing. In this example, the heading offset used during
aquisition was 45deg, and the "phase" from earlier codas calibrations
was 0.55deg. The geometry varies with the instrument, but is
usually convex for the OS and concave for NB and BB instruments.
-------------------------
OH = 45.0 + .55; %Mounting Angle + phase
hdg = data.heading;
[data.enu data.xyze] = beam_enu(data.vel, OH + hdg, 0*hdg, 0*hdg', ...
'geometry','convex');
-------------------------
Sometimes if is nice to have all four beams available without having
to type squeeze. The function "restruct" returns variables starting
as [NBINS x NBEAMS x NPROFS] in four separate variables, each [NBINS x
NPROFS]. Restruct requires the instrument class as the first
argument. (The optional output structure config holds title
information used for plotting)
-------------------------
[dd, config] = restruct(bb, data); % "config" is used for plotting; see below
eg: data.vel: [50x4x50 double] (from original read)
turns into
dd.vel1: [50x50 double] (after restruct)
dd.vel2: [50x50 double]
dd.vel3: [50x50 double]
dd.vel4: [50x50 double]
-------------------------
"restruct" is most useful with fully navigated data.
From Example (1b) above:
------------------------------------------
dd= (same as original "data", plus some)
dday: [1x1200 double] original instrument decimal day
ens_num: [1x1200 double] profile number
cor1: [80x1200 double] \ correlation, beam 1 (for a NB, the
cor2: [80x1200 double] \ correlation, beam 2 field would be
cor3: [80x1200 double] / correlation, beam 3 "spectral width"
cor4: [80x1200 double] / correlation, beam 4 not correlation
amp1: [80x1200 double] \ signal/noise, beam 1
amp2: [80x1200 double] \ signal/noise, beam 2
amp3: [80x1200 double] / signal/noise, beam 3
amp4: [80x1200 double] / signal/noise, beam 4
pg1: [80x1200 double] \ flag: 0 or 1, beam 1
pg2: [80x1200 double] \ flag: 0 or 1, beam 2
pg3: [80x1200 double] / flag: 0 or 1, beam 3
pg4: [80x1200 double] / flag: 0 or 1, beam 4
vel1: [80x1200 double] \ if ORIGINALLY beam coords, these are
vel2: [80x1200 double] measured velocities for the 4 beams
vel3: [80x1200 double] if ORIGINALLY in earth coords, (eg. LTA)
vel4: [80x1200 double] / meas vel in [east, north , up, errvel]
xyze1: [80x1200 double] \ measured velocities in ship coords:
xyze2: [80x1200 double] \ starboard, forward, up, errvel
xyze3: [80x1200 double] /
xyze4: [80x1200 double] /
enu1: [80x1200 double] \ measured velocoties in earth coordinates
enu2: [80x1200 double] \ east,north,up,errvel
enu3: [80x1200 double] /
enu4: [80x1200 double] /
uship: [1x1200 double] ship's speed from navigation
vship: [1x1200 double] uses 'll2mps'
umeas: [80x1200 double] \ same as enu1: m/s
vmeas: [80x1200 double] \ same as enu2: m/s
wmeas: [80x1200 double] / same as enu3: m/s
emeas: [80x1200 double] / same as enu4: m/s
fmeas: [80x1200 double] measured velocity aligned with ship heading
pmeas: [80x1200 double] measured velocity aligned with port dir
corr_dday: [1x1200 double] --> UTC time
heading: [1x1200 double] heading from some source (depends on )
pitch: [1x1200 double] pitch from some source (data type and)
roll: [1x1200 double] roll from some source (acquisition )
temperature: [1x1200 double] temperature from thermistor
soundspeed: [1x1200 double] soundspeed used (instrument-dependent)
nav: [1x1 struct] navigation structure, UTC time, lon, lat
mps: [1x1200 double] ship speed (from ll2mps)
cog: [1x1200 double] course over ground
lon: [1x1200 double] lon from source
lat: [1x1200 double] lat from source
uabs: [80x1200 double] ocean u: umeas + uship (no corrections!!)
vabs: [80x1200 double] ocean v: vmeas + vship (no corrections!!)
fabs: [80x1200 double] ocean vel in direction of ship's heading
bt: [1x1 struct] bottom track data
bins: [80x1 double] bin count (positive away from instrument)
depth: [80x1 double] nominal depth (meters) away from instr
config: [1x1 struct] FULL OF USEFUL INFO, eg: bin, pulse...
------------------------------------------
After you load the data and restructure it, you can plot any 2-dimensional
parameter with "aplotit", for example:
----------------------
aplotit(data,config,'fn','amp1');
or
aplotit(data,config,'fn','amp1','cur_ax', subplot(211));
aplotit(data,config,'fn','cor1','cur_ax', subplot(212));
----------------------