3.1.5.12. NetCDF Files¶
3.1.5.12.1. NetCDF read from matlab¶
You can see the live processing directory and access the netCDF files
by looking at the shared data directory and following the example
in the next step. (see this link for
information about network shares).
To read a CODAS netCDF in matlab, use
load_netcdf.m
It behaves similarly to load_getmat.m.
mdata = load_netcdf(filepath)
3.1.5.12.2. NetCDF read in Python¶
To read a CODAS netCDF file using Python, do something like the following:
import pycurrents.adcp.panelplotter as pp
data = pp.get_data('os38nb.nc')
The variable names are in data.keys() and can be accessed
in either of two ways, as illustrated with this plotting
command:
import matplotlib.pyplot as plt
import pycurrents.adcp.panelplotter as pp
ncfilename = 'os38nb.nc' # path to your netcdf file
data = pp.get_data(ncfilename)
f,ax = plt.subplots(nrows=2, sharex=True)
ax[0].plot(data.dday, data.heading, 'g.')
ax[1].plot(data['dday'], data['tr_temp'],'r.')
plt.show()
Example NetCDF header from adcp_nc.py:
codaspy:(os38nb_postproc)-PY3$ ncdump -h contour/os38nb.nc
netcdf os38nb {
dimensions:
time = 532 ;
depth_cell = 70 ;
variables:
int trajectory ;
trajectory:standard_name = "trajectory_id" ;
double time(time) ;
time:long_name = "Decimal day" ;
time:units = "days since 2010-01-01 00:00:00" ;
time:C_format = "%12.5f" ;
time:standard_name = "time" ;
time:data_min = 29.7405439814815 ;
time:data_max = 31.6571875 ;
double lon(time) ;
lon:missing_value = 1.e+38 ;
lon:long_name = "Longitude" ;
lon:units = "degrees_east" ;
lon:C_format = "%9.4f" ;
lon:standard_name = "longitude" ;
lon:data_min = -125.921305555556 ;
lon:data_max = -124.399705555556 ;
double lat(time) ;
lat:missing_value = 1.e+38 ;
lat:long_name = "Latitude" ;
lat:units = "degrees_north" ;
lat:C_format = "%9.4f" ;
lat:standard_name = "latitude" ;
lat:data_min = 43.3832666666667 ;
lat:data_max = 44.5039083333333 ;
float depth(time, depth_cell) ;
depth:missing_value = 1.e+38f ;
depth:long_name = "Depth" ;
depth:units = "meter" ;
depth:C_format = "%8.2f" ;
depth:positive = "down" ;
depth:data_min = 46.96f ;
depth:data_max = 1703.f ;
float u(time, depth_cell) ;
u:missing_value = 1.e+38f ;
u:long_name = "Zonal velocity component" ;
u:units = "meter second-1" ;
u:C_format = "%7.2f" ;
u:data_min = -0.4183102f ;
u:data_max = 1.51168f ;
float v(time, depth_cell) ;
v:missing_value = 1.e+38f ;
v:long_name = "Meridional velocity component" ;
v:units = "meter second-1" ;
v:C_format = "%7.2f" ;
v:data_min = -1.206451f ;
v:data_max = 0.6233f ;
short amp(time, depth_cell) ;
amp:missing_value = 32767s ;
amp:long_name = "Received signal strength" ;
amp:C_format = "%d" ;
amp:data_min = 13s ;
amp:data_max = 207s ;
byte pg(time, depth_cell) ;
pg:missing_value = -1b ;
pg:long_name = "Percent good pings" ;
pg:C_format = "%d" ;
pg:data_min = 0b ;
pg:data_max = 100b ;
byte pflag(time, depth_cell) ;
pflag:long_name = "Editing flags" ;
pflag:C_format = "%d" ;
pflag:data_min = 0b ;
pflag:data_max = 7b ;
float heading(time) ;
heading:missing_value = 1.e+38f ;
heading:long_name = "Ship heading" ;
heading:units = "degrees" ;
heading:C_format = "%6.1f" ;
heading:data_min = -179.9927f ;
heading:data_max = 179.9755f ;
float tr_temp(time) ;
tr_temp:missing_value = 1.e+38f ;
tr_temp:long_name = "ADCP transducer temperature" ;
tr_temp:units = "Celsius" ;
tr_temp:C_format = "%4.1f" ;
tr_temp:data_min = 10.72588f ;
tr_temp:data_max = 11.63412f ;
short num_pings(time) ;
num_pings:long_name = "Number of pings averaged per ensemble" ;
num_pings:units = "None" ;
num_pings:C_format = "%d" ;
num_pings:data_min = 41s ;
num_pings:data_max = 51s ;
float uship(time) ;
uship:missing_value = 1.e+38f ;
uship:long_name = "Ship zonal velocity component" ;
uship:units = "meter second-1" ;
uship:C_format = "%9.4f" ;
uship:data_min = -6.441127f ;
uship:data_max = 6.690955f ;
float vship(time) ;
vship:missing_value = 1.e+38f ;
vship:long_name = "Ship meridional velocity component" ;
vship:units = "meter second-1" ;
vship:C_format = "%9.4f" ;
vship:data_min = -6.283762f ;
vship:data_max = 6.624887f ;
// global attributes:
:featureType = "trajectoryProfile" ;
:history = "Created: 2019-03-15 00:53:35 UTC" ;
:Conventions = "COARDS" ;
:software = "pycurrents" ;
:hg_changeset = "2886:51b2d38bdc5a" ;
:title = "Shipboard ADCP velocity profiles" ;
:description = "Shipboard ADCP velocity profiles from km1001c_demo using instrument os38nb" ;
:cruise_id = "km1001c_demo" ;
:sonar = "os38nb" ;
:yearbase = 2010 ;
:ship_name = "Kilo Moana" ;
and an explanation of CODAS processing in the global attributes:
CODAS_variables =
Variables in this CODAS short-form Netcdf file are intended for most end-user
scientific analysis and display purposes. For additional information see
the CODAS_processing_note global attribute and the attributes of each
of the variables.
============= =================================================================
time Time at the end of the ensemble, days from start of year.
lon, lat Longitude, Latitude from GPS at the end of the ensemble.
u,v Ocean zonal and meridional velocity component profiles.
uship, vship Zonal and meridional velocity components of the ship.
heading Mean ship heading during the ensemble.
depth Bin centers in nominal meters (no sound speed profile correction).
tr_temp ADCP transducer temperature.
pg Percent Good pings for u, v averaging after editing.
pflag Profile Flags based on editing, used to mask u, v.
amp Received signal strength in ADCP-specific units; no correction
for spreading or attenuation.
============= =================================================================
CODAS processing note:
======================
Overview
--------
The CODAS database is a specialized storage format designed for
shipboard ADCP data. "CODAS processing" uses this format to hold
averaged shipboard ADCP velocities and other variables, during the
stages of data processing. The CODAS database stores velocity
profiles relative to the ship as east and north components along with
position, ship speed, heading, and other variables. The netCDF *short*
form contains ocean velocities relative to earth, time, position,
transducer temperature, and ship heading; these are designed to be
"ready for immediate use". The netCDF *long* form is just a dump of
the entire CODAS database. Some variables are no longer used, and all
have names derived from their original CODAS names, dating back to the
late 1980's.
Post-processing
---------------
CODAS post-processing, i.e. that which occurs after the single-ping
profiles have been vector-averaged and loaded into the CODAS database,
includes editing (using automated algorithms and manual tools),
rotation and scaling of the measured velocities, and application of a
time-varying heading correction. Additional algorithms developed more
recently include translation of the GPS positions to the transducer
location, and averaging of ship's speed over the times of valid pings
when Percent Good is reduced. Such post-processing is needed prior to
submission of "processed ADCP data" to JASADCP or other archives.
Full CODAS processing
---------------------
Whenever single-ping data have been recorded, full CODAS processing
provides the best end product.
Full CODAS processing starts with the single-ping velocities in beam
coordinates. Based on the transducer orientation relative to the
hull, the beam velocities are transformed to horizontal, vertical, and
"error velocity" components. Using a reliable heading (typically from
the ship's gyro compass), the velocities in ship coordinates are
rotated into earth coordinates.
Pings are grouped into an "ensemble" (usually 2-5 minutes duration)
and undergo a suite of automated editing algorithms (removal of
acoustic interference; identification of the bottom; editing based on
thresholds; and specialized editing that targets CTD wire interference
and "weak, biased profiles". The ensemble of single-ping velocities
is then averaged using an iterative reference layer averaging scheme.
Each ensemble is approximated as a single function of depth, with a
zero-average over a reference layer plus a reference layer velocity
for each ping. Adding the average of the single-ping reference layer
velocities to the function of depth yields the ensemble-average
velocity profile. These averaged profiles, along with ancillary
measurements, are written to disk, and subsequently loaded into the
CODAS database. Everything after this stage is "post-processing".
note (time):
------------
Time is stored in the database using UTC Year, Month, Day, Hour,
Minute, Seconds. Floating point time "Decimal Day" is the floating
point interval in days since the start of the year, usually the year
of the first day of the cruise.
note (heading):
---------------
CODAS processing uses heading from a reliable device, and (if
available) uses a time-dependent correction by an accurate heading
device. The reliable heading device is typically a gyro compass (for
example, the Bridge gyro). Accurate heading devices can be POSMV,
Seapath, Phins, Hydrins, MAHRS, or various Ashtech devices; this
varies with the technology of the time. It is always confusing to
keep track of the sign of the heading correction. Headings are written
degrees, positive clockwise. setting up some variables:
X = transducer angle (CONFIG1_heading_bias)
positive clockwise (beam 3 angle relative to ship)
G = Reliable heading (gyrocompass)
A = Accurate heading
dh = G - A = time-dependent heading correction (ANCIL2_watrk_hd_misalign)
Rotation of the measured velocities into the correct coordinate system
amounts to (u+i*v)*(exp(i*theta)) where theta is the sum of the
corrected heading and the transducer angle.
theta = X + (G - dh) = X + G - dh
Watertrack and Bottomtrack calibrations give an indication of the
residual angle offset to apply, for example if mean and median of the
phase are all 0.5 (then R=0.5). Using the "rotate" command,
the value of R is added to "ANCIL2_watrk_hd_misalign".
new_dh = dh + R
Therefore the total angle used in rotation is
new_theta = X + G - dh_new
= X + G - (dh + R)
= (X - R) + (G - dh)
The new estimate of the transducer angle is: X - R
ANCIL2_watrk_hd_misalign contains: dh + R
====================================================
The profile flags indicate bins that have already been determined to contain bad data based on various criteria. Profile flags should always be applied prior to use. A profile flag of 0 means “nothing is flagged”, i.e. it should be used, not discarded.
Profile flags
-------------
Profile editing flags are provided for each depth cell:
binary decimal below Percent
value value bottom Good bin
-------+----------+--------+----------+-------+
000 0
001 1 bad
010 2 bad
011 3 bad bad
100 4 bad
101 5 bad bad
110 6 bad bad
111 7 bad bad bad
-------+----------+--------+----------+-------+
