.. _read_netcdf: NetCDF Files ++++++++++++++ NetCDF read from matlab ^^^^^^^^^^^^^^^^^^^^^^^^^^ You can see the live processing directory and access the netCDF files by looking at the shared ``data`` directory and following the example in the next step. (see `this link `_ for information about network shares). To read a CODAS netCDF in matlab, use :download:`load_netcdf.m ` It behaves similarly to ``load_getmat.m``. :: mdata = load_netcdf(filepath) .. NetCDF read in Python ^^^^^^^^^^^^^^^^^^^^^^^^^^ To read a CODAS netCDF file using Python, do something like the following: :: import pycurrents.adcp.panelplotter as pp data = pp.get_data('os38nb.nc') .. The variable names are in ``data.keys()`` and can be accessed in either of two ways, as illustrated with this plotting command: :: import matplotlib.pyplot as plt import pycurrents.adcp.panelplotter as pp ncfilename = 'os38nb.nc' # path to your netcdf file data = pp.get_data(ncfilename) f,ax = plt.subplots(nrows=2, sharex=True) ax[0].plot(data.dday, data.heading, 'g.') ax[1].plot(data['dday'], data['tr_temp'],'r.') plt.show() .. ------ Example NetCDF header from adcp_nc.py: :: codaspy:(os38nb_postproc)-PY3$ ncdump -h contour/os38nb.nc netcdf os38nb { dimensions: time = 532 ; depth_cell = 70 ; variables: int trajectory ; trajectory:standard_name = "trajectory_id" ; double time(time) ; time:long_name = "Decimal day" ; time:units = "days since 2010-01-01 00:00:00" ; time:C_format = "%12.5f" ; time:standard_name = "time" ; time:data_min = 29.7405439814815 ; time:data_max = 31.6571875 ; double lon(time) ; lon:missing_value = 1.e+38 ; lon:long_name = "Longitude" ; lon:units = "degrees_east" ; lon:C_format = "%9.4f" ; lon:standard_name = "longitude" ; lon:data_min = -125.921305555556 ; lon:data_max = -124.399705555556 ; double lat(time) ; lat:missing_value = 1.e+38 ; lat:long_name = "Latitude" ; lat:units = "degrees_north" ; lat:C_format = "%9.4f" ; lat:standard_name = "latitude" ; lat:data_min = 43.3832666666667 ; lat:data_max = 44.5039083333333 ; float depth(time, depth_cell) ; depth:missing_value = 1.e+38f ; depth:long_name = "Depth" ; depth:units = "meter" ; depth:C_format = "%8.2f" ; depth:positive = "down" ; depth:data_min = 46.96f ; depth:data_max = 1703.f ; float u(time, depth_cell) ; u:missing_value = 1.e+38f ; u:long_name = "Zonal velocity component" ; u:units = "meter second-1" ; u:C_format = "%7.2f" ; u:data_min = -0.4183102f ; u:data_max = 1.51168f ; float v(time, depth_cell) ; v:missing_value = 1.e+38f ; v:long_name = "Meridional velocity component" ; v:units = "meter second-1" ; v:C_format = "%7.2f" ; v:data_min = -1.206451f ; v:data_max = 0.6233f ; short amp(time, depth_cell) ; amp:missing_value = 32767s ; amp:long_name = "Received signal strength" ; amp:C_format = "%d" ; amp:data_min = 13s ; amp:data_max = 207s ; byte pg(time, depth_cell) ; pg:missing_value = -1b ; pg:long_name = "Percent good pings" ; pg:C_format = "%d" ; pg:data_min = 0b ; pg:data_max = 100b ; byte pflag(time, depth_cell) ; pflag:long_name = "Editing flags" ; pflag:C_format = "%d" ; pflag:data_min = 0b ; pflag:data_max = 7b ; float heading(time) ; heading:missing_value = 1.e+38f ; heading:long_name = "Ship heading" ; heading:units = "degrees" ; heading:C_format = "%6.1f" ; heading:data_min = -179.9927f ; heading:data_max = 179.9755f ; float tr_temp(time) ; tr_temp:missing_value = 1.e+38f ; tr_temp:long_name = "ADCP transducer temperature" ; tr_temp:units = "Celsius" ; tr_temp:C_format = "%4.1f" ; tr_temp:data_min = 10.72588f ; tr_temp:data_max = 11.63412f ; short num_pings(time) ; num_pings:long_name = "Number of pings averaged per ensemble" ; num_pings:units = "None" ; num_pings:C_format = "%d" ; num_pings:data_min = 41s ; num_pings:data_max = 51s ; float uship(time) ; uship:missing_value = 1.e+38f ; uship:long_name = "Ship zonal velocity component" ; uship:units = "meter second-1" ; uship:C_format = "%9.4f" ; uship:data_min = -6.441127f ; uship:data_max = 6.690955f ; float vship(time) ; vship:missing_value = 1.e+38f ; vship:long_name = "Ship meridional velocity component" ; vship:units = "meter second-1" ; vship:C_format = "%9.4f" ; vship:data_min = -6.283762f ; vship:data_max = 6.624887f ; // global attributes: :featureType = "trajectoryProfile" ; :history = "Created: 2019-03-15 00:53:35 UTC" ; :Conventions = "COARDS" ; :software = "pycurrents" ; :hg_changeset = "2886:51b2d38bdc5a" ; :title = "Shipboard ADCP velocity profiles" ; :description = "Shipboard ADCP velocity profiles from km1001c_demo using instrument os38nb" ; :cruise_id = "km1001c_demo" ; :sonar = "os38nb" ; :yearbase = 2010 ; :ship_name = "Kilo Moana" ; .. and an explanation of CODAS processing in the global attributes:: CODAS_variables = Variables in this CODAS short-form Netcdf file are intended for most end-user scientific analysis and display purposes. For additional information see the CODAS_processing_note global attribute and the attributes of each of the variables. ============= ================================================================= time Time at the end of the ensemble, days from start of year. lon, lat Longitude, Latitude from GPS at the end of the ensemble. u,v Ocean zonal and meridional velocity component profiles. uship, vship Zonal and meridional velocity components of the ship. heading Mean ship heading during the ensemble. depth Bin centers in nominal meters (no sound speed profile correction). tr_temp ADCP transducer temperature. pg Percent Good pings for u, v averaging after editing. pflag Profile Flags based on editing, used to mask u, v. amp Received signal strength in ADCP-specific units; no correction for spreading or attenuation. ============= ================================================================= .. :: CODAS processing note: ====================== Overview -------- The CODAS database is a specialized storage format designed for shipboard ADCP data. "CODAS processing" uses this format to hold averaged shipboard ADCP velocities and other variables, during the stages of data processing. The CODAS database stores velocity profiles relative to the ship as east and north components along with position, ship speed, heading, and other variables. The netCDF *short* form contains ocean velocities relative to earth, time, position, transducer temperature, and ship heading; these are designed to be "ready for immediate use". The netCDF *long* form is just a dump of the entire CODAS database. Some variables are no longer used, and all have names derived from their original CODAS names, dating back to the late 1980's. Post-processing --------------- CODAS post-processing, i.e. that which occurs after the single-ping profiles have been vector-averaged and loaded into the CODAS database, includes editing (using automated algorithms and manual tools), rotation and scaling of the measured velocities, and application of a time-varying heading correction. Additional algorithms developed more recently include translation of the GPS positions to the transducer location, and averaging of ship's speed over the times of valid pings when Percent Good is reduced. Such post-processing is needed prior to submission of "processed ADCP data" to JASADCP or other archives. Full CODAS processing --------------------- Whenever single-ping data have been recorded, full CODAS processing provides the best end product. Full CODAS processing starts with the single-ping velocities in beam coordinates. Based on the transducer orientation relative to the hull, the beam velocities are transformed to horizontal, vertical, and "error velocity" components. Using a reliable heading (typically from the ship's gyro compass), the velocities in ship coordinates are rotated into earth coordinates. Pings are grouped into an "ensemble" (usually 2-5 minutes duration) and undergo a suite of automated editing algorithms (removal of acoustic interference; identification of the bottom; editing based on thresholds; and specialized editing that targets CTD wire interference and "weak, biased profiles". The ensemble of single-ping velocities is then averaged using an iterative reference layer averaging scheme. Each ensemble is approximated as a single function of depth, with a zero-average over a reference layer plus a reference layer velocity for each ping. Adding the average of the single-ping reference layer velocities to the function of depth yields the ensemble-average velocity profile. These averaged profiles, along with ancillary measurements, are written to disk, and subsequently loaded into the CODAS database. Everything after this stage is "post-processing". note (time): ------------ Time is stored in the database using UTC Year, Month, Day, Hour, Minute, Seconds. Floating point time "Decimal Day" is the floating point interval in days since the start of the year, usually the year of the first day of the cruise. note (heading): --------------- CODAS processing uses heading from a reliable device, and (if available) uses a time-dependent correction by an accurate heading device. The reliable heading device is typically a gyro compass (for example, the Bridge gyro). Accurate heading devices can be POSMV, Seapath, Phins, Hydrins, MAHRS, or various Ashtech devices; this varies with the technology of the time. It is always confusing to keep track of the sign of the heading correction. Headings are written degrees, positive clockwise. setting up some variables: X = transducer angle (CONFIG1_heading_bias) positive clockwise (beam 3 angle relative to ship) G = Reliable heading (gyrocompass) A = Accurate heading dh = G - A = time-dependent heading correction (ANCIL2_watrk_hd_misalign) Rotation of the measured velocities into the correct coordinate system amounts to (u+i*v)*(exp(i*theta)) where theta is the sum of the corrected heading and the transducer angle. theta = X + (G - dh) = X + G - dh Watertrack and Bottomtrack calibrations give an indication of the residual angle offset to apply, for example if mean and median of the phase are all 0.5 (then R=0.5). Using the "rotate" command, the value of R is added to "ANCIL2_watrk_hd_misalign". new_dh = dh + R Therefore the total angle used in rotation is new_theta = X + G - dh_new = X + G - (dh + R) = (X - R) + (G - dh) The new estimate of the transducer angle is: X - R ANCIL2_watrk_hd_misalign contains: dh + R ==================================================== .. The **profile flags** indicate bins that have already been determined to contain bad data based on various criteria. Profile flags should always be applied prior to use. A profile flag of 0 means "nothing is flagged", i.e. it should be used, not discarded. :: Profile flags ------------- Profile editing flags are provided for each depth cell: binary decimal below Percent value value bottom Good bin -------+----------+--------+----------+-------+ 000 0 001 1 bad 010 2 bad 011 3 bad bad 100 4 bad 101 5 bad bad 110 6 bad bad 111 7 bad bad bad -------+----------+--------+----------+-------+ ..