LAST CHANGED : 2023/09/14 03:51:59 CRUISE NAME(S) : I05 (rr2308) CRUISE DATES : 2023/07/01 to 2023/09/01 SHIP NAME : Roger Revelle PORTS : Fremantle, Australia to Cape Town, South Africa CHIEF SCIENTIST : Brendan Carter DATABASE NAME : a_rr DATA FILES : rr2023_202_24582.raw to rr2023_256_07200.raw STATUS : to do done ------ ----------- averaged [ X ] loaded [ X ] NOTE: heading correction instrument exists NOTE: time-dependent heading corrections applied IN the ensembles (see cal/rotate/ens_hcorr.ang) check heading correction [ X ] calibration [ X ] edited [ X ] re-check heading correction [ X ] check editing [ X ] figures [ X ] INSTRUMENT : os150 ACQUISITION : PROGRAM : uhdas PROCESSING : python LOGGING : PARAMETERS : BT : bottom track mode (on or off) SI : sampling interval or averaging period for ensemble (sec) NB : number of bins BL : bin length (m) TD : transducer depth (m) BK : blanking length (m) HO : heading offset applied by DAS (deg) HB : heading bias (deg) CRPH : compensation for roll-pitch-heading, 1:on, 0:off) yy/mm/dd hh:mm:ss BT SI NB BL TD BK HO HB CRPH 2023/07/22 06:54:42 off 300 60 8 6 8 41.75 0.00 0001 HEADING : PRIMARY : heading from gyro CORRECTION : heading correction from seapath NOTE: time-dependent heading corrections applied IN the ensembles (see cal/rotate/ens_hcorr.ang) POSITIONS : gps positions from seapath CALIBRATION : (check original processing parameters) additional rotation 0 final transducer angle is: 41.75 applied scale factor 1.007 additional scale factor 1.007 (3) ADCP (dx=starboard, dy=fwd) meters from GPS original: xducer_dx: -1 xducer_dy: 9 correction 0 0 final offset -1 9 COMMENTS : This cruise had to be reprocessed because there was a power loss in the middle of the cruise, resulting in NULL characters at the end of various files. Notes about solving this are at the end of this document. Postprocessing: No gaps in the heading correction exist. A slight adjustment was made to the scale factor. A scattering layer is visible for some sections of the cruise. Overall a very clean cruise with little issues. PROCESSOR : Jamie --- original processing parameters ---------- ## (determined from "sonar"): model = os ## (determined from "sonar"): frequency = 150 ## (determined from "sonar"): instname = os150 ## (determined from "sonar"): pingtype = nb badbeam None beam_order None beamangle 30 configtype python cruisename RR2308 datatype uhdas dbname aship ens_len 300 fixfile aship.gps pgmin 50 pingpref None proc_engine python ref_method refsm refuv_smoothwin 3 refuv_source nav sonar os150nb txy_file aship.agt xducer_dx -1 xducer_dy 9 yearbase 2023 ================================================== POST PROCESSING STEPS: ================================================== -------------------------------------------------------------------------------- ### 1. Check visual oddities -------------------------------------------------------------------------------- # I ran this to look for gaps in the cruise track. plot_nav.py nav/a*.gps figview.py # No gaps exist. # I ran this to check watertrack calibration after patching hcorr. catwt.py catxy.py **watertrack** ----------- Number of edited points: 389 out of 390 median mean std amplitude 1.0070 1.0067 0.0070 phase 0.0050 -0.0195 0.5005 ----------- **transducer-gps offset** ----------- guessing ADCP (dx=starboard, dy=fwd) meters from GPS positions from aship.agt calculation done at 2024/02/29 21:34:50 xducer_dx = 0.126196 xducer_dy = -0.387473 signal = 3591.897555 ----------- # The xducer dy is just on the cusp of needing an adjsutment. # I will adjust the scale factor. -------------------------------------------------------------------------------- ### 2. Running the calibrations -------------------------------------------------------------------------------- quick_adcp.py --steps2rerun rotate:apply_edit:navsteps:calib --rotate_amplitude 1.007 --auto # I recheck the calibrations. catwt.py **watertrack** ----------- Number of edited points: 388 out of 390 median mean std amplitude 1.0000 0.9999 0.0067 phase 0.0035 -0.0259 0.4856 ----------- -------------------------------------------------------------------------------- ### 3. Editing points -------------------------------------------------------------------------------- # I ran this to go through the dataset and edit out bad values. dataviewer.py -e # I ran this to recompute the calibration residuals. quick_adcp.py --steps2rerun navsteps:calib --auto # I ran this: catwt.py catxy.py **transducer-gps offset** ----------- guessing ADCP (dx=starboard, dy=fwd) meters from GPS positions from aship.agt calculation done at 2024/03/05 22:38:28 xducer_dx = 0.072018 xducer_dy = -0.480521 signal = 3591.897555 ----------- **watertrack** ----------- Number of edited points: 388 out of 389 median mean std amplitude 1.0000 0.9997 0.0066 phase 0.0000 -0.0256 0.4858 ----------- -------------------------------------------------------------------------------- ### 4. Re-check heading correction and other figures -------------------------------------------------------------------------------- # I ran this to see if there was anything strange with the figures. figview.py # Yep looks okay to me. -------------------------------------------------------------------------------- ### 5. Check edited, calibrated dataset against original dataset -------------------------------------------------------------------------------- # There are no other sonars to compare this to. -------------------------------------------------------------------------------- ### 6. Make plots and files -------------------------------------------------------------------------------- quick_web.py --interactive # I ran this to extract matlab files. quick_adcp.py --steps2rerun matfiles --auto # I ran this to extract a netCDF file. adcp_nc.py adcpdb contour/os150nb RR2308 os150nb --ship_name "Roger Revelle" #---- final processing parameters ------------------------------------------- # # written 2024/02/29 21:14:03 # this file contains CODAS quick_adcp.py info #name, value pairs #-------------------- ## (determined from "sonar"): model = os ## (determined from "sonar"): frequency = 150 ## (determined from "sonar"): instname = os150 ## (determined from "sonar"): pingtype = nb badbeam None beam_order None beamangle 30 configtype python cruisename RR2308 datatype uhdas dbname aship ens_len 300 fixfile aship.gps pgmin 50 pingpref None proc_engine python ref_method refsm refuv_smoothwin 3 refuv_source uvship sonar os150nb txy_file aship.agt xducer_dx -1 xducer_dy 9 yearbase 2023 #======================================================= NOTES about removing nulls from files (caused by hard power-cycle) This cruise had - 2 hard resets (dday 212 and dday 215) requiring manual trimming of files - gyro gap of 6000 seconds (dday 212) - phins stepped backwards 26000 seconds (briefly) - abxtwo went out and phins had a 0.5-day gap (dday 240) - abxtwo stepped backwards by 30seconds - abxtwo had no headings (python3) [jules@joules revelle]$ uhdas_powerloss_adcp.py --uhdas_dir RR2308 2024/02/29 09:15:26 testing /Users/jules/data/uhdas_data_archive/evaluate/revelle/RR2308 Loss of power during logging results in NULLs at the end of the .log file and zeros at the end of the .log.bin file. There is also junk at the end of the corresponding raw data file. filename, python index, number of failed lines RR2308/raw/os150/rr2023_212_00000.raw.log.bin, last good line = 4836, num FAILED 23 RR2308/raw/os150/rr2023_215_14400.raw.log.bin, last good line = 4848, num FAILED 4 To fix the processing, one needs to - COPY THE ORIGINAL DATA FOR SAFEKEEPING We will be altering those files in the data directory. (1) identify the last good line in the .log file (printed below - these are PYTHON indexes) (2) remove the NULLS that follow. If 1000 is printed, use "head -1001" (edit, or use "head"). replace the original log file with a clean one) (3) remake the .log.bin file: (make_rawlogbin.py ) (4) similarly, make a new .raw file with the bad parts removed: (cut_raw_adcp.py os ukdy2021_137_72000.raw 0 1000 ukdy2021_137_72000A.raw; mv ukdy2021_137_72000A.raw ukdy2021_137_72000.raw) (5) reprocess the data with the new files (python3) [jules@joules revelle]$ pwd /Users/jules/data/uhdas_data_archive/evaluate/revelle ================================================================== Look at the .log files: (python3) [jules@joules RR2308]$ tail ./raw/os150/rr2023_212_00000.raw.log 212.0682883 5648760 1170 695983 -19358.0000054 174.3978267 2023 212.0683024 5649930 1170 695984 -19358.0000030 174.3978408 2023 212.0683165 5651100 1170 695985 -19358.0000005 174.3978548 2023 212.0683305 5652270 1170 695986 -19358.0000096 174.3978689 2023 212.0683446 5653440 1170 695987 -19358.0000071 174.3978829 2023 212.0683586 5654610 1170 695988 -19358.0000046 174.3978970 2023 212.0683727 5655780 1170 695989 -19358.0000022 174.3979111 2023 212.0683868 5656950 1170 695990 -19358.0000112 174.3979251 2023 212.0684008 5658120 1170 695991 -19358.0000088 174.3979392 2023 (python3) [jules@joules RR2308]$ od -c ./raw/os150/rr2023_212_00000.raw.log | tail 1132460 0 0 1 1 2 1 7 4 . 3 9 7 9 2 5 1132500 1 2 0 2 3 \n 2 1 2 . 0 6 8 4 0 1132520 0 8 5 6 5 8 1 2 0 1 1 7 0 1132540 6 9 5 9 9 1 - 1 9 3 5 8 . 0 0 1132560 0 0 0 8 8 1 7 4 . 3 9 7 9 3 9 1132600 2 2 0 2 3 \n \0 \0 \0 \0 \0 \0 \0 \0 \0 1132620 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 1135500 \0 \0 \0 \0 \0 \0 \0 1135507 (python3) [jules@joules RR2308]$ od -c ./raw/os150/rr2023_215_14400.raw.log | tail 1111120 0 0 0 0 1 1 3 3 . 1 6 5 6 5 9 1111140 1 2 0 2 3 \n 2 1 5 . 2 3 4 9 4 1111160 2 7 5 6 7 2 1 6 0 1 1 7 0 1111200 2 1 1 3 3 9 - 1 9 3 5 8 . 0 0 1111220 0 0 0 8 8 3 . 1 6 5 6 7 3 2 1111240 2 0 2 3 \n \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 1111260 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 1111620 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 1111635 ============================================ (1) identify the last good line in the .log file (printed below - these are PYTHON indexes) (2) remove the NULLS that follow. If 1000 is printed, use "head -1001" (edit, or use "head"). replace the original log file with a clean one) RR2308/raw/os150/rr2023_212_00000.raw.log.bin, last good line = 4836, num FAILED 23 RR2308/raw/os150/rr2023_215_14400.raw.log.bin, last good line = 4848, num FAILED 4 # will do this: cd RR2308/raw/os150 mv rr2023_212_00000.raw.log busted_rr2023_212_00000.raw.log head -4837 busted_rr2023_212_00000.raw.log | od -c | tail #yes, proper content head -4837 busted_rr2023_212_00000.raw.log > rr2023_212_00000.raw.log ## do the other file mv rr2023_215_14400.raw.log busted_rr2023_215_14400.raw.log head -4849 busted_rr2023_215_14400.raw.log | od -c | tail #yes, proper content head -4849 busted_rr2023_215_14400.raw.log > rr2023_215_14400.raw.log (3) remake the .log.bin file: # remove those 2 files first \rm rr2023_212_00000.raw.log.bin rr2023_215_14400.raw.log.bin make_rawlogbin.py rr2023_212_00000.raw.log rr2023_215_14400.raw.log (4) similarly, make a new .raw file with the bad parts removed: (cut_raw_adcp.py os ukdy2021_137_72000.raw 0 1000 ukdy2021_137_72000A.raw; mv ukdy2021_137_72000A.raw ukdy2021_137_72000.raw) In our case: mv rr2023_212_00000.raw busted_rr2023_212_00000.raw cut_raw_adcp.py os busted_rr2023_212_00000.raw 0 4836 rr2023_212_00000.raw mv rr2023_215_14400.raw busted_rr2023_215_14400.raw cut_raw_adcp.py os busted_rr2023_215_14400.raw 0 4848 rr2023_215_14400.raw #======================================================= Now we do the other raw files od -c gyro/rr2023_212_00000.hdg | tail od -c phins/rr2023_212_00000.gps | tail od -c seapath/rr2023_212_00000.sea | tail od -c abxtwo/rr2023_212_00000.adu | tail od -c gyro/rr2023_215_14400.hdg | tail od -c phins/rr2023_215_14400.gps | tail od -c seapath/rr2023_215_14400.sea | tail od -c abxtwo/rr2023_215_14400.adu | tail (1) Edit these to remove nulls (2) remake the rbins we are in the RR2308/raw directory # gyro:hdg asc2bin.py -y 2023 --redo -v -m hdg -o ../rbin/gyro gyro/rr2023_212_00000.hdg gyro/rr2023_215_14400.hdg # abxtwo:adu # abxtwo:gps asc2bin.py -y 2023 --redo -v -m adu -o ../rbin/abxtwo abxtwo/rr2023_212_00000.adu abxtwo/rr2023_215_14400.adu asc2bin.py -y 2023 --redo -v -m gps -o ../rbin/abxtwo abxtwo/rr2023_212_00000.adu abxtwo/rr2023_215_14400.adu # phins:gps # phins:pmv asc2bin.py -y 2023 --redo -v -m pmv -o ../rbin/phins phins/rr2023_212_00000.gps phins/rr2023_215_14400.gps asc2bin.py -y 2023 --redo -v -m gps -o ../rbin/phins phins/rr2023_212_00000.gps phins/rr2023_215_14400.gps # seapath:gps # seapath:sea asc2bin.py -y 2023 --redo -v -m sea -o ../rbin/seapath seapath/rr2023_212_00000.sea seapath/rr2023_215_14400.sea asc2bin.py -y 2023 --redo -v -m gps -o ../rbin/seapath seapath/rr2023_212_00000.sea seapath/rr2023_215_14400.sea (python3) [jules@joules RR2308_JH]$