.. _ENR_dbmaker_index: ENR processing with adcp_database_maker.py ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. NOTE:: Processing ENR data requires that **ALL** the original files of a logging series be present for the following suffixes: - ENR, N1R, N2R (if it exists), N3R (if exists) It is also useful to have these for diagnostic purposes: - LTA (or STA), VMO, LOG The **preliminary processing** tools were written to cope with VmDAS data collected with one file prefix, hence it is required that **ascii sorting must be the same as time sorting**. The **preliminary processing** tools were written assuming all files from the VmDAS dataset are present. Any deviations from these requirements may result in problems processing the data. ------ .. NOTE:: **Directory Strategy** To simplify the documentation, these instructions assume you have set up your directories as in :ref:`Directory Setup` and you are doing your processing demos in **my_codas_demos/adcp_pyproc** :: your practice directory source ----------------------- ----------- my_codas_demos my_codas_demos/adcp_pyproc (new, WORKING IN HERE) my_codas_demos/uhdas_data copy of codas_demos/uhdas_data my_codas_demos/vmdas_data copy of codas_demos/vmdas_data my_codas_demos/uhdas_style_data (new, empty, for VmDAS conversion) ---- You should have already done the LTA ``adcp_database_maker.py`` processing. This retrieves all kinds of useful information as well as giving you a first look at the data. Even if you do not post-process the data, it is extremely valuable to take that step. The paradigm of ``adcp_database_maker.py`` is to keep ADCP processing directories in the same place, for easier comparison. -------- Click the image of the **adcp_database_maker.py** overview to enlarge. .. image:: /images/adcp_database_maker/dbmaker-overview.png :alt: adcp_database_maker.py overview :scale: 20 ------ If you just did the LTA processing, just click on **Convert \*.ENR Files**. Otherwise you have to pick the VmDAS data directory and put the cruise name in again. Run ``adcp_database_maker.py`` from your ``adcp_pyproc`` directory (check the :ref:`adcp_database_maker.py overview ` example: :: cd ~/my_codas_demos/adcp_pyproc adcp_database_maker.py .. ------ The ``adcp_database_maker.py`` window will pop up: .. image:: /images/adcp_database_maker/dbmaker_vmdas_launch.png :alt: dbmaker-launch :scale: 70 | ---- Navigate up one directory so you can choose the **vmdas_data** directory .. image:: /images/adcp_database_maker/after-splash_annot.png :alt: vmdas data selection :scale: 70 | ---- Select the **vmdas_data** directory... .. image:: /images/adcp_database_maker/after-splash-vmdas_annot.png :alt: vmdas data selection :scale: 70 | ---- ... and then choose **ps0918** .. image:: /images/adcp_database_maker/select_data_annot.png :alt: vmdas form before filling :scale: 70 | ---- Choose a **Project Directory** (**ps0918_vmdas**) and a cruise name (**ps0918**) for titles and some filenames .. image:: /images/dbmaker_ENR/05-vmdas-to-uhdas-after_annot.png :alt: 05-vmdas-to-uhdas-after_annot :scale: 70 | When you click the button to **convert \*.ENR** files the program will run ``vmdas_info.py`` This will create a file with information discovered by this program. You can also read the files created during the LTA processing (:ref:`Here` is the manual way to get that information in the ENR single-ping processing demo, i.e. manually running commands) | ---- .. NOTE:: The next form discovers information about the vmdas data and converts the VmdAS data to UHDAS-style data. We need to identify the location where the uhdas-style data will go, and specify a cruise name. ----- Click **Browse** to select the location for the uhdas-style data. .. image:: /images/dbmaker_ENR/reform-vmdas-splash_annot.png :alt: vmdas data selection :scale: 70 | Instructions for selecting the uhdas-style data directory follow (2 steps) ---- Navigate up one directory so you are looking at the directories with all the types of data: .. image:: /images/adcp_database_maker/after-splash_annot.png :alt: vmdas data selection :scale: 70 | ---- Pick (or create) **uhdas_style_data**, a directory alongside the other data directories: | .. image:: /images/dbmaker_ENR/06-choose-uhdas-style-data_annot.png :alt: 06-choose-uhdas-style-data_annot :scale: 70 ---- **(1) Left Button: make the conversion files** .. image:: /images/dbmaker_ENR/11-reform-vmdas-before-make-conversion_annot.png :alt: 11-reform-vmdas-before-make-conversion_annot :scale: 60 | (:ref:`Here` is the manual way to make that ``config`` directory and populate it with appropriate information) | ---- **(2) Middle Button: Convert the VmDAS data to UHDAS-style data.** This can be time-consuming if the cruise is over a week long. | .. image:: /images/dbmaker_ENR/12-reform-vmdas-before-convert-to-uhdas_annot.png :alt: 12-reform-vmdas-before-convert-to-uhdas_annot :scale: 60 (:ref:`Here` is the manual way to convert Vmdas data to UHDAS-style data) | ---- **(3) Right Button: Set up Processing Configuration** This takes us to the form where we choose settings that are cruise-dependent (i.e. work for multiple sonars) Next we use that information to convert ("reform") the VmDAS data into UHDAS-style data. The GUI is described :ref:`here `. .. image:: /images/dbmaker_ENR/13-reform-uhdas-before-set-up-proc_cfg_annot.png :alt: 13-reform-uhdas-before-set-up-proc_cfg_annot :scale: 60 | ---- .. NOTE:: The next for sets up the **processing configuration** file. ---- The big difference in this GUI compared to older versions is that we can see the estimates for transducer angle at the bottom. Use that guidance to fill in the clockwise transducer angle. (:ref:`Here` is the manual way to make that ``config`` directory and populate it with appropriate information) .. image:: /images/dbmaker_ENR/44-proc-starter-before-make-config_annot.png :alt: 44-proc-starter-before-make-config_annot :scale: 70 | ---- After you choose your settings, click the next button to **Set up Processing Directories**. The design allows you to save multiple ``*_proc.py`` config files before moving on to do the processing. Normally we would only make one here, and then go to the next form .. image:: /images/dbmaker_ENR/45-proc-starter-after-make-config_annot.png :alt: 45-proc-starter-after-make-config_annot :scale: 60 | ---- **Stage 3: Set up and run Single-ping processing** You can get the ADCP starboard and forward location relative to the position device, if you look at the xducer_dx and xducer_dy offsets from the LTA data. The "Max depth for bottom search" refers to using a topographic dataset to estimate the depth. If the depth is shallower than this, **DO** try to find the bottom in the ADCP data and remove junk below the bottom. Links to the same steps in example 4a are below: (1) Click :ref:`Create Processing Dir` (to create the **processing directory** with adcptree.py). This is fast. .. image:: /images/dbmaker_ENR/50-adcptree-before-procdir_annot.png :alt: 50-adcptree-before-procdir_annot :scale: 70 | (:ref:`Here` is the manual way to make the **CODAS Processing** directory) ---- (2) Click :ref:`Create q_cnt control file` (more processing commands). After you choose your settings (or use the defaults) this is also fast .. image:: /images/dbmaker_ENR/51-before-qpy_cnt_annot.png :alt: 51-before-qpy_cnt_annot :scale: 70 | ---- (3) Click :ref:`Create Codas Database` (to create the :term:`CODAS averages`). This takes longer than the earlier discovery stage, sometimes 15min. .. image:: /images/dbmaker_ENR/52-before-processing_annot.png :alt: 52-before-processing_annot :scale: 60 | ---- You'll know it is done when: - the spinning ball (or clock face) icon goes away - all the buttons are gray - the Log Messages window down below, both tell you to go run dataviewer.py .. image:: /images/dbmaker_ENR/60-after-quickadcp_annot.png :alt: 60-after-quickadcp_annot :scale: 60 Now proceed with the post-processing steps. The next section shows the postprocessing steps for this VmDAS cruise. The UHDAS :ref:`Post-processing Demo` demo is another resource. - get into the command line terminal - check the figures - check the calibration - use dataviewer to look at (and edit) the data - use **dataviewer.py -c** to compare with the LTA data in the previous section | .. include:: /txtfiles/available_demos.txt