Installing a Python working environment with UH software

These instructions are specific to installing on a reasonably recent Mac, but most of the steps are similar on other machines. Some notes about Linux are included. For simplicity, the instructions are based on using the Anaconda software distribution, which provides platform-independent package management for Python and other software in self-contained user-specific environments. This is highly recommended, and will be required for this course.

The instructions also assume that you are installing most things from scratch; if you have remnants of earlier installations or attempts, conflicts could occur, and these can be very confusing and difficult to track down.

Install Anaconda base

To begin, install the current Anaconda base that matches your operating system. You will need to enter your email address. (Don’t worry, this does not trigger an avalanche of spam.) Choose Python 3. On a Mac you can use either a graphical installer or a command-line installer; only the latter is available for Linux. Keep the default location for installation, which will be the anaconda3 subdirectory in your home directory. If the graphical installer shows a message saying that you cannot install in this location, chances are it is a bug in the installer. You can work around it by clicking on the option below and then on the original default. After shaking it this way it will work. (The other option is a system-wide installation, but for our purposes this has no advantages and makes package management more complicated.) When the installation has finished, you will have an extensive Python-based suite of software.

The installer will have put the anaconda3/bin subdirectory at the head of your PATH environment variable in your .bash_profile or in a related hidden shell configuration file. When you quit the terminal and then start a new terminal session, it should therefore find the Anaconda versions of python, ipython, etc. To verify this, execute:

which python
which ipython
python --version

This last command should show something like:

Python 3.5.2 :: Anaconda 4.2.0 (x86_64)

Install additional Anaconda packages

One of the new commands that will be found in anaconda3/bin is conda, which is the package management tool. Packages can be installed from the default location or from alternatives. The primary alternative for our purposes is called conda-forge, and it often has better versions of packages than the default location. Therefore I recommend that you configure conda to give it priority by adding a file to your home directory. It must be called .condarc and contain the following three lines:

channels:
    - conda-forge
    - defaults

Now you are ready to install additional packages that we will need, using the following commands:

conda install basemap
conda install netcdf4
conda install future

Install Mercurial

Install the Mercurial package directly from its web site; it will work with the Python 2 version that is included with OS X. (In general, it is best not to install packages into the OS X Python; this is an exception.)

If you are on Linux, Mercurial might already be installed; if not, then install it using your Linux distribution’s package manager. On Ubuntu, for example, it would be:

sudo apt-get install mercurial

On either OS, when you install Mercurial, it is a good idea to use a text editor to put a minimal configuration file in your home directory. It must be called .hgrc, and should look something like this:

[ui]
username= John Doe <jdoe@hawaii.edu>

[extensions]
purge=
pager=

[diff]
git = True

[pager]
pager = less -FRX

Install a C compiler (OS X only)

You will need the C compiler that goes with OS X. You can get it by installing all of Xcode (it’s huge), and then the command-line tools, but I recommend that you just install the latter directly via the following command:

xcode-select --install

The C compiler (gcc) is usually installed by default and is always available from the native package manager on Linux.

Install gfortran, pandoc, and the fortran netcdf4 library

This section is optional.

For pygamma and pytide, we need gfortran. There are probably other ways of getting a working version, but the way I have done it on my machine is using Homebrew:

brew install gcc

In earlier Homebrew versions gfortran was available as an independent installation, but now it is available only as part of the gcc suite.

Now install pandoc and the fortran libraries for netcdf:

brew tap homebrew/science
brew install netcdf --with-fortran
brew install pandoc

On Linux, use your package manager to install gfortran, e.g., “sudo apt-get install gfortran”. pandoc and the netcdf4 libraries will also be available via the package manager. On Ubuntu: “sudo apt-get install libnetcdf-dev” and “sudo apt-get install pandoc”.

Install UH software

Select a location for the source code, all of which will be downloaded as Mercurial repositories. I recommend the following sequence of commands, which will create the directory structure that I use:

cd
mkdir currents
mkdir currents/programs
mkdir currents/tide
mkdir currents/topog
mkdir currents/topog/etopo

In the following I will assume that this is what you have done. Next, download all the repos:

cd currents/programs
hg clone http://currents.soest.hawaii.edu/hgstage/codas3
hg clone http://currents.soest.hawaii.edu/hgstage/pycurrents
hg clone http://currents.soest.hawaii.edu/hgstage/pytide
hg clone http://currents.soest.hawaii.edu/hgstage/pygamma

Compile and install codas3 first:

cd codas3
./waf configure --python_env
./waf build
./waf install

Note

If the build stage gives an error on OSX, try re-doing the configure stage with an additional specification:

./waf configure AR=/usr/bin/ar --python_env
./waf build
./waf install

The --python_env option above will make the installation be in the Anaconda environment instead of the default /usr/local. Either will work, but keeping everything together makes it easier to keep track of what you have installed, and to delete it all if you want to make a fresh start some time.

Next, install the pycurrents Python package, which depends on successful installation of codas3 as described above:

cd ../pycurrents
./runsetup.py

The next two Python packages are optional. They include Fortran extension code, so they require a compatible Fortran compiler and environment. If you run into an error, check to see whether your environment includes Fortran-related variables, and/or whether you have a Fortran compiler other than gfortran installed.

Now pytide:

cd ../pytide
pip install .

And pygamma:

cd ../pygamma
pip install .

For pycurrents we need topography data; we will use Etopo1:

cd ../../topog/etopo

Now download ftp://currents.soest.hawaii.edu/pub/outgoing/etopo1_for_pycurrents.zip and unzip it into the location to which you cd’ed above. In trying to be helpful, gui tools often do the wrong thing, and this is a typical example. First, if you download by clicking with some browsers like Safari, the file name might get an extension added or removed. Second, either the downloader or the file browser might “helpfully” expand the archive, put it in the wrong place, add extensions to the file names, etc. Therefore you need to make sure that one way or another you end up with something like this:

(test) efiring@manini2:~/currents/topog/etopo$ ls -l
total 1738224
-rw-rw-r--  1 efiring  efiring  365714041 Jan  9  2015 etopo1_for_pycurrents.zip
-rw-rw-r--  1 efiring  efiring  466624802 May  5  2011 etopo1_ice_g_i2.bin
-rw-rw-r--  1 efiring  efiring        264 Dec  4  2009 etopo1_ice_g_i2.hdr
-rw-rw-r--  1 efiring  efiring   51854400 Aug  6  2011 etopo1_ice_g_i2_s3.bin
-rw-rw-r--  1 efiring  efiring    5764800 Aug  6  2011 etopo1_ice_g_i2_s9.bin

You don’t need to keep the zip file, but you need the four others.

For pytide we need tide model data from OSU. We will start with the TPXO7.2 global model and the Hawaii model, reformatted for more efficient access. Change directory to its destination:

cd ../../tide

Download ftp://currents.soest.hawaii.edu/pub/outgoing/tidemodels_for_pytide.zip and unzip it into your current location.

Again, make sure you end up with 4 files in that directory. Here they are, as listed in the zip archive using the unzip tool, which is also the best tool to use for unzipping:

(test) efiring@manini2:~/currents/tide$ unzip -l tidemodels_for_pytide.zip
Archive:  tidemodels_for_pytide.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
       57  03-09-10 10:51   Model_tpxo7.2
       45  07-23-10 09:19   Model_haw
329350721  12-02-10 12:48   uh_le_tpxo7.2
106296262  10-08-11 16:12   uh_le_haw
 --------                   -------
435647085                   4 files

Last, we need to symbolically link the data directories into the Anaconda environment so that the python modules using the data can find it. The following assumes your Anaconda installation was made in the “anaconda3” subdirectory of your home directory, which is the default location:

cd ~/anaconda3
ln -s ~/currents/topog .
ln -s ~/currents/tide .

Using symbolic links like this means that if something ever goes horribly wrong with your Anaconda installation, you can delete it and install a new one without having to download the data files again–you only need to redo this linking step.

Finished!

Quick test

To check some basic functionality, change to a directory that is not inside any of the source code directories and run ipython. For example:

cd
ipython

At the IPython prompt, execute:

%matplotlib qt5
from pycurrents.plot.maptools import mapper
m = mapper([-163, -153], [18, 25])
m.topo()
m.grid()

You should see a nice map of the main Hawaiian Islands.

If you installed gfortran and pytide, then you can also check pytide in the same IPython session:

import numpy as np
import matplotlib.pyplot as plt
from pytide import model
tidemod = model('haw')
t = np.linspace(0, 30, 1000)
vel = tidemod.velocity(2015, t, -158, 22.75)
h = tidemod.height(2015, t, -158, 22.75)
fig, (axv, axh) = plt.subplots(nrows=2, sharex=True)
axv.plot(t, vel.u, 'r', label='U')
axv.plot(t, vel.v, 'b', label='V')
axh.plot(t, h.h, 'g', label='H')
axh.set_xlabel('days in 2015')
axv.set_ylabel('m/s')
axh.set_ylabel('m')
axv.set_title('Barotropic velocity')
axh.set_title('Tide height')
axv.legend(loc='upper right')

You just plotted the tide model hindcast of elevation and velocity at Station ALOHA for the first month of 2015.