Installing and Running OCSSW Command-line Tools¶

Authors: Carina Poulin (NASA, SSAI), Ian Carroll (NASA, UMBC), Anna Windle (NASA, SSAI)

The following notebooks are **prerequisites** for this tutorial. - Learn with OCI: [Data Access][oci-data-access]

An [Earthdata Login][edl] account is required to access data from the NASA Earthdata system, including NASA ocean color data.

Summary¶

SeaDAS is the official data analysis sofware of NASA's Ocean Biology Distributed Active Archive Center (OB.DAAC); used to process, display and analyse ocean color data. SeaDAS is a dektop application that includes the Ocean Color Science Software (OCSSW) libraries. There are also command line programs for the OCSSW libraries, which we can use to write processing scripts or notebooks. This tutorial will show you how to install OCSSW and test it by processing a Level-1B (L1B) file from PACE OCI to a Level-2 (L2) file using l2gen. The installation can find OCSSW tools that work on modern Linux and macOS operating systems but not Windows.

Learning Objectives¶

At the end of this notebok you will know:

How to install OCSSW on your server
How to set up your OCSSW session
How to process a L1B file to L2 using l2gen

1. Setup¶

This tutorial is written in a Jupyter notebook connected to a Bash kernel. If you have downloaded this Jupyter notebook and want to run it, you also need to connected it to a Bash kernel. Alternatively, you can copy the code cells to the Terminal application found in the JupyterLab Launcher, which speaks Bash or something close enough.

(Optional) Use a Bash Kernel¶

Conda uses a lot of memory while configuring your environment. Choose an option with more than about 5GB of RAM from the JupyterHub Control Panel, or your install will fail.

Run the following command. If the terminal asks you to update conda, type Y to accept.

In [ ]:

%conda install bash_kernel

Follow the prompts from conda to proceed with any installs and updates. If prompted, enter "y" to accept.

Confirm the bash kernel is installed by starting a new Launcher. You should see the bash kernel along with Python and other kernels installed in your JupyterHub. You should now change the kernel of the notebook by clicking on the kernel name in the upper-right corner of the window and selecting the Bash kernel before moving on to the rest of the tutorial.

2. Install OCSSW¶

The OCSSW software is not a Python package and not available from conda or any other repository. To install it, we begin by aquiring an installer script from the OB.DAAC. This script is actually part of OCSSW, but we can use it independently to download and install the OCSSW binaries suitable for our system.

In [2]:

wget https://oceandata.sci.gsfc.nasa.gov/manifest/install_ocssw

--2024-06-11 16:39:01--  https://oceandata.sci.gsfc.nasa.gov/manifest/install_ocssw
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... 169.154.128.84, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 46637 (46K) [application/octet-stream]
Saving to: ‘install_ocssw.1’

install_ocssw.1     100%[===================>]  45.54K  --.-KB/s    in 0.1s    

2024-06-11 16:39:01 (408 KB/s) - ‘install_ocssw.1’ saved [46637/46637]

Similarly, we'll need the manifest module imported by the installer.

In [3]:

wget https://oceandata.sci.gsfc.nasa.gov/manifest/manifest.py

--2024-06-11 16:39:04--  https://oceandata.sci.gsfc.nasa.gov/manifest/manifest.py
Resolving oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)... 169.154.128.84, 2001:4d0:2418:128::84
Connecting to oceandata.sci.gsfc.nasa.gov (oceandata.sci.gsfc.nasa.gov)|169.154.128.84|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28390 (28K) [application/octet-stream]
Saving to: ‘manifest.py.1’

manifest.py.1       100%[===================>]  27.72K  --.-KB/s    in 0.08s   

2024-06-11 16:39:05 (344 KB/s) - ‘manifest.py.1’ saved [28390/28390]

Before you can use a downloaded script, you need to change its mode to be executable.

In [ ]:

chmod +x install_ocssw

Take a look at the different OCSSW "tags" you can install. It is recommended to use the most recent one for the installation, which is T2024.16 at the time of writing this tutorial. Tags starting with "V" are operational versions, and tags starting with "T" are test versions. Use "T" to process the latest data products, but keep in mind that processing can change a lot between tags. Other tags are deprecated, including those starting with "R".

In [4]:

./install_ocssw --list_tags

R2020.1
R2022.0
R2022.1
R2022.10
R2022.11
R2022.12
R2022.13
R2022.14
R2022.15
R2022.16
R2022.17
R2022.18
R2022.19
R2022.2
R2022.3
R2022.4
R2022.5
R2022.6
R2022.7
R2022.8
R2022.9
T2020.0
T2020.1
T2020.2
T2020.3
T2020.4
T2020.5
T2020.6
T2020.7
T2021.0
T2021.10
T2021.11
T2021.12
T2021.13
T2021.14
T2021.15
T2021.16
T2021.17
T2021.18
T2021.19
T2021.20
T2021.21
T2021.22
T2021.23
T2021.24
T2021.3
T2021.4
T2021.5
T2021.6
T2021.7
T2021.8
T2021.9
T2022.0
T2022.1
T2022.10
T2022.11
T2022.12
T2022.13
T2022.14
T2022.15
T2022.16
T2022.17
T2022.18
T2022.19
T2022.2
T2022.20
T2022.21
T2022.22
T2022.23
T2022.24
T2022.25
T2022.26
T2022.27
T2022.28
T2022.29
T2022.3
T2022.30
T2022.31
T2022.32
T2022.33
T2022.34
T2022.4
T2022.5
T2022.6
T2022.7
T2022.8
T2022.9
T2023.0
T2023.1
T2023.10
T2023.11
T2023.12
T2023.14
T2023.15
T2023.16
T2023.17
T2023.18
T2023.19
T2023.2
T2023.20
T2023.21
T2023.22
T2023.23
T2023.24
T2023.25
T2023.26
T2023.27
T2023.28
T2023.29
T2023.3
T2023.30
T2023.31
T2023.32
T2023.33
T2023.4
T2023.5
T2023.6
T2023.7
T2023.8
T2023.9
T2024.0
T2024.1
T2024.10
T2024.11
T2024.12
T2024.13
T2024.14
T2024.15
T2024.16
T2024.17
T2024.18
T2024.19
T2024.2
T2024.20
T2024.3
T2024.4
T2024.5
T2024.6
T2024.7
T2024.8
T2024.9
V2020.0
V2020.1
V2020.2
V2021.0
V2021.1
V2021.2
V2022.0
V2022.1
V2022.2
V2022.3
V2023.0
V2023.1
V2023.2
V2023.3
initial
v3.1.0
v3.1.0-rc
v3.1.0-rc1
v3.2.2
v3.2.3

Define an environmental variable called "OCSSWROOT" that specifies a directory for your OCSSW installation. Environment variables are set in Bash using the export command, and displayed with printenv

In [ ]:

export OCSSWROOT=/tmp/ocssw

In [ ]:

printenv OCSSWROOT

You will need to [repeat these installation steps](#all) if your OCSSWROOT directory does not persist between sessions.

The /tmp/ocssw folder, for instance, will not be present the next time JupyterHub creates a server. Consider the trade off between installation time, speed, and storage costs when choosing your OCSSWROOT. With the arguments below, the installation takes 11GB of storage space. We use the quick and cheap location for this tutorial.

Install OCSSW using the --tag argument to pick from the list above. Also provide optional arguments for sensors you will be working with. In this case, we will only be using OCI. A list of optional arguments can be found on the OCSSW webpage or with ./install_ocssw --help.

Tip: The process is not finished as long as the counter to the left of the cell shows [*]. It will take some time to install all the tools (7 of 7 installations).

In [ ]:

./install_ocssw --tag=T2024.19 --seadas --oci

Finish up by calling source on the "OCSSW_bash.env" file, which exports additional environment variables. This environment file specifies the locations of all files required by OCSSW, and must be exported in every Terminal or Bash kernel before you run l2gen or any other OCSSW command.

In [ ]:

source $OCSSWROOT/OCSSW_bash.env

Confirm the environment has been set by checking that another install_ocssw script is now discoverable by Bash at the newly installed location.

In [ ]:

which install_ocssw

You are now ready to run l2gen, the Level-2 processing function for all ocean color instruments under the auspices of the GSFC Ocean Biology Processing Group!

3. Process L1B Data with `l2gen`¶

Run l2gen by itself to view the extensive list of options available. You can find more information on the Seadas website.

In [ ]:

l2gen

Feel free to explore all of l2gen options to produce a highly customized Level-2 dataset for your research. Here we just scratch the surface.

To process a L1B file using l2gen you need, at a minimum, to set an input file name (ifile) and an output file name (ofile). You can also indicate a data suite; in this example, we will proceed with the Surface Reflectance suite used to make true color images (SFREFL). We turn off the atmospheric correction with atmocor=0 to save processing time.

For this example, we will be using the L1B file downloaded in the OCI Data Access notebook. Confirm that the L1B file to process is at the expected location by listing (with ls) the directory contents. If the directory is empty, check that you've completed the prerequiste notebooks for this tutorial!

In [ ]:

ls L1B

Create a directory for output files.

In [ ]:

mkdir L2

And run! Note, this may take several minutes.

In [ ]:

l2gen \
  ifile=L1B/PACE_OCI.20240501T165311.L1B.nc \
  ofile=L2/PACE_OCI.20240501T165311.L2.SFREFL.nc \
  suite=SFREFL \
  atmocor=0

Scroll down in the output above to see updates on processing. Upon completion, you should have a new processed L2 file in your L2 folder.

4. All-in-One¶

In case you need to run the sequence above in a terminal regularly, here are all the commands to copy and run together. This assumes you already used wget to persist the install script and manifest.json in the current directory.

In [ ]:

export OCSSWROOT=/tmp/ocssw
./install_ocssw --tag=T2024.19 --seadas --oci
source $OCSSWROOT/OCSSW_bash.env

Or, if that assumption is wrong and you also need to wget those files, copy and run the following.

In [ ]:

wget https://oceandata.sci.gsfc.nasa.gov/manifest/install_ocssw
wget https://oceandata.sci.gsfc.nasa.gov/manifest/manifest.py
chmod +x install_ocssw
export OCSSWROOT=/tmp/ocssw
./install_ocssw --tag=T2024.19 --seadas --oci
source $OCSSWROOT/OCSSW_bash.env

You have completed the notebook on installing OCCSW. Check out the notebook on Processing with OCSSW Tools.

Installing and Running OCSSW Command-line Tools¶

Summary¶

Learning Objectives¶

Contents¶

1. Setup¶

(Optional) Use a Bash Kernel¶

2. Install OCSSW¶

3. Process L1B Data with l2gen¶

4. All-in-One¶

3. Process L1B Data with `l2gen`¶