Not logged inOcean Color Forum

The forum is locked.

The Ocean Color Forum has transitioned over to the Earthdata Forum ( The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.

Up Topic SeaDAS / SeaDAS - General Questions / gpt merge (locked)
- By bruce Date 2016-10-21 13:34
Where is it documented?  With all of the available options...

I'm trying to add a new product to an existing file.  I can create the new product with out problem and if I use the Write operator it gets written on it's own to a new file but when I try and merge using the following node (basic structure from the help file and a couple online examples, NewProduct is what it's called in the bandMaths node)

  <node id="mergeNode">

It fails with the ever helpful
  at org.esa.beam.gpf.operators.standard.MergeOp.copyBandWithFeatures(
  at org.esa.beam.gpf.operators.standard.MergeOp.initialize(
  at org.esa.beam.framework.gpf.internal.OperatorContext.initializeOperator(
  at org.esa.beam.framework.gpf.internal.OperatorContext.getTargetProduct(
  at org.esa.beam.framework.gpf.Operator.getTargetProduct(
  at org.esa.beam.framework.gpf.graph.NodeContext.initTargetProduct(
  at org.esa.beam.framework.gpf.graph.GraphContext.initNodeContext(
  at org.esa.beam.framework.gpf.graph.GraphContext.initNodeContext(
  at org.esa.beam.framework.gpf.graph.GraphContext.initOutput(
  at org.esa.beam.framework.gpf.graph.GraphContext.<init>(
  at org.esa.beam.framework.gpf.graph.GraphContext.<init>(
  at org.esa.beam.framework.gpf.graph.GraphProcessor.executeGraph(
  at org.esa.beam.framework.gpf.main.DefaultCommandLineContext.executeGraph(
  at org.esa.beam.framework.gpf.main.CommandLineTool.executeGraph(
  at org.esa.beam.framework.gpf.main.CommandLineTool.runGraph(
  at org.esa.beam.framework.gpf.main.CommandLineTool.runGraphOrOperator(
  at org.esa.beam.framework.gpf.main.GPT.main(
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(
  at java.lang.reflect.Method.invoke(
  at com.bc.ceres.launcher.Launcher.launch(
  at com.bc.ceres.launcher.Launcher.main(

Error: java.lang.NullPointerException
- By bruce Date 2016-10-21 14:41
OK, got it working (mostly)


  <node id="mergeNode">

works, but it dumps all of the data to "target.dim" and the directory "".  which leads to 2 questions.  Is it possible to write it back into the L2_LAC_OC file?  and if not, how do I control the output filename? 

I have to say, while it's a powerful tool, the documentation (not a reflection on seadas, but rather on ESA and beam) "inhales briskly".
- By gnwiii Date 2016-10-23 10:47
Have you considered using beampy to add a product to level-2 NetCDF4-CF files?  Here is an example:

#! /usr/bin/env python3
# -- add spm_645 to an existing level-2 file
# note that some processing programs require the new product to appear in the products.xml file
# uses Unidata netCDF4 module. To install:  "pip3 install --user netCDF4" 
import numpy as np
from netCDF4 import Dataset
import argparse

parser = argparse.ArgumentParser(description='Add spm_645 to Level-2 file with Rrs_645',
    usage='calc_spm_645 l2name', epilog='''
        Given a level-2 file with Rrs_645, apply the method of
        Nechad et al, RSE 2010, to compute spm_645 and add the
        band to the existing level-2 file.''')

parser.add_argument('l2name', nargs='+', help='name of the level-2 file')

args = parser.parse_args()
l2name = args.l2name[0]
nc=Dataset(l2name, 'a')

v = gd.createVariable('spm_645','f',('number_of_lines', 'pixels_per_line'),
v.long_name = 'suspended particulate matter'
v.units = 'gm-3'
v.standard_name = 'spm_645'
v.valid_min = 0.01
v.valid_max = 1000.0
v.reference = 'Nechad, Ruddick, and Park, RSE 2010'
v[:] = 3.14*0.529*Rrs_645[:]
v[:] = 258.85*v[:]/(1-v[:]/0.1641)

Here are the changes to product.xml as a context diff:

*** product+spm.xml  2016-06-30 07:47:55.000000000 -0300
--- product.xml  2016-05-31 14:50:46.000000000 -0300
*** 4621,4638 ****
              <description>Optimal SIOP combination, GIOP SVD_SIOP algorithm</description>
-     <product name="spm">
-         <units>gm^-3</units>
-         <category>Derived</category>
-   <displayScale>log</displayScale>
-         <range>
-           <validMin>0.01</validMin>
-           <validMax>1000</validMax>
-         </range>
-         <algorithm name="spm">
-     <suffix>_645</suffix>
-           <cat_ix>999</cat_ix>
-           <description>Suspended particulate matter from 645 nm band</description>
-         </algorithm>
-     </product>
- By bruce Date 2016-10-24 10:38
Thanks George...
A few questions.  I'm not a python wiz, and have run into a bit of a snag.  Any chance you can advise?
The new product equation depends on the value of 2 others...  In pseudo code.

     if old1 > 0 then new = old1 else if old2 > 1 then new = old2 else new=nan

v=np.where(old1>0,old1,old2) fails this way

Traceback (most recent call last):
  File "", line 40, in <module>
TypeError: __array__() takes no arguments (1 given)

Attempts at creating indices using
useold2=old1<=0 and old2>1

fail thusly
Traceback (most recent call last):
  File "", line 43, in <module>
  File "netCDF4.pyx", line 2900, in netCDF4.Variable.__getitem__ (netCDF4.c:35565)

Any thoughts? (I realize this is a python question, not a seadas question)
- By aynur Date 2016-10-24 15:30
To save the resulted file in a format other than "dim", you can use the "Write" operator after "Merge" and choose one of the "export" formats that are listed under File->Export in SeaDAS GUI. It's not possible to write it back into the L2_LAC_OC file yet.
- By gnwiii Date 2016-10-25 07:51
I'm no Python wizard, but I'm guessing that old1 and old2 are <type "netCDF4._netCDF4.Variable">.  You need to convert them to numpy.ndarray types.

$ python3.5
Python 3.5.2 (default, Oct 12 2016, 07:59:06)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>> import numpy as np
>>> old1=np.array([0,1,2])
>>> type(old1)

<class 'numpy.ndarray'>

>>> old2 = 10+old1
>>> np.where(old1 >0, old1, old2)

array([10,  1,  2])

You may need to sprinkle some [:] incantations on your script.   See
- By bruce Date 2016-10-25 13:44
got it working...  Posted here to help someone else in the future.  I don't know if the reshapes are strictly necessary, but it works as it is, so I'm happy.

#! /usr/bin/python
import sys
import numpy as np
from netCDF4 import Dataset
from copy import copy

l2name = sys.argv[1]
nc=Dataset(l2name, 'a')
no_lines = len(nc.dimensions[u'number_of_lines'])
no_pixels = len(nc.dimensions[u'pixels_per_line'])


flat1 = np.reshape(old1,no_lines*no_pixels)
flat2 = np.reshape(old2,no_lines*no_pixels)

v = gd.createVariable('new','f',('number_of_lines', 'pixels_per_line'),
v.long_name = 'New algorithm'
v.units = 'units'
v.standard_name = 'new'

tvar = np.where(flat1>0,flat1,np.where(flat2>0.1,flat2,-32767.0))
tvar = tvar.reshape(no_lines,no_pixels)

Up Topic SeaDAS / SeaDAS - General Questions / gpt merge (locked)

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill