SeaDAS 4.9 Update 04 includes new automatic search functions that determine the optimum MET, OZONE, and OISST ancillary data files for msl12 processing. After the update, please run seadas_setup to activate this function.

If the required ancillary data files aren't present on the user's local hard disk, this new feature will optionally download the files from an OBPG ftp server. If the ancillary data is not successfully retrieved, the climatologies are substituted. There are three ways to make use of this new SeaDAS feature:

1) The new functionality is built directly into the msl12 GUI window.

2) On the SeaDAS/IDL command line, the msl12 command now accepts the following arguments:

  metsearch=metsearch, ozonesearch=ozonesearch, sstsearch=sstsearch, disableftp=disableftp

For example, to disable ftp searches add disableftp=1 or /disableftp to your command.

3) On the UNIX command line, the new feature can be accessed using any one of the 3 new SeaDAS auto-ancillary scripts ms_met.csh, ms_ozone.csh, or ms_oisst.csh. Each script will create a text file containing search results that can be supplied to the msl12 processing binary. The scripts' associated README file is $SEADAS/etc/README.ms_met_ozone_oisst and includes a description of the logic used.

-----

Users of a shared SeaDAS installation can activate their own custom ancillary directories by copying the $SEADAS/config/seadas.env_user_bash or $SEADAS/config/seadas.env_user_csh file into their own ~/.seadas/ directory and uncommenting lines containing the following variables:

MSL12_METOZ
MSL12_OISST
AUTO_MET
AUTO_OZONE
AUTO_SST_MODIS
AUTO_SST_OTHER
ENABLE_CURL

Thanks for your upadates mark, this updates is essential for us since the GUI is changed after updating the software.
Could you please explain me why the oisst files were required for two days ( it is required to have the previous day) for processing ?.

With regards
RM Narayanan

I don't quite understand your question.. only one OISST file is ever used for processing and will be the first file preceding the time of the pass or else the climatology. See http://oceancolor.gsfc.nasa.gov/seadas/README.ms_met_ozone_oisst for the logic used and http://oceancolor.gsfc.nasa.gov/seadas/doc/toplevel/anc_info.html#OISST for more OISST info.

Hi all,

I can't seem to get the automatic metseach=metseach and ozonesearch=ozonesearch working from msl12.  It just chooses the climatologies instead.

Here is my MSL12 invoking line:
msl12,4, ifile=L1B_file, geofile=geofile,$
ofile1=L2_file1,l2prod1=l2_prods1,ofile2=L2_file2,l2prod2=l2_prods2,$
metsearch=metseach,ozonesearch=ozonesearch

Here are the uncommented lines in my seadas.env_user_bash entry.
export MODIS_ATTEPH=/ancillary/atteph
export MSL12_METOZ=/ancillary/metoz
export MSL12_OISST=/ancillary/oisst
export AUTO_MET=FTP
export AUTO_OZONE=FTP

Now to get around this problem, I can invoke the csh lookup scripts like this:
;    spawn, 'ms_met.csh '+L1B_file
;               spawn, 'ms_ozone.csh '+L1B_file

and then look in the file and specify the met1=met1 , ozone1=ozone1 etc in msl12 blah...

This works about 80% of the time and it searches the right places, and will auto download from your ftp site...
but every now and then when I run the ms_met.csh or the ms_ozone.csh command, I get an error message:
set: Variable name must begin with a letter.
...and then the corresponding list file is not created....
but it appears to happen randomly and then my script dies. 
Right now I just re-run the script until it doesn't die but I was just wondering if there's something im doing wrong in a seemingly random fashion.  But even before this, I don't know why the metsearch=metsearch isnt working on my system.

Seadas 4.9.4
FC4
P4 3.0Ghz 2Gb ram

Just wondering if anyone else has encountered this problem, and if there's a solution

Cheers
Matt

Hi Matt,

First, "metsearch=metsearch" isn't working because this isn't the correct syntax. (I realise the help page is a bit confusing and could lead to this mistake.) The correct syntax is either "metsearch=1" or simply "/metsearch", e.g.:

msl12, 4, ..., metsearch=1
or
msl12, 4, ..., /metsearch

Second, the "set: Variable name must begin with a letter" error is due to a kernel instability issue with FC4. This error should go away if you turn off VM mapping randomization as outlined in the SeaDAS FAQ: http://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?pid=2253

Cheers Mike,

Yeah I was thrown by the help page.  I will ammend my scripts once I no longer get the randomize error using the manual met lookup approach.
Actually its now processed 4 files to L2 without dying so its looking good so far.

just a note - my /etc/sysctl.conf did not contain the randomise = 1 line, but grepping the /sbin/sysctrl did reveal the randomise=1 line.
I put in the line into my sysctrl.conf line (as it is with other versions of linux), and did

> /sbin/sysctl -w kernel.randomize_va_space=0

kernel.randomize_va_space = 0

... so far so good.  I will see what happens when I reboot.

Thanks for a quick reply - happy days!

Matt

Greetings mike,
                     Can I put metsearch into the .par file? It doesn't seem to work when I put metsearch=1 in the .par file (example below).
......
filter_opt=       1
filter_file=$MSL12_DATA/modisa/msl12_filter.dat
metsearch=1
ozonesearch=1
sstsearch=1
land=$MSL12_DATA/common/landmask.dat
water=$MSL12_DATA/common/watermask.dat
......

Cheers

Lan-Wei

No you can't put the metsearch parameter in a par file since this feature is completely independent of the msl12 binary and so msl12 won't recognize this option. You must either specify metsearch=1 on the SeaDAS/IDL command line or use the ms_met.csh UNIX script to predetermine the met files.

If you use the ms_met.csh script, you can then either append the results to you par file, e.g.:

cat  file.met_list  >>file.par

...or you can feed the msl12 UNIX binary multiple par files, e.g.:

msl12  par=file.par  par=file.met_list  (specify file.met_list last so it will override possible met files specified in file.par)

Greetings Mike,
                     This is a problem I encounter very recently and hope you may have some suggestion about this:

I have been running MODIS data for the last three weeks (SEADAS4.9 with all four updates) with no problem. For the recent MODIS passes (21 Aug ~23 Aug), I foud that my script stoped at "Running validation test on geolocation file ". One MODIS pass did go through validation test and returns a fail message:

-----------------------------------------------------------
Running validation test on geolocation file...                
/usr/local/seadas4.9/etc/modis_geocheck.csh ./seadas_GEO.HDF 95
Number of pixel with missing gelocation: 744700
Percent of geolocated pixels (rounded):86
......
please ensure utcpole.dat and leapsec.dat are up-to-date in:
/usr/local/seadas4.9/data/modis/static

--------------------------------------------------------------------------------------------
I checked /usr/local/seadas4.9/data/modis/static and found that the generation date for utcpole.dat/leapsec.dat is today (may be FTP by PDS_to_L1A processing?)

I also checked the LogStatus file, there are some error message as well:

------------------------------------------------------------------------------------------------

GEO_location_main.c, main():MODIS_U_GEO_BEGIN:288778294
Thu Aug 24 10:05:38 2006
Seed file:$Revision: 1.1.1.1 $ Running MOD_PR03 MODIS Geolocation main() 5.1 compiled on Mar 12 2006 at 13:54:17

PGS_TD_timeCheck():PGSTD_M_ASCII_TIME_FMT_B:26112
the time string passed in is in proper CCSDS ASCII Time Format B

PGS_TD_timeCheck():PGSTD_M_ASCII_TIME_FMT_B - 521 additional occurrences

GEO_del_limit_check.c, GEO_del_limit_check():MODIS_E_NO_VAL_VEL_SAMP:288779815
Thu Aug 24 10:05:41 2006
No valid velocity samples.

GEO_prepare_mirr_data.c, GEO_prepare_mirr_data():MODIS_E_DATA_SCAN:288779781
Thu Aug 24 10:05:41 2006
No valid encoder data in scan in scan 153

---------------------------------------------------------------------------------------------------

If it only happened for one pass, it could be noisy PDS data. But it's happening for all of the recent passes, I am not sure what's the reason? Any comments?

Regards

Lan-Wei

Lan-Wei,

Mike is on vacation until September 5th, so I will take a crack at your question.  I am going to assume that this is direct broadcast data that you are processing?  If so, then I have to guess that there is some problem with your data.  I agree that it is unlikely that noise would be the problem for every pass, but I would look for a problem with your data acquisition first.  Since the software has not changed, it makes sense to me that there is something different with the data, somewhere in the acquisition process.  Since the error does not show up until the geolocation processing, I will assume that the level 1a processing is successful.  If so, then how does the level 1a data look?  Can you display it in seadas?  Does it show any obvious gaps in the data?  Have there been any significant changes in your data capture software/hardware since August 21?

Regards, Mark

Greetings Mark,
                     Sorry about the late reply. After a few more days of close minitoring, such error did not occur again. So I guess it just BAD PDS data causing the problem. 

                    And yes, I am working with the direct broadcast data. We are involved in a environmental monitoring project (dredging to increase Port efficiency). On a daily basis, we process yesterday's Terra/Aqua to nLw, then convert them to TSS (we have some field data and do the regression between measured-TSS VS nLw_551, I didn't have the chance to compare measured-TSS VS Rrs_551 yet, any comments?). 

                      In fact, I am having another problem now. I just download/installed sedas5.0 (yesterday) but I was unable to generate MODIS 250m L2 output. Then I download/installed Update01 (although I shouldn't need to do that) still with no success. Please see the error message below:
---------------------------------------------------------------------------------------------------------------------------------------

Computing pixel/line coordinates.  Standby......
$SEADAS/bin/MODll2snpx /data/projects/haypoint/work/data/modis/35207/P0420064AAAAAAAAAAAAAA06213002912001_GEO.hdf 147 -23 151 -19
GENERIC_FILE_TYPE detected a MODIS file.
Created MSL12 parameter file: /data/projects/haypoint/work/data/modis/35207/kill.L2.par
UNIX command: /bin/csh -f -c "msl12 par=/data/projects/haypoint/work/data/modis/35207/kill.L2.par"
Input file /data/projects/haypoint/work/data/modis/35207/A2006109180000.L1B_QKM is MODIS Terra Level-1B HDF-EOS product at 250m Resolution.
Loading default parameters for HMODIST from /usr/local/seadas5.0/data/hmodist/msl12_defaults.par

Loading user parameters for HMODIST

Opening filter file /usr/local/seadas5.0/data/modisa/msl12_filter.dat
Setting 7 x 5 straylight filter on HILT mask

Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1

Minimum fill set to 1 pixels

Setting 7 x 5 straylight filter on CLDICE mask

Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1

Minimum fill set to 1 pixels

Loading characteristics for HMODIST
Opening sensor information file /usr/local/seadas5.0/data/hmodist/msl12_sensor_info.dat
  Bnd Lam       Fo    Tau_r     k_oz    k_no2    t_co2   awhite       aw      bbw
    0 412  172.718 3.109e-01 2.440e-03 5.732e-19 1.000e+00 1.000e+00 2.027e-02 3.242e-03
    1 443  187.688 2.375e-01 3.830e-03 4.916e-19 1.000e+00 1.000e+00 1.110e-02 2.454e-03
    2 469  205.733 1.915e-01 1.040e-02 3.896e-19 1.000e+00 1.000e+00 1.033e-02 1.962e-03
    3 488  194.973 1.596e-01 2.250e-02 2.802e-19 1.000e+00 1.000e+00 1.677e-02 1.622e-03
    4 531  185.820 1.131e-01 6.630e-02 1.437e-19 1.000e+00 1.000e+00 4.496e-02 1.132e-03
    5 551  186.594 9.940e-02 8.360e-02 1.124e-19 1.000e+00 1.000e+00 5.492e-02 9.890e-04
    6 555  183.760 9.480e-02 9.300e-02 8.769e-21 1.000e+00 1.000e+00 5.835e-02 9.380e-04
    7 645  157.807 5.070e-02 7.110e-02 7.089e-21 1.000e+00 8.892e-01 3.458e-01 4.902e-04
    8 667  152.163 4.460e-02 4.850e-02 8.000e-21 1.000e+00 8.892e-01 4.920e-01 4.262e-04
    9 678  148.097 4.160e-02 3.950e-02 8.500e-21 1.000e+00 8.892e-01 5.118e-01 3.968e-04
   10 748  128.123 2.860e-02 1.190e-02 1.500e-21 1.000e+00 7.600e-01 2.801e+00 2.671e-04
   11 859   97.174 1.640e-02 3.130e-03 0.000e+00 1.000e+00 6.449e-01 4.395e+00 1.473e-04
   12 869   95.849 1.550e-02 2.630e-03 0.000e+00 1.000e+00 6.449e-01 4.727e+00 1.404e-04
   13 1240   43.670 3.600e-03 0.000e+00 0.000e+00 9.994e-01 0.000e+00 1.147e+02 0.000e+00
   14 1640   23.800 1.200e-03 0.000e+00 0.000e+00 9.890e-01 0.000e+00 6.690e+02 0.000e+00
   15 2130    9.890 4.000e-04 0.000e+00 0.000e+00 9.696e-01 0.000e+00 2.800e+03 0.000e+00

Parameter input error:  Number of input gains must equal 16.
-E- msl12: Error parsing input parameters.

I do have the MODIS files named as
A2006109180000.L1B_LAC
A2006109180000.L1B_HKM
A2006109180000.L1B_QKM

At the same GUI, if I select the A2006109180000.L1B_LAC (and re-select GEO) and click "run", L2 processing completed successfully.

Can you attach an example .par file for me to give it a try? I can replace the ifile/geofile/ofile1 with my own file and give it a try.
 
 

lanwei,

The reason you had the problem with 250m is because you were using the 250m file as the input file.  Always use the 1km file as the input file, even if you want to produce 500m or 250m.  We are going to change this in the future so that you can specify the 500m or 250m file as input, but until then, you need to specify the 1KM file as the input file.

I also notice something unusual with the following from your output:

Input file /data/projects/haypoint/work/data/modis/35207/A2006109180000.L1B_QKM is MODIS Terra Level-1B HDF-EOS product at 250m Resolution.

You have somehow specified an Aqua filenaming convention(A2006...), but the msl12 program is reading it as Terra data.  Either the level 1B file was not named correctly, or the msl12 program is mistaking it for Terra data.  Please look at your input files and verify that they are Aqua data, and if so, let us know so that we can figure out why msl12 is mistaking it for Terra data.  If it is really Terra data, you should probably change the "A" to a "T".

You can easily generate a parameter file with the GUI, by making all of the selections necessary for a processing, and then specifying a parameter file output name down at the bottom of the first tab on the msl12 GUI, then clicking on the Options button and selecting Save.  Then you can go into the saved parameter file and change any of the parameters as you see fit.  The same parameter file can be loaded back in by clicking on Options, then Load.

Regards, Mark

Greetings Mark,
                     I read the instruction regarding the file naming convention (my original file name is 1km, 05km and 025km), so I simply grab an Aqua pass's file name to rename my Terra file nemes for testing. msl12 didn't mistaking with Terra/Aqua data.

Anyway, I process a new pass and genetare the L1B files:

P1540064AAAAAAAAAAAAAA06244040743001.L1B_LAC
P1540064AAAAAAAAAAAAAA06244040743001.L1B_HKM
P1540064AAAAAAAAAAAAAA06244040743001.L1B_QKM

Below please find the steps I did:
1. Click  Process->MODIS->msl12,4. and select P1540064AAAAAAAAAAAAAA06244040743001.L1B_LAC and GEO file
2. Click on the Processing Options tab and select 250m (interpolated 1km/500m +250m)
3. Click on subset/subsample tab and specify South-West corner        -23   147
                                                               North-East corner        -19   151
4. Click on file selection tab and select L2 products, I selected all for nLw and click OK

5. Click Run, and shows the following error message:

SeaDAS>
GENERIC_FILE_TYPE detected a MODIS file.
Created MSL12 parameter file: /data/projects/haypoint/work/data/modis/P1540064AAAAAAAAAAAAAA06244040743001.L2.par
UNIX command: /bin/csh -f -c "msl12 par=/data/projects/haypoint/work/data/modis/P1540064AAAAAAAAAAAAAA06244040743001.L2.par"
Invalid argument "gas_opt=      15"
-E- msl12_input.c Line 297: Error reading input parameters.
-E- msl12: Error parsing input parameters.

6. If I go to the Processing options and select only Ozone for Gaseous transmittance and run again, it shows the following error message:

SeaDAS>
GENERIC_FILE_TYPE detected a MODIS file.
Created MSL12 parameter file: /data/projects/haypoint/work/data/modis/P1540064AAAAAAAAAAAAAA06244040743001.L2.par
UNIX command: /bin/csh -f -c "msl12 par=/data/projects/haypoint/work/data/modis/P1540064AAAAAAAAAAAAAA06244040743001.L2.par"
Invalid argument "gas_opt=       1"
-E- msl12_input.c Line 297: Error reading input parameters.
-E- msl12: Error parsing input parameters.

7. If I only select nLw645 (250m band) as L2 product, same error message occured.

Any idea where I did wrong?

Lanwei,

I don't see anything wrong, but gas_opt is a recently added option in msl12.  Is it possible that you have an old version of seadas executables in your path? What version of msl12 is displayed when you run?  Approximately 1 page down in the output messages it should say:

Begin MSl12 Version 5.5.2 Processing
Sensor is HMODISA

If that looks O.K., I'd suggest posting the par file so Mark can test it on Tuesday (Monday is a holiday here).

-- bryan

Lanwei,

I agree with Bryan, this must be an old version of msl12 in your path.  Check your .cshrc or .bashrc file and remove any paths to old seadas's.

Mark

Hi All,
Im now using SEADAS 5.2, with "l2gen" and the command line metsearch=1 or /metsearch options are not working.

ie
Invalid argument "ozonesearch=1"
-E- msl12_input.c Line 331: Error reading input parameters.
-E- l2gen: Error parsing input parameters.

etc...

The l2gen Unix command doesn't accept the metsearch option (this option is only for the l2gen IDL command). But it's good that you're using l2gen on the Unix command-line. For Unix, what you need to do is use the ms_met.csh and ms_ozone.csh auto-ancillary Unix scripts in tandem with the l2gen Unix command as outlined in option three of this FAQ.

argh,
Well i'm not totally free from IDL, but weening myself off it
I now "SPAWN" the l2gen from an IDL script.  I have a stand-alone copy of IDL on my machine.
Okies, so I guess I can spawn these csh scripts too.
Cheers!
Matt

Matt, as you might be realizing, it is often best to just use a master Unix script with a FOR loop that calls SeaDAS in batch mode whenever you need to execute IDL commands, e.g.

#!/bin/sh
for FILE in <pattern>
do
  ...Unix commands...
  seadas -b batchfile1
  ...more Unix commands...
  seadas -b batchfile2
  ...more Unix commands...
done
...more Unix commands...
seadas -b batchfile3
...more Unix commands...