Ocean Color Forum - Not logged in
Forum Ocean Color Home Help Search Login
Previous Next Up Topic SeaDAS / SeaDAS 6.x - General Questions / Seadas l3bin batch process stops after 30 days (locked) (3060 hits)
By Andrea Date 2008-01-03 23:25 Edited 2008-01-04 20:33
Hello, I am running the newest SEADAS version for Intel MAC, with the non-embedded IDL version.  I was using the embedded version before I purchased IDL and ran into the same problem when SEADAS halts after running 30 days of time-binned files. I bought IDL hoping to processes large data sets. Could you please tell me how to either change the code or how to run the program so that it does not stop at 30 days?  In the past I ran the same code on a Unix machine and I have never had it stop after the 30th day. Is this a MAC problem?

Here is my sample code and I am using the %seadas -b l3bin_timenew  command to start the processing.  (with a shell script called l3bin_timenew)
Many thanks!

PRO l3bin_timenew, INDIR=indir, OUTDIR=outdir

IF (NOT(KEYWORD_SET(INDIR))) THEN indir ='/desktop/lakes/superior/2002/spacebin/'
IF (NOT(KEYWORD_SET(OUTDIR))) THEN outdir='/desktop/lakes/superior/2002/timebin/'

; VARIABLES FOR BINNING
;
mysyear='2002'
mysday=185
myeyear='2002'
myeday=336
step=1

j=-1
for x=mysday,myeday-1,step DO BEGIN

       daystring=STRTRIM(STRING(x),1)
       IF strlen(daystring) NE 3 THEN daystring='0'+daystring
       IF strlen(daystring) NE 3 THEN daystring='0'+daystring

       myfiles=findfile(indir+'A'+mysyear+daystring+'*UMD*')

       ;this section prints only the day of interest to the inlist.txt file that is executed in the l3bin command below
       print, myfiles
       openw, lun, 'inlist.txt', /get_lun
       printf,lun, myfiles
       free_lun,lun

       outfile=outdir+'/A'+mysyear+'_'+daystring+'_'+'L3b_UMD'
       compressed=outfile+'.bz2'

       ;MAIN PROCESSING LINE

       checkfile=findfile(outfile) & IF checkfile[0] ne "" THEN PRINT, 'File Exists! Skipping...' ELSE BEGIN
       checkfile=findfile(compressed) & IF checkfile[0] ne "" THEN PRINT, 'File Exists! Skipping...' ELSE BEGIN
       infile=indir+'inlist.txt'

       l3bin, ifile=infile, ofile=outfile, l3bprod=['sst'], noext=1,latsouth=46.0000, latnorth=49.5000, lonwest=-94.0000, loneast=-83.00

       endelse
       endelse
print, x
print, mysday

endfor
;SPAWN,'rm '+filelist
end
By mark Date 2008-01-07 19:16
Andrea,

It is possible that there is a problem in our l3bin code that has not been detected yet.  But I am curious for you to try one thing first.  Can you change your call to free_lun to force the file close, and maybe that will fix the problem?  You can do it like this:

free_lun,lun,/force

I'm thinking that maybe there is some error occuring when free_lun tries to close the file.  The /force flag forces it to close regardless of error condition.  If that does not work, can you please post the entire output error message?

Regards, Mark
By Andrea Date 2008-01-07 20:35
Thanks Mark, I tried that and received the same error.  The number 32 at the end is the number of days it cycled through and 1 is the day it started with.

l3bin.pro: cmd_final =
/bin/bash -c "l3bin
IN=/Users/desktop/lakes/superior/2005/spacebin/inlist.txt
OUT=/Users/desktop/lakes/superior/2005/timebin//A2005_032_L3b_UMD
latsouth=46.0000 latnorth=49.5000 lonwest=-94.0000 loneast=-83.0000
noext=1 OUT_PARM='sst'"
L3BIN 2.17 (Aug 22 2007 12:14:17)
1 input files
0 /Users/desktop/lakes/superior/2005/spacebin/A2005032183500.L3_UMD_sLAC
open status: 0
nbins: 380187134
nbins: 380187134
Setting Up: SEAGrid 1073741918
Writing 1 records to SEAGrid (1)
Detaching: SEAGrid
irow:     0 of    17280 Mon Jan  7 14:16:14 2008
irow:   500 of    17280 Mon Jan  7 14:16:14 2008
irow:  1000 of    17280 Mon Jan  7 14:16:15 2008
irow:  1500 of    17280 Mon Jan  7 14:16:15 2008
irow:  2000 of    17280 Mon Jan  7 14:16:16 2008
irow:  2500 of    17280 Mon Jan  7 14:16:16 2008
irow:  3000 of    17280 Mon Jan  7 14:16:17 2008
irow:  3500 of    17280 Mon Jan  7 14:16:17 2008
irow:  4000 of    17280 Mon Jan  7 14:16:18 2008
irow:  4500 of    17280 Mon Jan  7 14:16:18 2008
irow:  5000 of    17280 Mon Jan  7 14:16:18 2008
irow:  5500 of    17280 Mon Jan  7 14:16:19 2008
irow:  6000 of    17280 Mon Jan  7 14:16:19 2008
irow:  6500 of    17280 Mon Jan  7 14:16:20 2008
irow:  7000 of    17280 Mon Jan  7 14:16:20 2008
irow:  7500 of    17280 Mon Jan  7 14:16:20 2008
irow:  8000 of    17280 Mon Jan  7 14:16:21 2008
irow:  8500 of    17280 Mon Jan  7 14:16:21 2008
irow:  9000 of    17280 Mon Jan  7 14:16:22 2008
irow:  9500 of    17280 Mon Jan  7 14:16:22 2008
irow: 10000 of    17280 Mon Jan  7 14:16:23 2008
irow: 10500 of    17280 Mon Jan  7 14:16:23 2008
irow: 11000 of    17280 Mon Jan  7 14:16:23 2008
irow: 11500 of    17280 Mon Jan  7 14:16:24 2008
irow: 12000 of    17280 Mon Jan  7 14:16:24 2008
irow: 12500 of    17280 Mon Jan  7 14:16:25 2008
irow: 13000 of    17280 Mon Jan  7 14:16:25 2008
irow: 13500 of    17280 Mon Jan  7 14:16:25 2008
irow: 14000 of    17280 Mon Jan  7 14:16:26 2008
irow: 14500 of    17280 Mon Jan  7 14:16:27 2008
irow: 15000 of    17280 Mon Jan  7 14:16:27 2008
irow: 15500 of    17280 Mon Jan  7 14:16:27 2008
irow: 16000 of    17280 Mon Jan  7 14:16:28 2008
irow: 16500 of    17280 Mon Jan  7 14:16:28 2008
irow: 17000 of    17280 Mon Jan  7 14:16:29 2008
Setting Up: BinList 1073741919
     32
      1
/Users/desktop/lakes/superior/2005/spacebin/A2005033174000.L3_UMD_sLAC
/Users/desktop/lakes/superior/2005/spacebin/A2005033192000.L3_UMD_sLAC
% HDF_SD_START: Invalid HDF file or filename
(/Users/desktop/lakes/superior/2005/spacebin/A2005033174000.L3_UMD_sLAC).
% Execution halted at: CHECK_L3_FILE_FOR_PRODUCTS   18
/Applications/seadas5.1/idl_lib/check_l3_file_for_products.pro
%                      L3BIN              94
/Applications/seadas5.1/idl_lib/l3bin.pro
%                      L3BIN_TIMENEW      46
/Users/Desktop/lakes/superior/2005/spacebin/l3bin_timenew.pro
%                      $MAIN$
By mark Date 2008-01-07 21:44
Andrea,

I'd like for you to try using a new file that I have placed on our ftp site, and see whether that makes a difference.  It is on samoa.gsfc.nasa.gov in seadas/pub/andrea.  Get the get_swf_l3_prodnm.pro file and place it in your $SEADAS/idl_lib directory and try it again.  Let me know what happens.

Thanks, Mark
By Andrea Date 2008-01-07 22:57 Edited 2008-01-07 23:16
Mark, I downloaded the file and placed it in idl_lib. This time it ends at 29 days. I tried it a couple of times and it stopped at 29 days both runs.  Here is the error, same as before:
number of days processed      29
/Users/desktop/lakes/superior/2005/spacebin/A2005100181000.L3_UMD_sLAC
/Users/desktop/lakes/superior/2005/spacebin/A2005100195000.L3_UMD_sLAC
% HDF_SD_START: Invalid HDF file or filename (/Users/desktop/lakes/superior/2005/spacebin/A2005100181000.L3_UMD_sLAC).
% Execution halted at: CHECK_L3_FILE_FOR_PRODUCTS   18 /Applications/seadas5.1/idl_lib/check_l3_file_for_products.pro
%                      L3BIN              94 /Applications/seadas5.1/idl_lib/l3bin.pro
%                      L3BIN_TIMENEW      46 /Users/Desktop/lakes/superior/2005/spacebin/l3bin_timenew.pro
%                      $MAIN$         

Thanks for your help,
andrea
By mark Date 2008-01-09 19:58
Andrea,

We are still looking at this.  Will let you know when we find the problem.

Mark
By Andrea Date 2008-01-09 20:24
Mark, no worries, I figured as much. I am just restarting the program each time right now to cycle through the 365 days.
Thanks again,
Andrea
By mark Date 2008-01-15 20:56
Andrea,

We think we found the problem.  Go to samoa.gsfc.nasa.gov and look in seadas/pub/l3bin, get check_l3_file_for_products.pro and put it in your $SEADAS/idl_lib and $SEADAS/src/idl_procs/l3bin directories.  Try it again and let us know if that helps.

Regards, Mark
By Andrea Date 2008-01-16 21:04
Thanks Mark, it is chugging away now... flying past that 30 days point. What a relief! Thanks! It is so nice to not have to keep restarting it.
Many thanks,
andrea
Previous Next Up Topic SeaDAS / SeaDAS 6.x - General Questions / Seadas l3bin batch process stops after 30 days (locked) (3060 hits)



Responsible NASA Official: Gene C. Feldman
Curator: OceanColor Webmaster
Authorized by: Gene C. Feldman
Updated: 27 November 2007
Privacy Policy and Important Notices NASA logo