Ocean Color Forum - Not logged in
By Andrea
Date 2008-01-03 23:25
Edited 2008-01-04 20:33
Hello, I am running the newest SEADAS version for Intel MAC, with the non-embedded IDL version. I was using the embedded version before I purchased IDL and ran into the same problem when SEADAS halts after running 30 days of time-binned files. I bought IDL hoping to processes large data sets. Could you please tell me how to either change the code or how to run the program so that it does not stop at 30 days? In the past I ran the same code on a Unix machine and I have never had it stop after the 30th day. Is this a MAC problem?
Here is my sample code and I am using the %seadas -b l3bin_timenew command to start the processing. (with a shell script called l3bin_timenew)
Many thanks!
PRO l3bin_timenew, INDIR=indir, OUTDIR=outdir
IF (NOT(KEYWORD_SET(INDIR))) THEN indir ='/desktop/lakes/superior/2002/spacebin/'
IF (NOT(KEYWORD_SET(OUTDIR))) THEN outdir='/desktop/lakes/superior/2002/timebin/'
; VARIABLES FOR BINNING
;
mysyear='2002'
mysday=185
myeyear='2002'
myeday=336
step=1
j=-1
for x=mysday,myeday-1,step DO BEGIN
daystring=STRTRIM(STRING(x),1)
IF strlen(daystring) NE 3 THEN daystring='0'+daystring
IF strlen(daystring) NE 3 THEN daystring='0'+daystring
myfiles=findfile(indir+'A'+mysyear+daystring+'*UMD*')
;this section prints only the day of interest to the inlist.txt file that is executed in the l3bin command below
print, myfiles
openw, lun, 'inlist.txt', /get_lun
printf,lun, myfiles
free_lun,lun
outfile=outdir+'/A'+mysyear+'_'+daystring+'_'+'L3b_UMD'
compressed=outfile+'.bz2'
;MAIN PROCESSING LINE
checkfile=findfile(outfile) & IF checkfile[0] ne "" THEN PRINT, 'File Exists! Skipping...' ELSE BEGIN
checkfile=findfile(compressed) & IF checkfile[0] ne "" THEN PRINT, 'File Exists! Skipping...' ELSE BEGIN
infile=indir+'inlist.txt'
l3bin, ifile=infile, ofile=outfile, l3bprod=['sst'], noext=1,latsouth=46.0000, latnorth=49.5000, lonwest=-94.0000, loneast=-83.00
endelse
endelse
print, x
print, mysday
endfor
;SPAWN,'rm '+filelist
end
By mark
Date 2008-01-07 19:16
Andrea,
It is possible that there is a problem in our l3bin code that has not been detected yet. But I am curious for you to try one thing first. Can you change your call to free_lun to force the file close, and maybe that will fix the problem? You can do it like this:
free_lun,lun,/force
I'm thinking that maybe there is some error occuring when free_lun tries to close the file. The /force flag forces it to close regardless of error condition. If that does not work, can you please post the entire output error message?
Regards, Mark
By Andrea
Date 2008-01-07 20:35
Thanks Mark, I tried that and received the same error. The number 32 at the end is the number of days it cycled through and 1 is the day it started with.
l3bin.pro: cmd_final =
/bin/bash -c "l3bin
IN=/Users/desktop/lakes/superior/2005/spacebin/inlist.txt
OUT=/Users/desktop/lakes/superior/2005/timebin//A2005_032_L3b_UMD
latsouth=46.0000 latnorth=49.5000 lonwest=-94.0000 loneast=-83.0000
noext=1 OUT_PARM='sst'"
L3BIN 2.17 (Aug 22 2007 12:14:17)
1 input files
0 /Users/desktop/lakes/superior/2005/spacebin/A2005032183500.L3_UMD_sLAC
open status: 0
nbins: 380187134
nbins: 380187134
Setting Up: SEAGrid 1073741918
Writing 1 records to SEAGrid (1)
Detaching: SEAGrid
irow: 0 of 17280 Mon Jan 7 14:16:14 2008
irow: 500 of 17280 Mon Jan 7 14:16:14 2008
irow: 1000 of 17280 Mon Jan 7 14:16:15 2008
irow: 1500 of 17280 Mon Jan 7 14:16:15 2008
irow: 2000 of 17280 Mon Jan 7 14:16:16 2008
irow: 2500 of 17280 Mon Jan 7 14:16:16 2008
irow: 3000 of 17280 Mon Jan 7 14:16:17 2008
irow: 3500 of 17280 Mon Jan 7 14:16:17 2008
irow: 4000 of 17280 Mon Jan 7 14:16:18 2008
irow: 4500 of 17280 Mon Jan 7 14:16:18 2008
irow: 5000 of 17280 Mon Jan 7 14:16:18 2008
irow: 5500 of 17280 Mon Jan 7 14:16:19 2008
irow: 6000 of 17280 Mon Jan 7 14:16:19 2008
irow: 6500 of 17280 Mon Jan 7 14:16:20 2008
irow: 7000 of 17280 Mon Jan 7 14:16:20 2008
irow: 7500 of 17280 Mon Jan 7 14:16:20 2008
irow: 8000 of 17280 Mon Jan 7 14:16:21 2008
irow: 8500 of 17280 Mon Jan 7 14:16:21 2008
irow: 9000 of 17280 Mon Jan 7 14:16:22 2008
irow: 9500 of 17280 Mon Jan 7 14:16:22 2008
irow: 10000 of 17280 Mon Jan 7 14:16:23 2008
irow: 10500 of 17280 Mon Jan 7 14:16:23 2008
irow: 11000 of 17280 Mon Jan 7 14:16:23 2008
irow: 11500 of 17280 Mon Jan 7 14:16:24 2008
irow: 12000 of 17280 Mon Jan 7 14:16:24 2008
irow: 12500 of 17280 Mon Jan 7 14:16:25 2008
irow: 13000 of 17280 Mon Jan 7 14:16:25 2008
irow: 13500 of 17280 Mon Jan 7 14:16:25 2008
irow: 14000 of 17280 Mon Jan 7 14:16:26 2008
irow: 14500 of 17280 Mon Jan 7 14:16:27 2008
irow: 15000 of 17280 Mon Jan 7 14:16:27 2008
irow: 15500 of 17280 Mon Jan 7 14:16:27 2008
irow: 16000 of 17280 Mon Jan 7 14:16:28 2008
irow: 16500 of 17280 Mon Jan 7 14:16:28 2008
irow: 17000 of 17280 Mon Jan 7 14:16:29 2008
Setting Up: BinList 1073741919
32
1
/Users/desktop/lakes/superior/2005/spacebin/A2005033174000.L3_UMD_sLAC
/Users/desktop/lakes/superior/2005/spacebin/A2005033192000.L3_UMD_sLAC
% HDF_SD_START: Invalid HDF file or filename
(/Users/desktop/lakes/superior/2005/spacebin/A2005033174000.L3_UMD_sLAC).
% Execution halted at: CHECK_L3_FILE_FOR_PRODUCTS 18
/Applications/seadas5.1/idl_lib/check_l3_file_for_products.pro
% L3BIN 94
/Applications/seadas5.1/idl_lib/l3bin.pro
% L3BIN_TIMENEW 46
/Users/Desktop/lakes/superior/2005/spacebin/l3bin_timenew.pro
% $MAIN$
By mark
Date 2008-01-07 21:44
Andrea,
I'd like for you to try using a new file that I have placed on our ftp site, and see whether that makes a difference. It is on samoa.gsfc.nasa.gov in seadas/pub/andrea. Get the get_swf_l3_prodnm.pro file and place it in your $SEADAS/idl_lib directory and try it again. Let me know what happens.
Thanks, Mark
By Andrea
Date 2008-01-07 22:57
Edited 2008-01-07 23:16
Mark, I downloaded the file and placed it in idl_lib. This time it ends at 29 days. I tried it a couple of times and it stopped at 29 days both runs. Here is the error, same as before:
number of days processed 29
/Users/desktop/lakes/superior/2005/spacebin/A2005100181000.L3_UMD_sLAC
/Users/desktop/lakes/superior/2005/spacebin/A2005100195000.L3_UMD_sLAC
% HDF_SD_START: Invalid HDF file or filename (/Users/desktop/lakes/superior/2005/spacebin/A2005100181000.L3_UMD_sLAC).
% Execution halted at: CHECK_L3_FILE_FOR_PRODUCTS 18 /Applications/seadas5.1/idl_lib/check_l3_file_for_products.pro
% L3BIN 94 /Applications/seadas5.1/idl_lib/l3bin.pro
% L3BIN_TIMENEW 46 /Users/Desktop/lakes/superior/2005/spacebin/l3bin_timenew.pro
% $MAIN$
Thanks for your help,
andrea
By mark
Date 2008-01-09 19:58
Andrea,
We are still looking at this. Will let you know when we find the problem.
Mark
By Andrea
Date 2008-01-09 20:24
Mark, no worries, I figured as much. I am just restarting the program each time right now to cycle through the 365 days.
Thanks again,
Andrea
By mark
Date 2008-01-15 20:56
Andrea,
We think we found the problem. Go to samoa.gsfc.nasa.gov and look in seadas/pub/l3bin, get check_l3_file_for_products.pro and put it in your $SEADAS/idl_lib and $SEADAS/src/idl_procs/l3bin directories. Try it again and let us know if that helps.
Regards, Mark
By Andrea
Date 2008-01-16 21:04
Thanks Mark, it is chugging away now... flying past that 30 days point. What a relief! Thanks! It is so nice to not have to keep restarting it.
Many thanks,
andrea