Hi again.  Several processes (MODIS_geo.py, for example) have the ability to retrieve the necessary ancillary files on the fly by accessing the OBPG servers.  I'm running a script that goes through a lot of data files, and every once in an while, the ephemeris retrieval fails - see terminal output below - but no log files or any other indicators of failure are generated (other than the terminal output which I'm usually not sitting around watching).  These failures seem to be network related and the only way I've found to discover if there has been a problem is to go back and check to see if the proper ephemeris files are in their target directories (e.g. $SEADAS/run/var/modis/atteph/YYYY/DDD/).  The directories are created by MODIS_geo  even if the retrieval has failed, so I have to delete the directories with the bad or missing ephemeris files and then run MODIS_geo again for those days.  I guess I have three (the third is a little facetious) questions:

1.  Is there any way to capture an error code or something from the MODIS_geo process (and other similar ones) that could be used in my shell script to delete the just-created directory and try the retrieval again?

2.  Is there any way to add a retry parameter or extend the time-out cutoff in the process that contacts the server to get past the network problems that seem to cause the failures?

3.  Is this a reason not to ignore the ancillary files that are provided with data orders (as I have done because I figured it was easier to let the processes load the files they needed into the proper places rather than download them with the requested_files and then sort them into where they belong??)

Thanks, Barry

Working on A2008077181000.L1A_LAC.x.hdf.bz2
Unzipping A2008077181000.L1A_LAC.x.hdf.bz2
A2008077181000.L1A_LAC.x.hdf.bz2 unzipped to A2008077181000.L1A_LAC.x.hdf
We failed to reach a server.
URL attempted: http://oceandata.sci.gsfc.nasa.gov/cgi/getfile/PM1EPHND.P2008077.1200.003
Reason:  timed out
Traceback (most recent call last):
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modis_GEO.py", line 118, in <module>
    m.atteph()
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modules/modis_GEO_utils.py", line 193, in atteph
    get.locate()
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modules/anc_utils.py", line 445, in locate
    if re.search('^\d{.*}$', status):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer
Done
Working on A2008078185500.L1A_LAC.x.hdf.bz2
Unzipping A2008078185500.L1A_LAC.x.hdf.bz2
A2008078185500.L1A_LAC.x.hdf.bz2 unzipped to A2008078185500.L1A_LAC.x.hdf
Traceback (most recent call last):
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modis_GEO.py", line 118, in <module>
    m.atteph()
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modules/modis_GEO_utils.py", line 193, in atteph
    get.locate()
  File "/Volumes/1TBSATDATA/sw/seadas6.3/run/scripts/modules/anc_utils.py", line 444, in locate
    uncompress=True)
  File "/Users/seadas/seadas6.3/run/scripts/modules/ProcUtils.py", line 45, in httpdl
    shutil.copyfileobj(response, file)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/shutil.py", line 27, in copyfileobj
    buf = fsrc.read(length)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 351, in read
    data = self._sock.recv(left)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/httplib.py", line 537, in read
    s = self.fp.read(amt)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/socket.py", line 351, in read
    data = self._sock.recv(left)
socket.timeout: timed out
Done

Barry,

1) Yes.  An attempt to do that is in there, but it's broke - thus the error spew you see.  A fix is in the works.

2) Likely a yes.  We'll investigate

3) I'd say no, still OK to ignore the files staged with the order. 
Yesterday was a bad day for the OBPG webservers - a very anomalous condition resulted in the
webservers being very unhappy and dropping connections.   This is NOT a regular occurrence.

Sean

Thanks, Sean. Have a good weekend.  Barry

Hello again to my favorite SeaDAS experts,

I think I have a similar problem, except that it doesn't seem to be related to the network connection, because I don't get the "network connection" error in the output.

Terminal output:

Determining required attitude and ephemeris files...
Searching database: /home/ack/Desktop/seadas6.3/log/ancillary_data.db

Input file: /media/FAT_1500/2007/A2007003112000.L1A_LAC
Start time: 2007003112000
End time: 2007003112459

Traceback (most recent call last):
  File "/home/ack/Desktop/seadas6.3/run/scripts/modis_GEO.py", line 118, in <module>
    m.atteph()
  File "/home/ack/Desktop/seadas6.3/run/scripts/modules/modis_GEO_utils.py", line 226, in atteph
    self.attfile1 = os.path.basename(get.files['att1'])
KeyError: 'att1'

NOTE:  I have 'modis_GEO_utils.pyc', not  'modis_GEO_utils.py' --> not sure if that makes a difference. I had the attitude & ephemeris files in the correct (default) directory but it appears that the problem occurs even before we get to accessing the 'PM...' file because deleting the file doesn't change the error output... unless I simply don't have the right att and eph files for some reason. (I used the auto-download as recommended.)

Suggestions?

Thank you again for all of your help.

Best,
Allyn

It may be that the local ancillary database doesn't have the full list of files.
Try running:

$SEADAS/run/scripts/modis_atteph.py --refreshDB A2007003112000.L1A_LAC

Or try upgrading to SeaDAS6.4

Sean

Thanks Sean, upgrading to 6.4 did the trick and all has been running very smoothly! ...Until now.

My harddrive (where I have my seadas home directory) has very little free memory left, which I think might be interfering with downloading the attephm files. I'm trying to see if moving my seadas home directory to my external harddrive will fix it, but since that is taking so long for some unknown reason (and late on a Friday night), I thought I'd ask you, too, because for all I know, this may be a python error or some glitch in the new version.

Once again, I appreciate all of your help.

Best,
Allyn

TERMINAL OUTPUT:

GENERATING NEW GEOFILE...
Determining required attitude and ephemeris files...
Searching database: /home/ack/Desktop/seadas6.4/run/var/ancillary_data.db

Input file: /media/NTFS_3000/files/redo_full/round1/A2004198110000.L1A_LAC
Start time: 2004198110000
End time: 2004198110459

Traceback (most recent call last):
  File "/home/ack/Desktop/seadas6.4/run/scripts/modis_GEO.py", line 125, in <module>
    m.atteph()
  File "/home/ack/Desktop/seadas6.4/run/scripts/modules/modis_GEO_utils.py", line 207, in atteph
    get.findweb()
  File "/home/ack/Desktop/seadas6.4/run/scripts/modules/anc_utils.py", line 387, in findweb
    atteph=self.atteph)
  File "/home/ack/Desktop/seadas6.4/run/scripts/modules/ancDB.py", line 96, in insert_record
    c.execute('UPDATE satfiles set attephstat = ?', [dbstat])
sqlite3.OperationalError: disk I/O error

An error of disk I/O isn't a python problem.  It suggests there is an issue with the disk.  If it was a space issue you'd get a different error (no space left on device).
Sean

Hmm. Ok. How about this one?

Processing MODIS L1A file to L1B...
MYD_PR02 version 6.1.15b built on Jun  1 2012, at 12:34:49
/home/ack/Desktop/seadas6.4/run/bin/linux/l1bgen_modisa exit status: 1
ERROR: MODIS L1B processing failed.
Please examine the LogStatus and LogUser files for more information.

Thanks again,
Allyn

Note: The LogUser file doesn't add any information not already in the LogStatus file (matches the 1st part).

The log file is telling you exactly what the problem is:

Read_LUT_Tables():MODIS_F_FILE_NOT_OPENED:295817228
Could not open file for SD read access.
Call to SDstart() failed.
File LUN:  700050
File type: Reflective_Lookup_Tables_file
The file may be missing, corrupted or not an HDF-4 file.

If the disk is going bad, what appear to be strange things can happen,
so if processing once worked, but now doesn't, there are some things you
can do to help figure out why.

The PCF file (which should have been retained if the process failed) will tell you
the path to LUN 700050 (one of  the calibration LUTs).  Check the path,
which should be under $SEADAS/run/var/modisa/cal/OPER/
should look something like:
# Lookup Tables
700050|MYD02_Reflective_LUTs.V6.1.15.4.hdf|/Users/Shared/SeaDAS/seadas6.4/run/var/modisa/cal/OPER||MYD02_Reflective_LUTs.V6.1.15.4.hdf||1
700060|MYD02_Emissive_LUTs.V6.1.15.4.hdf|/Users/Shared/SeaDAS/seadas6.4/run/var/modisa/cal/OPER||MYD02_Emissive_LUTs.V6.1.15.4.hdf||1
700070|MYD02_QA_LUTs.V6.1.15.4.hdf|/Users/Shared/SeaDAS/seadas6.4/run/var/modisa/cal/OPER||MYD02_QA_LUTs.V6.1.15.4.hdf||1

Make sure the path exists and that the result of the "file" command indicates the file is an HDF4 file:

e.g.
file /Users/Shared/SeaDAS/seadas6.4/run/var/modisa/cal/OPER/MYD02_QA_LUTs.V6.1.15.4.hdf
/Users/Shared/SeaDAS/seadas6.4/run/var/modisa/cal/OPER/MYD02_QA_LUTs.V6.1.15.4.hdf: Hierarchical Data Format (version 4) data

If it doesn't report the file to be an HDF4 file, then somehow the file was damaged.

Sean

Everything checked out with the HDF4 files. You're probably right about weird errors being related to some problem with the disk. I ended up creating a new VA and everything is running fine. Thanks again for your help!

AK