Ocean Color Forum - Not logged in
Forum Ocean Color Home Help Search Login
Previous Next Up Topic Products and Algorithms / Satellite Data Products & Algorithms / L3 SMI SST 8-day composite has missing data with 4-pixel sampling (locked) (1611 hits)
By Easton Date 2012-04-13 15:23 Edited 2012-04-13 17:06
When I load the 8-day composites for L3 SST SMI files into SeaDas, I have to use the 4 pixel sample rate because SeaDas shuts down.  I compare this image to my ArcMap produced image of the same file (without the 4-pixel sample rate) to make sure that the values are comparable.  My comparison of Chlorophyll data looks great but the SST files differ.  In the SeaDas image there are large areas without data that just show a lower temperature (compared to the surrounding area) in the file directly imported into ArcMap.  I entered 65535 as the no data value, and there are some areas that show no data on both files.  Are the values from the 8-days averaged in the SMI file so that one day of no data will show up as a lower temp (lower value than 65535)?  I am extracting the sea surface tempertures in ArcMap, so I want to be confident in the values I extract before I use them in later analyses.  Thank you.
By WhiteG Date 2012-04-13 17:18
You don't give enough detail for us to help you.  SeaDAS loads such files at full resolution on an older iMac with 2G RAM, and memory usage peaks at about 600MB, so if you are using the virtual appliance with 512M or less that could be a problem.  

Normally, 8-day SMI files are generated from level-3 binned files, using well documented calculations.  You can dump the file headers to determine which input files were used and work back all the way to the level-2 files if you want to check for errors in the processing.

What do you mean by "shut down"?   Do you get a message from SeaDAS (maybe in the terminal where seadas was started)?   Are there messages in the system logs? What version of SeaDAS and what platform (linux or macosx version)?   How much memory (RAM) is installed?  For testing you can choose a smaller region to load at full resolution.  What SST quality level did you choose?   Are you using files downloaded from the oceancolor site or do you get them some other way?  What is the name of the file that has the problem (so others can try to reproduce it)? 

You can dump ASCII values for a small area to compare, and it might also help if you post some images that show the issue to the forum.  There are independent sources of SST such as Pathfinder 5.2 that could be used for comparisons.  If there is a problem with the data on the oceancolor site it may also show up in Giovanni.  
By Easton Date 2012-04-13 23:39
Sorry that I was unclear in my earlier post. 
One file I am using that has a large area off the southern coast of California with the problem above (apparent anomalously cold areas that correspond to no-data areas in images produced with 4-pixel sample rate in SeaDAS) is the MODIS AQUA SMI file downloaded from the OceanColor L3 browser is A20081932008200.L3m_8D_SST_4.  I am using SeaDAS on a Virtual Machine, but I am more worried about the raw data than the functions of SeaDAS.  Since your reply, I opened the BIN file (A20081932008200.L3b_8D_SST.main) for the same data to see if it shows the same anomalously cold areas when viewed in SeaDAS as the SMI file does in ArcMap.  It does show cooler areas near zones that have no data for the 8-day composite.  Could you direct me to a link that would tell me how to look for errors in the processing of the composites when compared to the L2 files?

The images for all of the files I looked at are below.  The second map from the left on the top row is the example from the SMI file listed above.

This image is of the SMI files at a 4-pixel sample rate.  I saved the GeoTiff from SeaDAS, and imported it into ArcMap.  Note the large black area along southern California.


This image is of the SMI files imported directly into ArcMap with the equation 0.000717*(l3m_data) - 2.0 to convert the raw data to temperature values (and 65535 as NO DATA).  Note the cooler areas (more blue) that mirror the black area in the image above.


This image is of the BIN file listed above and produced in SeaDAS.  Note the cooler area (darker) that corresponds to the same area off of California.  This image is consistent with second image in this post.


By comparing the first two images, you can see that this issue is persistent in these files.
By WhiteG Date 2012-04-14 12:50
The SMI file contains both l3m_data and l3m_qual "datasets" (the structure is easy to see using the HDFview application from the hdfgroup.org).   Have you tried using l3m_qual to mask out lower quality data in ArcMap?

It is not unusual to see areas in time-binned files with valuess higher or lower than adjacent areas.   This often happens if the overall region has a time trend and (due to clouds, etc) you end up with some sections of the image based on early pass dates and other based on later dates within the chosen interval.

Use ncdump_hdf to see the list of input files used to construct each file:

$ ncdump_hdf -h A20081932008200.L3b_8D_SST.main | grep Input\ Files | sed -e 's:/data4/sdpsoper/vdc/vpu3/workbuf/::g'
    :Input Files = "A2008196.L3b_DAY_SST.main,A2008198.L3b_DAY_SST.main,A2008193.L3b_DAY_SST.main,A2008195.L3b_DAY_SST.main,A2008197.L3b_DAY_SST.main,A2008199.L3b_DAY_SST.main,A2008200.L3b_DAY_SST.main,A2008194.L3b_DAY_SST.main" ;
By earnesteck Date 2012-05-14 21:20
Easton,
I am working with SST 8 day composites and attempting to get them into ARCGis. I see here that you are importing these files directly to ARC and would love to know if your data turned out to be good, and most importantly how you got them into ARC. I have been fighting with this for weeks now. So you know, I am not good at or understand python etc. so I would be doing it solely with the tools in the tool box. Thanks
Previous Next Up Topic Products and Algorithms / Satellite Data Products & Algorithms / L3 SMI SST 8-day composite has missing data with 4-pixel sampling (locked) (1611 hits)



Responsible NASA Official: Gene C. Feldman
Curator: OceanColor Webmaster
Authorized by: Gene C. Feldman
Updated: 27 November 2007
Privacy Policy and Important Notices NASA logo