I have downloaded some seasonal climatologies HDFs (9km) from http://oceancolor.gsfc.nasa.gov/cgi/climatologies.pl and I am having serious difficulties translating then importing them into ArcGIS 8.3. 

After bunzipping them and viewing the HDFs in ERMapper they look visually OK but ERMapper does not seem to recognize the fact there is a projection to the data nor the cell size of 9km.  I have exported them out of ERMapper as both a GeoTiff and BIL (Binary Interleaved by Line) which produce world files and/or header files.  I can edit these files to reflect the 9km cell size and later set the projection in ArcGIS to Equidistant Cylindrical which according to http://oceancolor.gsfc.nasa.gov/DOCS/ocformats.htm should be correct.  However, after importing to ArcGIS, the climatologies data do not seem to be lining up correctly with other data that I have in my GIS and it is clear that some projection issue remains, I am obviously doing something wrong. 

So I had a few questions.  Have others had problems of bringing HDF data into a GIS at the global scale ?  Are the seasonal climatologies in an Equidistant Cylindrical projection ?  - The original Level 3 data start out in the Sinusoidal.  If the HDFs are in Equidistant Cylindrical, what are the Standard Parallel, Prime Meridian, Datum, and Geographic Coordinate System for this implementation of the projection?  Also, are the pixels exactly 9km on a side ?  Finally, is there a 4km resolution product ?

Thanks !

-Don

Hi Don,

You are not the first to have had trouble with our mapped data.
The data are stored in an equidistant cylindrical projection having
the equator as the standard parallel.  This projection is also known
by the names  simple cylindrical and Plate Carrée.  The prime meridian
is at Greenwich.  At 9km resolution your choice of datum will make
no noticeable difference, and you may as well use a spherical model
of the Earth.  Also, you will not find any projection of the Earth (including
this one) whose pixels are exactly nine kilometers on a side;  nor is this
projection equal area or conformal -- having high amounts of distorsion
in the high latitudes.

To find the center lat/lon of any pixel having x/y coordinates where
y goes from 0 to 2159 (north to south) and x goes from 0 to 4319 (west
to east), use the following.


incr = 360/4320
lat =   90 - y*incr - incr/2
lon = -180 + x*incr + incr/2


We only provide 9 kilometer resolution for SeaWiFS level-3 products;
MODIS products, however, also come in nominal 4 kilometer resolution.

Regards,

Norman

Hi All,

Thanks for the reply Norman but I still seem to be having troubles and it gets deeper all the time. 

I had read elsewhere Plate Carrée is equivalent to equidistant cylindrical projection and actually tried that before and it still did not work properly.  After bringing the HDF to BIL (or a GeoTIFF) and then the image into ArcGIS, the rest of my data is shifted approximately 21,000km to the North and West.    This occurs even if I reproject my GIS data to make sure it is Plate Carrée.  The shapes of the continents seem to be the same for the two datasets so I believe the projections are the same, it seems to me that the origin is somehow shifted. 

What points me to this direction is that when you look at the equator, greenwich on my GIS data the coordinates are 0 00 00E and 0 00 00N while the HDF imported file is at 180 00 00W and 90 00 00S.  My GIS header requires the Upper Left X and Upper Left Y.  What would the Upper X and Upper Y coordinate be in meters for the imagery?  I have fooled around with this for a couple of days and am stumped.  I tried to convert my GIS data to a BIL or GeoTiff and look at the header information and it says the ULX and ULY should be (-20033007.06716184300000, 9302768.33361105430000).  According to the information you gave me the ULX and ULY of the image should be -180Long, 90Lat.  Any ideas ?

So I figured that if I got to look at the HDF header I might get a better understanding of what is going on.  I downloaded HEGTools (v2.6) and it does not recognize the HDF files the error I get is "Can not load c:\filename.hdf"  Error:  unknown error".  So are the oceancolor HDF in a non-standard HDF ?  It seems wierd that ERMapper can not read the appropriate HDF header information which I thought tells the software what is the projection and pixel size.  Any ideas ?

Hi Don,

The Plate Carrée projection that we use is a specific instance of the
equidistant cylindrical projection where the equator is the standard parallel
and the graticule forms a grid of squares i.e. the spacing between parallels
is the same as the spacing between meridians.

I suspect that the software that you are using to read the data is misinterpreting
our HDF files which are in HDF version 4 (not HDF-EOS which I understand the
HEG Tools were designed to read).  Standard HDF tools from NCSA like hdp
work fine with our HDF files.  You also may want to consider getting the SeaDAS
package which is designed specifically to work with the data that we produce.

You also seem to be confusing latitude and longitude values,
which have units of degrees in the formulae I mentioned earlier,
with Cartesian coordinates such as the ULX and ULY you
mentioned, which commonly are given in meters in the cartographic
literature.

I'm not sure I can be any clearer about the image projection.
Does anyone with ERMapper experience have any hints for Don?

Regards,

Norman

Hi All,

Thanks again Norman for your assistance I think I FINALLY got it and it was truly painful. After much to much time experimenting,
downloading HDF viewers, and other ancillary programs it turned out to be an apples and oranges type of thing.  BTW, I only use ERMapper because it can read HDF files, my real goal is to get the data into ArcGIS, it would be nice to find a HDF4->GeoTiff converter but I guess a lot of people have asked for that ....  I have found a HDF4->JPG

First off, I can't use SeaDAS as I have a PC and don't really have any interest in LINUX as an OS.  I have been able to view the HDF using an HDF Viewer (NCSA HDFView 2.3 but that doesn't convert the HDF files)... 

So, going from HDF to GIS seems to be a problem that a lot of people have had.  GeoTIFF is an easy file format for ArcGIS to import so that is why I have been trying to use it.  I searched on the forum's list (and the WWW) and found a couple of suggestions however, this post is only relevant for HDF-EOS so that won't work (http://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?pid=534;hlm=adv;hl=geotiff#pid534 ) and all the rest suggest using SeaDAS (see above). 

The key to getting GeoTiff data into ArcGIS is moving from image space (2160 columns by 4320 rows of square pixels) to real-world coordinate space defined with a map projection.  The GeoTiff world file (tfw) is used for georeferencing purposes and has the following format:
Line Number  Values
Line1            Dimension of pixels in X direction
Line2            Rotation in X direction
Line3            Rotation in Y direction
Line4            Dimension of pixels in Y direction
Line5            Insertion point in X direction
Line6            Insertion point in Y direction

ERMapper seems to read the HDF fine, but not any of the associated metadata and/or projection info so an HDF comes in as a 2160 x 4320 raw image, not as a Plate Carre projected file.  I then set the ERMapper import parameters so the HDF is projected in Geographic (Lat/Long) with Decimal Degrees as the units with the pixel size to .083 x .083 and the origin as -180.0, 90.  Then I output the file as a GeoTiff (with a tfw, the header info is below) and now I am finally done !

tfw (Tiff World File)
0.083333333333332954
0
0
-0.083329999999999974
-179.96527777777774
89.965279166666633

Boy it was a long process and I have read on the internet that people have a little bit of a problem with HDF-EOS but I could find no where that people were going from HDF4 to GeoTiff.

-Don

Don,

I appreciate you posting these details.  I suspect that many people have gone through this same process or similar, but they never share the solution.  We have no knowledge 'round here of ArcGIS or even GeoTIFF, but I was suspecting that GeoTIFF is a simple format to master.  I believe it would be relatively trivial for us to develop the converter to go from our HDF4 maps to GeoTIFF. It would be great if you could post your GeoTIFF version of one of our HDF4 files (somewhere), so we have a working example. 

Of course, we'd still have the OS problem, 'cause while you have no interest in Linux, we are not even allowed to have Windows machines on the project due to security risks.   It is possible, though, that we could offer an online converter, as well as the source code.

-- Bryan

I have tried my hand at turning our SMI files into Geotiffs.
The following shows one way to do it.  This method requires
these freely available software tools.

  bunzip     (http://www.bzip.org/)
  hdp        (http://hdf.ncsa.uiuc.edu/hdp.html)
  rawtopgm   (http://netpbm.sourceforge.net/doc/rawtopgm.html)
  pnmtotiff  (http://netpbm.sourceforge.net/doc/pnmtotiff.html)
  geotifcp   (http://www.remotesensing.org/geotiff/geotifcp.html)


The following sequence assumes a unix user.
I started with the following two sample SMI files.

  http://oceancolor.gsfc.nasa.gov/cgi/getfile/A20053052005334.L3m_MO_CHLO_9.bz2
  http://oceancolor.gsfc.nasa.gov/cgi/getfile/A20053052005334.L3m_MO_CHLO_4.bz2

bunzip A20053052005334.L3m_MO_CHLO_9.bz2
hdp dumpsds -n l3m_data -b -d -o A20053052005334.L3m_MO_CHLO_9.raster A20053052005334.L3m_MO_CHLO_9
rawtopgm 4320 2160 A20053052005334.L3m_MO_CHLO_9.raster | pnmtotiff > A20053052005334.L3m_MO_CHLO_9.plain.tif
geotifcp -g Geotiff_metadata.9kmSMI A20053052005334.L3m_MO_CHLO_9.plain.tif A20053052005334.L3m_MO_CHLO_9.tif

bunzip A20053052005334.L3m_MO_CHLO_4.bz2
hdp dumpsds -n l3m_data -b -d -o A20053052005334.L3m_MO_CHLO_4.raster A20053052005334.L3m_MO_CHLO_4
rawtopgm 8640 4320 A20053052005334.L3m_MO_CHLO_4.raster | pnmtotiff > A20053052005334.L3m_MO_CHLO_4.plain.tif
geotifcp -g Geotiff_metadata.4kmSMI A20053052005334.L3m_MO_CHLO_4.plain.tif A20053052005334.L3m_MO_CHLO_4.tif


You can find the two Geotiff metadata files I used here.

  Geotiff_metadata.9kmSMI
  Geotiff_metadata.4kmSMI

(These are both more or less human readable text files.)

You can also get the resulting Geotiff images (grayscale only) here.

  A20053052005334.L3m_MO_CHLO_9.tif
  A20053052005334.L3m_MO_CHLO_4.tif


Regards,

Norman

Dear Don,

First of all I would say I really appreciate you posted your experienced here. At the moment I am also facing the same problem to open the data in ArcGis. I'm dealing with Level 2 SST and chlorophyll data. I need to have the data in the same projection before analysing it. I am not really understand about your explanation below :

Line1            Dimension of pixels in X direction
Line2            Rotation in X direction
Line3            Rotation in Y direction
Line4            Dimension of pixels in Y direction
Line5            Insertion point in X direction
Line6            Insertion point in Y direction

Could you explain a bit more or can we communicate through email?

Thank you