Overview of MODIS Aqua Data Processing and Distribution
Donna Thomas and Bryan Franz May 2005
I. Introduction
The Ocean Biology Processing Group (OBPG) is
responsible for the production and distribution of the ocean color data
products from the MODIS sensor on Aqua.This document provides details about the processing flow and data
archive and distribution. Figure 1 gives an overview of the MODIS data flow
within the OBPG.
Figure 1: Processing
flow from Level-0 through Level-2.
The processing flow describes two distinct branches: a
near real time stream and a refined stream.The near real time stream uses the best available ancillary
information to produce Level-2 products without delay.The products from the near real time
stream are replaced after 1-3 days with products from the refined processing
stream, which makes use of the best possible ancillary inputs to produce the
highest quality products.
For both the near real time or refined products, the
processing begins with Level-0 data acquired in 5-minute granules through the
EOS Data Operations System (EDOS) by way of the NOAA Near Real Time (NRT)
system.In most cases, the data is
received at OBPG within 6 hours of observation, and processed to Level-2 NRT
products within twenty minutes of receipt.Occasionally, granules are dropped between EDOS and NRT, and
in those cases the OBPG obtains the re-transmitted granules from the Goddard
Distributed Active Archive Center (GDAAC), after some delay.
II. Level-1A Processing
Processing from Level-0 to Level-1A is performed using the standard code developed by the
MODIS Science Data Support Team (SDST), known as MOD_PR01 (modis_l1agen in SeaDAS)
The output of MOD_PR01 is a 5-minute Level-1A granule in HDF
format.These standard Level-1A files
are then reduced in the process called MYD_L1A_SS, where excess bands and data
that are not utilized by Oceans are removed. The resulting Level 1A files are
smaller in size and easier to work with.A standard MODIS Level-1A product is 575MB (about 220MB compressed),
while the reduced Level-1A is 215MB (about 50MB compressed). The reduced Ocean
Level-1A file format retains the following band information:
|
MODIS
Band
|
Wavelength
(nm)
|
MODIS
Band
|
Wavelength
(um)
|
|
8
|
412
|
20
|
3.7
|
|
9
|
443
|
22
|
3.9
|
|
10
|
488
|
23
|
4.0
|
|
11
|
531
|
24
|
4.5
|
|
12
|
551
|
26
|
1.3
|
|
13lo
|
667
|
27
|
6.7
|
|
13hi
|
667
|
28
|
7.3
|
|
14lo
|
678
|
29
|
8.5
|
|
14hi
|
678
|
31
|
11
|
|
15
|
748
|
32
|
12
|
|
16
|
869
|
|
|
The format of the Ocean Level-1A file is near identical to
that of the standard (full) Level-1A file, except that the 250-meter and
500-meter band Science Data Sets are completely removed, and the 1-km bands not
listed in the table above are zeroed-out to enhance compressibility.
Beginning in February of 2005, a further modification was
made to the Ocean Level-1A format, to incorporate additional information from
the high-resolution MODIS land bands.
The primary goal of this change was to add the information needed to
produce a quasi true-color image, without increasing the size of the reduced
Level-1A file. The visible ocean bands (8-16) typically reach their 12-bit
saturation limit over land and clouds, making for poor quality red-green-blue
(RGB) composites.The Level-1A file format uses 16-bit integers to store the 12-bit data, so 4 bits of unused
space are available. Using correlation between the high resolution land bands
and the closest corresponding ocean bands, the counts stored in bands 10, 12,
13lo, and 16 were extended above their original 12-bit saturation limit up to
the 16-bit limit of the Level-1A format. This process is also performed by
MYD_L1A_SS.
The entire mission archive of Ocean Level-1A data is maintained online, and all
higher-level products are derived from this archive.The Level 1A file naming uses a SeaWiFS-like convention,
which indicates sensor (A for Aqua/MODIS, T for Terra/MODIS), sampling rate
(LAC for full 1-km sampling), and time of the first scan in the file
(Ayyyydddhhmmss.L1A_LAC).
III. Geolocation
The next step in the processing involves generation of the geolocation file (GEO).This is performed using standard SDST
code known as MOD_PR03 (geolocate in SeaDAS). For the NRT stream, the predicted
attitude and ephemeris files are used to produce Quick-Look GEO files.Several days later, in the Refined
processing stream, the definitive attitude and ephemeris files are used to
create the final GEO version.
GEO files are not maintained in the long-term archive, since they can be regenerated as
needed using the much smaller attitude and ephemeris files.Therefore, only a short-term rolling
archive is made available for distribution. The naming convention for GEO files
is similar to that of the Level-1A, e.g.: Ayyyydddhhmmss.GEO.
IV. Level-1B
After the geolocation step, the Level 1A file and the GEO file for each 5-minute granule
are fed into MYD_PR02 (aqua_l1bgen in SeaDAS) to
produce the corresponding Level-1B file.The standard MYD_PR02 code is
developed and maintained by the MODIS Calibration Support Team (MCST).The OBPG uses a slightly modified
version of MYD_PR02 to handle ocean subsetted Level-1A.The primary modification is to allow
processing to proceed when the high-resolution 250-meter and 500-meter SDS
fields are missing from the Level-1A file.Additional changes were made to handle the extended counts
in bands 10, 12, 13lo, and 16.The
standard Level-1B format for 1-km data includes a pair of SDS fields for
250-meter and 500-meter that have been aggregated to 1-km.Since the Ocean Level-1A does not
include the high-resolution bands, these aggregated fields would normally be
unfilled.The OBPG uses this free
space to store the extended ocean band information from bands 10, 12, 13lo, and
16, as these high reflectance values will not fit into the standard scaled
integer fields provided for the Level-1B reflectances (EV_1km_RefSB).
The MYD_PR02 Level-1B code makes use of an instrument calibration look-up table
(LUT).The LUT is derived from
offline analysis of the MODIS solar diffuser measurements, lunar observations,
and onboard lamps.MCST updates
these LUTs every few months, to improve tracking of the instrument calibration
changes.The OBPG does not use
these standard MCST LUTs or periodic updates in standard processing.To minimize the frequency of
calibration changes and potential discontinuities in the forward-stream
products, the OBPG generates an alternative LUT with a modified temporal
response trend, with hopes of improving extrapolation of instrument response
changes into the future.Deviations between the extrapolation and the measured instrument
response changes are monitored, and a reprocessing is initiated if the
deviations become too large.
V. Level-2
Level-2 processing is performed using the Multi-Sensor Level-1 to Level-2 (MSL12) code,
which is developed and maintained by the OBPG.MSL12 is used for the standard processing of all ocean
products distributed through the ocean color
web browsers. This software is capable of retrieving
oceanic optical properties and various derived products from the observed
top-of-atmosphere (TOA) radiances collected by a variety of ocean remote
sensing radiometers, including SeaWiFS, MODIS, OCTS, MOS, OSMI, POLDER, and
CZCS.Full documentation, updated
code source, sensor files, and output product descriptions can be found at
http://oceancolor.gsfc.nasa.gov/DOCS/MSL12/
For MODIS, the Level-1B file and the GEO file for each 5-minute granule
are input to MSL12.The Level-2 processing also makes use of meteorological and ozone information from ancillary sources. In the NRT stream, climatological meteorological and ozone data are used. The NRT Quick Look Level 1A, GEO, Level-2 products are then
immediately available on the ocean color browser and in the rolling archive within 6-10 hours of observation. When the definitive attitude and orbit and coincident meteorological and ozone data are received several days later, they are used in the Refined-processing stream and the Level-2 products are regenerated.
VI. Level-3
The Level-2 files are spatially averaged into Level-3
Daily binned files using the l2bin program developed by the OBPG (replaced
spacebin), and the Daily files are further composited into Weekly, Monthly,
Annual, Seasonal, and Climatological time periods using the l3bin code
(replaced timebin). The algorithms used for spatial and temporal binning are
described in Campbell,et al 1995.
The binned products are then used to generate mapped
products for chlorophyll, nLw(551), Angstrom(531), AOT(869), K(490), and
SST(11) on 4 and 9-km equi-rectangular projection.These image products are in standard mapped image (SMI)
format, and they are produced using the smigen program.
Processing of the NRT stream ends with the Daily
mapped products. The Weekly, Monthly, Annual, Seasonal, and Climatological
binned and mapped products are only produced in the Refined processing stream.
Figure 2: Data distribution for MODIS/Aqua Ocean products
produced by the OBPG.
VIII: Additional Details and Data Access
Standard operational products:
File format specifications: http://oceancolor.gsfc.nasa.gov/DOCS/ocformats.html
Data access:http://oceandata.sci.gsfc.nasa.gov
Latest reprocessing configuration:
http://oceancolor.gsfc.nasa.gov/REPROCESSING/
