Overview of MODIS Aqua Data Processing and Distribution

Donna Thomas and Bryan Franz May 2005

I. Introduction

The Ocean Biology Processing Group (OBPG) is responsible for the production and distribution of the ocean color data products from the MODIS sensor on Aqua.This document provides details about the processing flow and data archive and distribution. Figure 1 gives an overview of the MODIS data flow within the OBPG.

Figure 1: Processing flow from Level-0 through Level-2.

The processing flow describes two distinct branches: a near real time stream and a refined stream.The near real time stream uses the best available ancillary information to produce Level-2 products without delay.The products from the near real time stream are replaced after 1-3 days with products from the refined processing stream, which makes use of the best possible ancillary inputs to produce the highest quality products.

For both the near real time or refined products, the processing begins with Level-0 data acquired in 5-minute granules through the EOS Data Operations System (EDOS) by way of the NOAA Near Real Time (NRT) system.In most cases, the data is received at OBPG within 6 hours of observation, and processed to Level-2 NRT products within twenty minutes of receipt.Occasionally, granules are dropped between EDOS and NRT, and in those cases the OBPG obtains the re-transmitted granules from the Goddard Distributed Active Archive Center (GDAAC), after some delay.

II. Level-1A Processing

Processing from Level-0 to Level-1A is performed using the standard code developed by the MODIS Science Data Support Team (SDST), known as MOD_PR01 (modis_l1agen in SeaDAS) The output of MOD_PR01 is a 5-minute Level-1A granule in HDF format.These standard Level-1A files are then reduced in the process called MYD_L1A_SS, where excess bands and data that are not utilized by Oceans are removed. The resulting Level 1A files are smaller in size and easier to work with.A standard MODIS Level-1A product is 575MB (about 220MB compressed), while the reduced Level-1A is 215MB (about 50MB compressed). The reduced Ocean Level-1A file format retains the following band information:

*MODIS Band*	*Wavelength (nm)*	*MODIS Band*	*Wavelength (um)*
8	412	20	3.7
9	443	22	3.9
10	488	23	4.0
11	531	24	4.5
12	551	26	1.3
13lo	667	27	6.7
13hi	667	28	7.3
14lo	678	29	8.5
14hi	678	31	11
15	748	32	12
16	869

The format of the Ocean Level-1A file is near identical to that of the standard (full) Level-1A file, except that the 250-meter and 500-meter band Science Data Sets are completely removed, and the 1-km bands not listed in the table above are zeroed-out to enhance compressibility. Beginning in February of 2005, a further modification was made to the Ocean Level-1A format, to incorporate additional information from the high-resolution MODIS land bands. The primary goal of this change was to add the information needed to produce a quasi true-color image, without increasing the size of the reduced Level-1A file. The visible ocean bands (8-16) typically reach their 12-bit saturation limit over land and clouds, making for poor quality red-green-blue (RGB) composites.The Level-1A file format uses 16-bit integers to store the 12-bit data, so 4 bits of unused space are available. Using correlation between the high resolution land bands and the closest corresponding ocean bands, the counts stored in bands 10, 12, 13lo, and 16 were extended above their original 12-bit saturation limit up to the 16-bit limit of the Level-1A format. This process is also performed by MYD_L1A_SS.

The entire mission archive of Ocean Level-1A data is maintained online, and all higher-level products are derived from this archive.The Level 1A file naming uses a SeaWiFS-like convention, which indicates sensor (A for Aqua/MODIS, T for Terra/MODIS), sampling rate (LAC for full 1-km sampling), and time of the first scan in the file (Ayyyydddhhmmss.L1A_LAC).

III. Geolocation

The next step in the processing involves generation of the geolocation file (GEO).This is performed using standard SDST code known as MOD_PR03 (geolocate in SeaDAS). For the NRT stream, the predicted attitude and ephemeris files are used to produce Quick-Look GEO files.Several days later, in the Refined processing stream, the definitive attitude and ephemeris files are used to create the final GEO version. GEO files are not maintained in the long-term archive, since they can be regenerated as needed using the much smaller attitude and ephemeris files.Therefore, only a short-term rolling archive is made available for distribution. The naming convention for GEO files is similar to that of the Level-1A, e.g.: Ayyyydddhhmmss.GEO.

IV. Level-1B

After the geolocation step, the Level 1A file and the GEO file for each 5-minute granule are fed into MYD_PR02 (aqua_l1bgen in SeaDAS) to produce the corresponding Level-1B file.The standard MYD_PR02 code is developed and maintained by the MODIS Calibration Support Team (MCST).The OBPG uses a slightly modified version of MYD_PR02 to handle ocean subsetted Level-1A.The primary modification is to allow processing to proceed when the high-resolution 250-meter and 500-meter SDS fields are missing from the Level-1A file.Additional changes were made to handle the extended counts in bands 10, 12, 13lo, and 16.The standard Level-1B format for 1-km data includes a pair of SDS fields for 250-meter and 500-meter that have been aggregated to 1-km.Since the Ocean Level-1A does not include the high-resolution bands, these aggregated fields would normally be unfilled.The OBPG uses this free space to store the extended ocean band information from bands 10, 12, 13lo, and 16, as these high reflectance values will not fit into the standard scaled integer fields provided for the Level-1B reflectances (EV_1km_RefSB). The MYD_PR02 Level-1B code makes use of an instrument calibration look-up table (LUT).The LUT is derived from offline analysis of the MODIS solar diffuser measurements, lunar observations, and onboard lamps.MCST updates these LUTs every few months, to improve tracking of the instrument calibration changes.The OBPG does not use these standard MCST LUTs or periodic updates in standard processing.To minimize the frequency of calibration changes and potential discontinuities in the forward-stream products, the OBPG generates an alternative LUT with a modified temporal response trend, with hopes of improving extrapolation of instrument response changes into the future.Deviations between the extrapolation and the measured instrument response changes are monitored, and a reprocessing is initiated if the deviations become too large.

V. Level-2

Level-2 processing is performed using the Multi-Sensor Level-1 to Level-2 (MSL12) code, which is developed and maintained by the OBPG.MSL12 is used for the standard processing of all ocean products distributed through the ocean color web browsers and ftp sites. This software is capable of retrieving oceanic optical properties and various derived products from the observed top-of-atmosphere (TOA) radiances collected by a variety of ocean remote sensing radiometers, including SeaWiFS, MODIS, OCTS, MOS, OSMI, POLDER, and CZCS.Full documentation, updated code source, sensor files, and output product descriptions can be found at http://oceancolor.gsfc.nasa.gov/DOCS/MSL12/ For MODIS, the Level-1B file and the GEO file for each 5-minute granule are input to MSL12.The Level-2 processing also makes use of meteorological and ozone information from ancillary sources. In the NRT stream, climatological meteorological and ozone data are used. The NRT Quick Look Level 1A, GEO, Level-2 products are then immediately available on the ocean color browser and in the rolling ftp archive within 6-10 hours of observation. When the definitive attitude and orbit and coincident meteorological and ozone data are received several days later, they are used in the Refined-processing stream and the Level-2 products are regenerated.The standard products generated for MODIS/Aqua are listed here: http://oceancolor.gsfc.nasa.gov/PRODUCTS/.

VI. Level-3

The Level-2 files are spatially averaged into Level-3 Daily binned files using the l2bin program developed by the OBPG (replaced spacebin), and the Daily files are further composited into Weekly, Monthly, Annual, Seasonal, and Climatological time periods using the l3bin code (replaced timebin). The algorithms used for spatial and temporal binning are described in Campbell,et al 1995. The binned products are then used to generate mapped products for chlorophyll, nLw(551), Angstrom(531), AOT(869), K(490), and SST(11) on 4 and 9-km equi-rectangular projection.These image products are in standard mapped image (SMI) format, and they are produced using the smigen program. Processing of the NRT stream ends with the Daily mapped products. The Weekly, Monthly, Annual, Seasonal, and Climatological binned and mapped products are only produced in the Refined processing stream.

Figure 2: Data distribution for MODIS/Aqua Ocean products produced by the OBPG.

VIII: Additional Details and Data Access

Standard operational products: http://oceancolor.gsfc.nasa.gov/PRODUCTS/

File format specifications: http://oceancolor.gsfc.nasa.gov/DOCS/ocformats.html

Data access:http://oceancolor.gsfc.nasa.gov/ftp.html

Latest reprocessing configuration: http://oceancolor.gsfc.nasa.gov/REPROCESSING/

Curator: OceanColor Webmaster

Authorized by: gene carl feldman