PACE Science Data Reprocessing Version 3
The initial public release of PACE science data products (Version 1) began on 11 April 2024, and provided the science and applications user community with access to the Level-1 data and a limited suite of derived products from the OCI, HARP2, and SPEXone instruments, with the caveat that the data were in a highly preliminary state and should be used with caution. Reprocessing Version 2 was the first full mission reprocessing, and primarily served to incorporate improved calibration knowledge from on-orbit measurements collected by the three PACE instruments. Reprocessing Version 3 includes a further refinement of the calibration for the three instruments, as well as various algorithm refinements, bug fixes, data format improvements, and expanded product suites.
Data Levels and Formats
As in previous versions, all data products are distributed in self-describing netCDF4 format and are available in the following data processing levels.
- Level-1A, raw instrument data and spacecraft telemetry, reformatted to netCDF4
- Level-1B, calibrated & geolocated instrument data
- Level-1C, calibrated, geolocated, and co-registered to a common grid
- Level-2, derived geophysical science data products
- Level-3, temporally and spatially composited (binned and mapped) global products
- Level-4, geophysical products derived from combined Level-3 inputs and/or models
Level-1C and Level-2 data files are divided into 5-minute granules that start at the southern terminator crossing of the ascending daylight orbit and end at the northern terminator crossing. For OCI, and HARP2 the Level-1A and Level-1B data are also in 5-minute granules, while SPEXone is organized into continuous daylight orbit periods (~50 minutes).
Level-3 products are generally global daily, 8-day, or monthly composites that are distributed in multiple spatial resolutions, including 4.6km, 0.1-deg, and 1.0-deg maps (Level-3m). A single L3m file may contain one 2D mapped product, or one 3D cube (e.g., where the 3rd dimension is the spectral dimension), or multiple 2D and/or 3D mapped products. The mapped products are derived by reprojection of binned data (Level-3b), where the typical bin distribution is a quasi-equal-area 4.6km or 9.2km integerized sinusoidal grid.
The products available at this release are categorized into maturity levels as:
- Provisional: results have been reviewed and are in family with heritage data products or other basis of expectation, but which have not yet been validated and may still contain significant errors.
- Test: results have not yet been reviewed by algorithm developers and or may be known to have substantial errors in implementation that are under investigation.
- Diagnostic: products that are produced to support analysis of algorithm behavior, but that are not intended for science.
The notes that follow provide an overview of the changes introduced between Version 2 and Version 3, for the Level-1 products of each instrument as well as the derived science products currently available. For a comprehensive list of the science data products planned for PACE and the current status of those products and algorithms see the PACE Data Products Table.
PACE Level-1C Grid
Changes from Version 2:
- For all instrument, the Level-1C granules at high latitudes, crossing the day/night terminator, showed significant geometric artifacts and frequent failures. The issue was traced to a software error, which has been fixed in V3. In addition to resolving the high-latitude granules, the correction results in a slight shift in the geographic location of the grid centers for all Level-1C granules by 10s of meters.
OCI Level-1A/B/C Data (Provisional)
Changes from Version 2:
- The OCI absolute calibration coefficients were revised by relying exclusively on the on-orbit solar diffuser calibration (previously: the calibration was prelaunch-based with adjustments derived from the solar diffuser measurements). The update for V3 significantly increases the observed radiances in the UV spectral range, with more modest changes over the full spectral range.
- The OCI temporal gain degradation now takes into account the degradation of the reflectance of the daily solar diffuser, thus improving OCI temporal stability.
- Occasional outliers are seen in OCI dark data, especially in the area of the South Atlantic Anomaly. These outliers can corrupt the background subtraction for a complete scan line, leading to striping in the L1B data. The algorithm to calculate the background subtraction in the L1B code has been improved to remove these outliers end thus remove he striping artifacts. Single pixel outliers in the earth view data may still be present.
- Geolocation accuracy was further improved through use of definitive ephemerides, updated sensor alignments, and improved sensor scan geometry.
- The integrated solar irradiances for each OCI band, as provided in the Level-1B and used to normalize the observed radiance to a reflectance, were updated to use a more appropriate integration approach. This change reduces non-physical spectral variability in the derived reflectance, especially in the UV-blue spectral region.
Known Issues and Data Characteristics:
- Horizontal striping can occur for bands between 650nm and 900nm for scan angles from +3.9deg to +14.6deg (scan pixels 680 to 800). For open ocean scenes, the TOA signal usually varies less than 1%, but close to bright sources (such as a coastline) the striping over ocean can be several percent. The striping is usually too small to detect over land or clouds. The OCI calibration team has developed a correction approach, targeting an implementation for the V4 reprocessing. The most likely reason for the striping is spatial and spectral crosstalk impacting half-angle mirror side 2.
- Data in the transition region between the red and blue focal planes, between 590 and 610nm, often shows significant discontinuity. Measurements in this range have much higher uncertainty and should not be used for science algorithms.
- Some SWIR bands show significant signs of apparent band-to-band registration issues. Based on prelaunch measurements, bands 1038nm and 1250nm (Standard Gain), and 1615nm (Standard Gain) with large radiance gradients to neighboring pixels in scan direction are expected to be impacted the most.
- The optical design of the OCI SWIR detector assembly (SDA) causes the bands to view different locations along-scan at a given time, and the data are packetized by time. The bands are pixel-shifted into alignment with the hyperspectral bands. This results in fill pixels at the start or end of the scan, depending on the required pixel shift. The worst case is the 2260 nm band that has 13 fill pixels at the start of the scan. This is a feature of the instrument design.
- Data below 340nm has not been characterized prelaunch as completely and accurately as the data above 340nm. The data below 340nm is only released to facilitate assessment and potential refinement of radiometric accuracy and should not be considered as science quality.
HARP2 Level-1A/B/C Data (Provisional)
On-orbit calibration data has been collected during solar and lunar calibration exercises, and a refined calibration is included in the version 3 release.
Changes from Version 2:
SPEXone Level-1A/B/C Data (Provisional)
Changes from Version 2:
- Fixed a geolocation bug which assigned wrong latitudes and longitudes to water bodies in high elevation regions.
- Updated the algorithm for computing the Stokes Q and U and the rotation angle from the local meridian plane to the scattering plane. Whereas the degree of linear polarization was unaffected, there was ambiguity in the sign of Q and U and polarization reference planes used. In Version 3 Level-1C products, SPEXone and HARP2 use a common convention.
- Level 1C products now contain more radiance and polarization data near the poles.
- Updated bad pixel mask of the detector.
Known Issues:
- Improvement of radiometric and polarimetric performance at the edges of the wavelength grid under investigation.
OCI Level-2 Products
A further expanded set of OCI Level-2 science data products is released at this time. As in previous versions, they are organized into product suites, with each suite in one file per granule. In some cases, the Level-2 products suites contain a mix of Provisional, Test, and Diagnostic products.
OC_AOP, Ocean Color Apparent Optical Properties
Provisional Products
