Software changes as of January 14, 2011
Based on the results of analyses and responses for the Evaluation data set, the latest version of the software and products that we have put into production incorporate a number of changes from the Evaluation version. These changes constitute the software corresponding to version 1.0 of the data products:
1. Addition of the version number to the product file names and the Processing Version attribute to the metadata. Based on requests from team members to maintain and distinguish multiple versions of the data products, we will implement the strategy described in Gene's message of January 6. This will consist of adding a version string (V1.0, etc.) to all product names and as a Processing Version attribute to the product metadata. Sub-versions (V1.1, etc.) will be used to delineate test runs between major versions. The version that we are putting into production will be designated V1.0. For consistency the Level-2 product format version has also been reset to 1.0.
2. Correction of the radiometer ice flag by North-South flipping of the NCEP ice ancillary data. ESR performed analysis that showed that the ice flag was flipped North-South, and Frank confirmed that he was flipping the NCEP ice data during preprocessing. Joel has implemented the flip in the ADPS software.
3. Performance improvements in the scatterometer software, in the land/ice percentage calculation. Alex provided updates to improve the efficiency of the land and ice percentage calculation, which was taking the bulk of the processing time for the scatterometer. This has reduced the scatterometer processing time by about a factor of 3.
4. Case changes in some scatterometer field names for consistency. Alex pointed out that the strings "ant" and "toa" used different cases in different field names. These have been made all lower case. This change has also been made in the Level-2 product format specification.
5. Changes to the naming convention for the SAC-D ephemeris products for compliance with the latest SAC-D GS to Aquarius GS ICD. The names had been based on an earlier version of the ICD. I can provide details if anyone is interested.
Software changes as of February 4, 2011
1. The process for generating the ancillary ice mask file at 1/2 degree resolution (from the 1/12 degree NCEP data) was revised to run once on the daily NCEP file as it is received. Previously this was performed as part of the Level-1A-to-2 processing for each orbit.
2. The scatterometer faraday correction routine was modified so that a processing failure would affect only the current data block and not terminate the processing run.
Software changes as of March 21, 2011
We have made a few enhancements and corrections for the radiometer data processing to the Aquarius Level-2 processing software. These changes have been put into the forward stream on March 21, 2011 with the data on March 9, 2011 being the first data reflecting these changes. Because these changes have no effect on the salinity or related fields, we will not immediately reprocess the entire simulated data set, but will choose a later time for reprocessing when more software changes are made.
The specific changes are as follows:
1. The spacecraft orbit longitude, latitude and altitude will be added to the Navigation fields, as a result of recent discussions of the Level-3 Science Algorithm Requirements.
2. A number of previous unimplemented radiometer quality flags have been added: Sun Glint, Galactic Background Contamination, Solar Flux Contamination (both direct and reflected) and Moon Contamination.
3. New radiometer fields have been added in conjunction with these flags: Lunar reflection (V, H, 3rd Stokes) and solar backscatter (V, H, 3rd Stokes).
4. In the process of implementing the flags, it was discovered that the Galactic and Solar correction fields for the radiometer data had not been converted from Stokes to H and V before being written to the data products; this conversion has been added to the software.
5. An error was corrected in the calculation of the radiometer Ta values from the HYCOM salinity. The field names were also changed at the recommendation of Thomas Meissner, e.g., from rad_sim_TaH to rad_exp_TaH, and similarly for TaV and Ta3.
The updated Level-2 product format specification has been posted on the Wiki:
http://oceancolor.gsfc.nasa.gov/AQUARIUS/DOCS/Aquarius_Level-2_Data_Products17March2011.pdf
In addition, two changes have been made to the Level-1A processing software:
1. The SAC-D HKT telemetry block data object is now being populated in the Level-1A products.
2. An error in the orbit ascending node longitude has been corrected.
Software changes as of March 30, 2011
The algorithm used to compute the radiometer fraction of land contamination (field name rad_land_frac) was changed to use the full-beam integrated value (gland in the software) instead of the 3 db beam value (fland). This has had the effect of causing the land moderate contamination flag to be set for all of the data samples, since the gland value is never 0.0. A revised threshold for the land flag is under discussion. Products with this change will have the software ID. 0.965
Software changes as of May 26, 2011
The final prelaunch algorithm changes in the Aquarius Level-2 processing software, ID 0.980 have been implemented. These were put into the forward stream for evaluation on May 20, 2011 and the following files have been reprocessed to incorporate the changes:
The list of reprocessed L1 files is,
Q2011129185300.L1A_SCI Q2011129203100.L1A_SCI Q2011129220900.L1A_SCI Q2011129234700.L1A_SCI Q2011130012400.L1A_SCI Q2011130030200.L1A_SCI Q2011130044000.L1A_SCI Q2011130061800.L1A_SCI Q2011130075600.L1A_SCI Q2011130093400.L1A_SCI Q2011130111200.L1A_SCI Q2011130125000.L1A_SCI Q2011130142700.L1A_SCI Q2011130160500.L1A_SCI Q2011130174300.L1A_SCI Q2011130192100.L1A_SCI Q2011130205900.L1A_SCI Q2011130223700.L1A_SCI Q2011131001500.L1A_SCI Q2011131015200.L1A_SCI Q2011131033000.L1A_SCI Q2011131050800.L1A_SCI
The L3 products for day 2011130 (May 10) are the first to use the new L2 inputs.
After the evaluation is complete during the week of May 23rd, the entire simulated data set will be reprocessed.
The changes are as follows:
- Updated version delivered by JPL of the scatterometer code, including the following changes:
- Added bounds checks to ice mask table and Xcal table to prevent seg-faults when processing off-nominal data (way off nominal, i.e. cold sky cal)
- Fixed bug in IONEX TEC interpolator that never manifested in operational simulation given the way the IONEX data we are using is posted. (but could have at some point)
- Added faster land mask
- Added model function interpolation down to 0 m/s; before it always returned 1 m/s sigma0 even when requesting a sigma0 for a lesser speed.
- Added a sanity check in spline interpolator for ephemeris (lack of which may have caused a bug when orbit started very near the start/end of a day)
- Fixed a bug where the sigma0 error was not computed quite right (it was off by a multiplicative constant order 1)
- Added range-weighting and range-gating to our scat land/ice fractions
- Improved handling of off-earth cases (i.e. cold sky cal data)
- Logic for leap years was wrong in scat wind field interpolator
- Fixed mistake in ordering of scat RFI flags (it was in wrong order in output file)
- Updates delivered by RSS to the radiometer code, to implement support for the Land corrections, including addition of new fields: rad_land_toa_V: TOA Tb V pol corrected for land contamination rad_land_toa_H: TOA Tb H pol corrected for land contamination SSS_land: Change in SSS due to land contamination
- Addition of a field SSS_error as a placeholder for the estimated error in salinity, needed to meet an L3 science requirement; currently set to -999.
- Modification of the land moderate contamination flag threshold to be non-zero.
- New metadata attributes containing the following, some of which are needed to meet science requirements:
- Input Ancillary data file names
- Radiometer calibration coefficient file names
- Radiometer table file names
- Scatterometer coefficent file names
- Setting the Cold Sky Calibration attribute using information from the Level-1A products.
- Changing the SSS_anc value over land from -999 to 35.
In addition, one change has been made to the Level-1A processing software, ID 0.63, to set the Cold Sky flag for each block in which the absolute value of the pitch angle exceeds 5 degrees.
