Stray Light Masking Change

Summary

The first round of work has been completed on relaxing the amount of flagging that is done by the stray light flag.  Indications are that this can be done and the amount of stray light flagged pixels can be reduced by 1/4 with little effect on the water-leaving radiances or chlorophyll.  A test case in the South Pacific was used for this test

A coastal region was studied in the same way the new stray light corrected pixels were corrected relatively well.  The results of that study appear further below.

The stray light masking change results in an increase in the samples and coverage in the level-3 time binned data.
Samples increased an average of 17%, versus the old stray light masking while the coverage increased by 19% for daily files, 10% for 8-day files and 4% for monthlies.  For an 8-day period, the global mean in nLw increased by 2 to 4% while the chlorophyll increased by about 2%.  More information appears below.

Introduction

The current stray light algorithm was developed using laboratory measurements of a bright target.  Pre-launch TM 31 discusses the derivation of the corrections and masking and TM 41 describes the application of the algorithm.  TM 31 states that it may be possible to relax the GAC masking so that two more samples along a scan with a bright target (the samples that are 2 pixels down- and up-scan of the bright target) may be corrected instead of masked.  Stray light is a major cause of masked pixels in a time-binned file, causing a 32% loss in samples, so reducing the stray light masking by 1/3 could result in 10% more samples recovered.

The changes in the code were fairly simple and TM 31 already had correction factors for these GAC samples.  The new correction was implemented and tested on a portion of a GAC file that passed entirely over the ocean in a region with some of the lowest chlorophyll - the South Pacific gyre.
 

Test in South Pacific gyre

In the test case, a portion of a GAC of 9 June, 2001 at 2006 GMT, the # of retrievals, using the standard L2 masks, was 46751 samples.  When the current operational stray light is turned on, 37890 retrievals are unmasked, a 19% loss in samples.  When the new stray light is used, 41010 samples are retrieved, a 14% loss in samples and a 5% increase in the number of unmasked samples.

An image of a portion of the region is shown below for the total radiance at 412 nm (Lt_412).  Regions that are masked for the basic level-2 masks are in black and the stray light mask is in red.  Note that the new pixels that are gained with the stray light change are the pixels immediately to the left and right of the cross-shaped stray light masked pixels.  These pixels visually show little affect of the adjacent bright target.  More striking is the effect of thin or sub-pixel clouds and their shadows.  In the normalized water-leaving radiances in the next image, the atmospheric correction has attributed the cloud signal to an aerosol effect and has flagged most of the cloud shadow areas with the low water-leaving radiance flag, as it should (flag not shown).  The chlorophyll-a (chlor_a) data does not show any effects at the stray light corrected pixels either.  The imagery confirms that the new stray light corrected pixels can be used.

Lt_412 image

nLw_412 image

chlor_a image

 

Distributions of data with stray light correction

The histogram below is a comparison of the retrieved Lt_412 pixels using the operational stray light (dotted line) and using the new stray light (solid line).

The addition of the new retrievals does not modify the distribution of Lt_412.  A comparison of the new retrievals gained using the new stray light algorithm (dotted line) and of pixels in the neighboring regions (in the solid line) is shown below.

The distributions match very closely, showing that the corrected pixels have no bias relative to the neighboring pixels.

The next 2 plots are for the 555 nm band:
Lt_555, operational retrievals, new stray light retrievals

Lt_555, new stray light pixels and neighboring pixels

In the above plots, some shifting of the mean exists, which could be of some concern to the quality of the ocean products.  In the plots of nLw below, most of this shift is lost.  The next 2 images are the same as above but for nLw_412 and nLw_555:
nLw_412, operational retrievals, new stray light retrievals

nLw_412, new stray light pixels and neighboring pixels

nLw_555, operational retrievals, new stray light retrievals

nLw_555, new stray light pixels and neighboring pixels

Also, the chlorophyll distribution is compared:

nLw_555, operational retrievals, new stray light retrievals

nLw_555, new stray light pixels and neighboring pixels

In addition to the above comparisons, the mean and standard deviation was computed for a number of different groups of samples in the scene.  The following groups were analyzed:

1. CONSERVATIVE - this is the set of retrievals that result if all of the stray light affected pixels (3 pixels to either side of the bright target) are masked out.
2. OPERATIONAL - the set of pixels if the operational stray light masks (which mask 2 pixels to either side of the bright target) are used.
3. NEW ST. LT. - new stray light - the set of pixels retrieved with the new stray light mask (1 pixel masked on either side of the bright target).
4. NO ST. LT. - no stray light - the set of pixels retrieved with the stray light mask turned off
--- the following sets are seperate groups of samples in the dataset
5. AWAY FROM MASK - the retrieved pixels that are at least 3 pixels away from the stray light corrected pixels.  This is a very pristine set of points that avoids being close to any areas with trouble.  However, the use of these points alone would severly reduce retrievals obtained.
6. FAR PROXIMATE - the retrieved pixels that are 2 pixels away from stray light corrected pixels with along-scan pixels excluded.  This set of points represents the neighborhood of the stray light pixels.
7. NEAR PROXIMATE - the retrieved pixels that are 1 pixel away from the stray light corrected pixels with along-scan pixels excluded.  This set of points also represents the neighborhood of the stray light pixels.
8. 3RD CORR STRAY - the set of stray light corrected pixels that are 3 pixels away from the bright target on the same line.  These pixels are the stray light pixels that are included currently in the operational processed data.
9. 2ND CORR STRAY - the set of stray light corrected pixels that are 2 pixels away from the bright target.  These are the pixels that are gained using the new stray light algorithm.
10. NOT CORR STRAY - the set of pixels immediately adjacent to the bright target (both along-scan and along-track).  These are the pixels considered uncorrectable with the current stray light algorithm.

Note that the plots above used the OPERATIONAL and NEW ST. LT. groups  and the NEAR PROXIMATE and 2ND CORR STRAY.

The plots below show the mean and standard deviation for each of the groups mentioned above for representative products (other products behaved in the same way):

Lt_555                                                  nLw_555

Chlor_a                                                La_555

The distributions of Lt show an increase in the mean as pixel groups closer to the stray light target are considered. This could show the effects of stray light but stray light is not the only effect, because the far proximate and near proximate groups, which should be unaffected by stray light, also show a similar increase in the mean.  More likely, the increase is due to thin or sub-pixel clouds that were not resolved.  The real question is whether the drift in the mean is seen in the water-leaving radiances.  In the Lw_555 plot above, most of the trend is removed.  The atmospheric correction attributes the extra radiance to the aerosols, as shown in the plot for the aerosol radiance at 555 nm (La_555).  The chlorophyll means are also little impacted by moving closer to the bright target.

In conclusion, the relaxing of the masking conditions in the stray light algorithm have little harmful effect on the water-leaving radiances or chlorophyll but allow more retrievals to be kept.
 

Test of performance in a coastal region

The last test workd with a fairly uniform ocean situation.  Another test of the stray light was run near the coast, in regions with changing chlorophyll and more continental aerosol properties.  A GAC file that passed off the East Coast of the Unites States was chosen.  The file was at 1626 GMT on 20 June, 2001.  The additional stray light corrected pixels showed no visual impact on the nLw or chlorophyll fields.  The distribution of the newly gained pixels (dotted line) compares well to the distribution of nearby pixels that are unaffected by stray light:

The means of the Lt and nLw appear much the same as in the Pacific gyre case except for a greater amount of variability in the data values that should be expected in this region.  The plot of the means and standard deviations of the chlor_a are shown here:

The chlor_a mean plot shows a greater increase in the mean chlorophyll compared to neighboring points than the gyre case showed.  Also, the standard deviation increased significantly.  This increase is probably due to  greater variability of chlorophyll in the coastal areas.  The accompanying chlorophyll distributions are beow

 

Effect on timebin files

The coverage increase attributed to the change in the stray light masking was evaluated by comparing the operational stray light masking to the new stray light masking and making no other algorithm changes.  The month of July, 1998 was processed both ways and daily, 8-day and monthly time bin files were compared.  For all the binning periods, the switch to the new stray light algorithm resulted in an average gain of 17% more samples than the current algorithm.  This number is lower than the initial mask analysis value of 32% because several other masks come into play in the final binned product.  The extra coverage of the earth, expressed in the number of bins was a function of the binning time period and decreased as longer binning times were used.  For the daily files, the coverage increased by 19% on average, for the 8-day files, the coverage increased by 10%, and for the monthly, the coverage increased by 4%.

The coverage increase was spread out fairly uniformly, as is seen in this image of extra coverage for a 8-day file (day 185-191 of 1998):

The following plots compare the 8-day timebin done for 4 - 11 July, 1998 using the new stray light algorithm versus the current algorithm.  The spread in the scatter plots is due mainly to the inclusion of new samples into bins.  There is a general tendancy for the distributions of nLw and chlorophyll to raise with the inclusion of the new stray light corrected points.  As said earlier, this could be due to the increased chlorophyll in coastal areas that are now included.  Means of the nLw increased by 2 - 4% while the chlorophyll mean increased by about 2%.

Lastly, the 2 images below show the chlorophyll for a portion of a daily timebin file (for 1998, day 197) and focuses on the region of central Europe.  The image made using the operational stray light masking is on the left and the new masking is on the right.  The gained bins compare well to the exisitng bins in the image.  The increase in coverage can be seen throughout the image and there are many places in the image that have coverage closer to the coast as a result of the new stray light algorithm.