Not logged inOcean Color Forum

The forum is locked.

The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.

Up Topic SeaDAS / SeaDAS - General Questions / Climate Standard
- By bbroman93 Date 2019-02-28 05:30
Hello,

I am trying to generate my own climatology file for the years 2002-2010.
I downloaded the summer months for the years in question. I organized the data as follows:
2002 - summer months data
2003 - summer months data
2004 - summer months data
etc etc
I have already processed the data through l3bin and l3mapgen and have collocated my files into one big file.
My plan was to use the math expression function to add 2002 + 2003 ..... + 2010 and then divide by 9. To get the mean. Right?

I tried it, and it appears to work...but I'm not sure sure about the climatology that is generated from that. I have my doubts about its accuracy. Could I be doing something wrong?

When I use my generated climatology to calculate anomaly, I do the following:

observed - climatology = anomaly for that time frame

And the resulting image simply cannot be right.
Any help is much appreciated. I'm loving the program so far, have generated some really cool images for 2011, 2015, and 2018 but that was using the climatology on the ocean color website. I am trying to show deviation from before 2011.

Thank you!
B
- By knowles Date 2019-02-28 13:02
Your use of the math expression is incorrect.  You cannot just sum the values for a group of bands for a given pixel and then divide by the number of bands because some bands may likely have NaN values for that pixel, hence the number you divide by would be pixel dependent.  The correct math expression would be huge and ugly.  The Math expression tool has it's place and use but it is not a good tool to use for aggregating files.  The primary tools for aggregating a lot of files are l3bin and the Mosaic Tool.

Since you are starting with level3 bin files the best technique to achieve what you are trying to accomplish is:
1. Create a level3 bin file for each summer by running l3bin on the set of bin files for each year's summer months
2. Create your "climatology" bin file by then running l3bin on the complete set of your summer bin files (the output files of step 1)
3. Run l3mapgen 

Danny
- By bbroman93 Date 2019-03-03 23:00
Alright, I have created the level3 bin file for each summer by running l3bin on the set of files for each year's summer months. Now, I'm trying to complete step 2, but I'm a little confused. Do I use another text file to unite the years I'm interested in using for my climatology? How do I run l3bin on a complete set of output files?
Also, I just wanted to say, this software is amazing. I'm still figuring it all out but it runs extremely smoothly on my Mac.

Bianca
- By gnwiii Date 2019-03-04 13:31
Many people find these workflows confusing so a bit of explanation may help you and hopefully others who encounter this thread.     Computers are very fussy about little details.   A general overview of the processing workflow may help distinguish between fussy details (formatting input text files on MacOS) as opposed to those caused by trying to do something the software wasn't designed to handle (e.g., hoping to view binned files in the SeaDAS GUI or using band maths to compute climatologies).

Level-2 files are difficult to use.  Although the data are stored in an array, the locations of the individual pixels are obtained from relatively complicated calculations and vary with each level-2 file.   Binning combines data that originated from multiple level-2 files into a common spatial structure (but not an array).  Each "bin" represents a region with that contains statistics for data in a certain range of longitudes and latitudes.   The bins are defined by dividing the earth's sea-level surface into a fixed number of bands, and then dividing up each latitude band into bins.  The "height" (latitude difference) of each bin is the same, while the width varies slightly in order to have an integer number of bins within each latitude band.  L2bin reads level-2 data from multiple files and saves  statistics (counts, sum, and sum-of-squares) for all the level-2 pixels that fall in each bin.   L3bin combines binned files by combining the statistics from all the input files for each bin.  It is simpler and faster than l2bin since the input files all have a common structure.    L3mapgen uses the statistics stored in a binned file to compute means/averages, variances, etc. as well as mapping the binned data into a 2-D array suitable for viewing.

Hopefully this is enough to allow you to answer your own question.
- By knowles Date 2019-03-04 14:35
Here's a specific example which creates an aggregated "climatology" file for 2003-2005 using the June, July and August level3 monthly bin files as input.

Step 1a:
l3bin ifile=2003_monthlies.txt ofile=A20031522003243.L3b_SNSU_CHL.nc
where 2003_monthlies.txt:
A20031522003181.L3b_MO_CHL.nc
A20031822003212.L3b_MO_CHL.nc
A20032132003243.L3b_MO_CHL.n
c

Step 1b:
l3bin ifile=2004_monthlies.txt ofile=A20041532004244.L3b_SNSU_CHL.nc
where 2004_monthlies.txt:
A20041532004182.L3b_MO_CHL.nc
A20041832004213.L3b_MO_CHL.nc
A20042142004244.L3b_MO_CHL.nc


Step 1c:
l3bin ifile=2005_monthlies.txt ofile=A20051522005243.L3b_SNSU_CHL.nc
where 2005_monthlies.txt:
A20051522005181.L3b_MO_CHL.nc
A20051822005212.L3b_MO_CHL.nc
A20052132005243.L3b_MO_CHL.nc


Step 2:
l3bin ifile=2003_2005_summers.txt ofile=A20031522005243.L3b_SCSU_CHL.nc
where 2003_2005_summers.txt:
A20031522003243.L3b_SNSU_CHL.nc
A20041532004244.L3b_SNSU_CHL.nc
A20051522005243.L3b_SNSU_CHL.nc


Note that you cannot actually make a "true" summer aggregated level3 file if your input is limited to monthlies.  For instance, the span of days when using June, July and August Monthly files for 2003 is from day 152 (June 1) to day 243 (August 31).  If you were to download the OBPG summer seasonal level3 file (A20031722003263.L3b_SNSU_CHL.nc) this actually spans from day 172 to day 263 (which is from June 21 summer solstice to September 20 autumn equinox).  In order to make a "true" summer file you could use the daily level3 files as your input to step 1.

Danny
- By bbroman93 Date 2019-03-04 17:27
Okay that's sort of what I was envisioning. I am trying to make a "climatology" for 2002-2010 so that I can then look for anomalies from 2011-2018. So I l3binned the summer months as you show, and that worked. So I tried to make a text document that looks as follows:

A20021852002244.L3b_CU_OC
A20031522003244.L3b_CU_OC
A20041532004245.L3b_CU_OC
A20051522005244.L3b_CU_OC
A20061522006244.L3b_CU_OC
A20071522007244.L3b_CU_OC
A20081532008245.L3b_CU_OC
A20091522009244.L3b_CU_OC
A20101522010244.L3b_CU_OC

(because those are the output results of the l3bin from step 1).

You're suggesting that I run that in l3bin, then l3mapgen, and then THAT output will represent a rough climatology for that time frame?
- By bbroman93 Date 2019-03-04 22:12
I am trying to run the files in l3bin, and I keep getting an error. I have attached an image of my files and the respective text file I am trying to use. Thank you!
B
- By bbroman93 Date 2019-03-04 22:18
Are you able to see my screenshots? I cannot seem to see them but I have tried to post them.
Bianca

/Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.29 PM.png
/Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.33 PM.png
- By gnwiii Date 2019-03-05 00:10
The images are not visible.   If you run the command in a terminal then you can paste the output as text directly into a post.   It may be useful to add verbose=on to the l3bin command line.
- By knowles Date 2019-03-05 12:50
What's the error?   To rule out a problem with your text file (which is a list of files) being incorrectly formatted you can instead try a single file to see if it works. 

Also the file name /Users/biancabroman/Desktop/Screen Shot 2019-03-04 at 4.11.29 PM.png contains spaces, perhaps this is causing your file upload problem.

Danny
- By bbroman93 Date 2019-03-06 17:13
Whenever I select the text file with the list, it will not even let me run the l3bin command.
- By bbroman93 Date 2019-03-06 17:14
/Users/biancabroman/Desktop/error.png
/Users/biancabroman/Desktop/text.png
Attachment: 2002_2010Climatology.txt (234B)
- By bbroman93 Date 2019-03-06 17:21
Okay, for whatever reason, I really cannot upload the images, but it looks like I succeeded at posting the txt file. Whenever I try to l3bin that file, the Run button doesn't even hyperlink. I can't run the command. The following are the contents of the folder which contains the txt file that I am trying to run. The txt file is just a list of these files. Am I missing something in that folder, perhaps?

2002_2010Climatology.txt
A20021852002243.L3b_CU_OC
A20031512003243.L3b_CU_OC
A20041522004244.L3b_CU_OC
A20051512005243.L3b_CU_OC
A20061512006243.L3b_CU_OC
A20071512007243.L3b_CU_OC
A20081522008244.L3b_CU_OC
A20091512009243.L3b_CU_OC
A20101512010243.L3b_CU_OC

Thank you very much.
B
- By knowles Date 2019-03-06 17:28
I cannot see your images.  So does this mean you could not do step1 which was also a text file run through l3bin.
- By bbroman93 Date 2019-03-06 17:37
Yes no matter what I try they do not seem to be uploading...

I ran all of my summer months for the years through the l3bin function, successfully. Then, I compiled the outputs of that into a folder, and created a txt document which reflected those output files. Then I tried to l3bin that txt file, and it will not let me. The folder looks like this:
(and the text file reflects those output files)

2002_2010Climatology.txt
A20021852002243.L3b_CU_OC
A20031512003243.L3b_CU_OC
A20041522004244.L3b_CU_OC
A20051512005243.L3b_CU_OC
A20061512006243.L3b_CU_OC
A20071512007243.L3b_CU_OC
A20081522008244.L3b_CU_OC
A20091512009243.L3b_CU_OC
A20101512010243.L3b_CU_OC
- By knowles Date 2019-03-06 17:41
The run button fails to be enabled whenever there is an unrecognizable file in the list or some extra hidden character is in the list.  Within the l3bin GUI, try selecting each of the files one at a time and see if the run button enables for each file.  Note that a blank line in the file will cause a problem as well.

Danny
- By bbroman93 Date 2019-03-06 17:48
The l3bin GUI has been working for me until now. Something must be wrong with how I'm formatting it. Otherwise does it look like my methods are more or less correct?
B
- By bbroman93 Date 2019-03-06 17:53
Alright I think I fixed it. I deleted the txt file and recreated it, and for whatever reason, now it appears to be working! I ran l3mapgen as well, and it worked also. So you're telling me that this output file is my climatology?
Bianca
Up Topic SeaDAS / SeaDAS - General Questions / Climate Standard

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill