Not logged inOcean Color Forum

The forum is locked.

The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.

Up Topic Products and Algorithms / Field Data - SeaBASS / Satellite validation matchup tool val_extract issues
- By cmitchell Date 2017-07-07 14:47
Hi,
I have been trying out the new matchup scripts provided in the ocssw source code: fd_matchup.py and mk_matchup.py. The fd_matchup.py script is running well, but I am having problems with mk_matchup.py script: it fails when trying to run val_extract.  Below is a stripped back example showing what I get when I run the command (note in my seabass file I have only one station, which does match in space and time with the provided VIIRS file).

cmitchell@ubuntu:/mnt/hgfs/SHARE/VIIRS_matchups_2017$ mk_matchup.py --seabass_file=VIIRSMatchupStations_test2.sb --sat_file='/mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc'
sat_file = /mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc

Calculating satellite match-up for L2 file:
Unknown option: global_att

usage: mk_matchup.py [-h] --sat_file SAT_FILE --seabass_file SEABASS_FILE
                     [--out_file OUT_FILE] [--clobber [CLOBBER]] [--slat SLAT]
                     [--elat ELAT] [--slon SLON] [--elon ELON]
                     [--box_size BOX_SIZE]
                     [--min_valid_sat_pix MIN_VALID_SAT_PIX]
                     [--max_time_diff MAX_TIME_DIFF]
                     [--max_coeff_variation MAX_COEFF_VARIATION]
mk_matchup.py: error: val_extract failed -- ensure that the val_extract binary is compiled and on your PATH and that /mnt/hgfs/SHARE/VIIRS_matchups_2017/sat_files/V2013233164200.L2_SNPP_OC.nc exists


Any help would be much appreciated. Thanks.
- By jpsco Date 2017-07-07 15:31
Hi,

Thank you for checking out the new stand-alone match-up tools, and I apologize that you are getting an error using mk_matchup.py.

From the mk_matchup.py output you are providing, it sounds like your copy of SeaDAS and its OCSSW components may be an outdated version of the val_extract binary that is used to extract the satellite data from your VIIRS file. To confirm if that is the case, would you please run on your command-line:

> val_extract version=1


and let us know what the output is. It should look something like this:

val_extract=2.6.2
val_extract-main=1.3.1

Cheers,
Joel
- By cmitchell Date 2017-07-07 15:43
Thanks for the quick response.

Looks like that may be it. I'm getting

val_extract=2.5.9
val_extract-main=1.2.2

Cheers,
Catherine
- By jpsco Date 2017-07-07 15:51
Hi Catherine,

I believe that version difference is the source of the problem.

However, I double-checked and val_extract version 2.5.9 is, in fact, the latest publicly available version of val_extract, which is due to a bug on our end.
We will push an updated version of the val_extract binaries to the SeaDAS / OCSSW code repository shortly (probably early next week sometime).

Once the version is updated on our end, you will then be able to update your OCSSW source code installation from the OCSSW menu in SeaDAS, and that should resolve the problem.

Thank you for your patience while we resolve this!

Cheers,
Joel
- By cmitchell Date 2017-07-07 15:55
OK great, thanks a lot Joel.

Please let me know when the updated version is available.
- By cmitchell Date 2017-07-13 10:52
Hi Joel,

Is there any progress on this?

Many thanks,
Catherine
- By seanbailey Date 2017-07-13 11:52
Catherine,
The fix is not as easy as Joel believed it would be. ...well, it's not hard, but it is more involved than a simple script update and requires attention that we don't have the time to give just yet.  Hopefully, things will settle down soon, but for now there are too many hamster wheels, not enough hamsters...

Sean
- By jpsco Date 2017-11-15 14:02
Hi,
The val_extract binary has been updated in the latest version of SeaDAS via its copy of OCSSW. To get the mk_matchup.py script to work, you will need to update your local val_extract binary to the latest public version, which should be: val_extract=2.6.2 and val_extract-main=1.3.1.
To do so, please open SeaDAS and from the "OCSSW" drop-down menu, select "Update OC Processors" be sure that at least one box is checked (e.g. - "Source Code, " etc) and click "Run." Upon successful execution of the update, your val_extract version should be compatible with the mk_matchup.py script, enabling the script's full functionality. Please post again on the forum should you have any additional trouble with the stand-alone match-up tools.

Cheers,
Joel
- By cmitchell Date 2017-11-17 10:11
Hi,

Thanks for letting me know about the update. I updated through the GUI as you described, but am still getting the same error message as I originally did when I then try to run the mk_matchup.py script. I ran

val_extract version=1

and am still getting val_extract=2.5.9 val_extract-main=1.2.2. I tried updating at the command line (running the install_ocssw.py script with the --src option) but that doesn't work either. I should maybe clarify: everything appears to update OK, but when checking the val_extract binary, it still seems to be the older version.

Thanks
Catherine
- By jpsco Date 2017-11-17 10:39
Hi Catherine,
I'm sorry that you are still experiencing difficulties with val_extract. Both methods that you described to update OCSSW in SeaDAS (GUI and command-line) are valid. Are you by any chance running on a Mac operating system instead of Linux?

It appears that the Linux binaries have been updated:~/seadas-7.4/ocssw/bin$ ./val_extract version=1
val_extract=2.6.2
val_extract-main=1.3.1

Yet, I believe the updated Mac build release is lagging a bit behind the Linux binary version release. My apologies for the confusion.

Cheers,
Joel
- By cmitchell Date 2017-11-17 10:48
No I'm not on a Mac. I'm running ubuntu 16.04 on a virtual machine on a Windows PC.
- By jpsco Date 2017-11-17 11:16
Hmm, I have a couple ideas that might resolve this issue.

The newest build of OCSSW reorganized things a bit. Therefore, try running: which val_extract
It should point to: ~/seadas-7.4/ocssw/bin/val_extract instead of the old version: ~/seadas-7.4/ocssw/run/bin/linux_64/val_extract

When you ran ~/seadas-7.4/ocssw/run/scripts/install_ocssw.py, the old binaries in the old location will still exist. This is not necessarily a problem, but you will need to update your ENV $PATH variable with the locations of the new binaries: ~/seadas-7.4/ocssw/bin. You likely have an ENV variable called $OCSSWROOT that points to ~/seadas-7.4/ocssw; if so you will just need to run: export PATH=$OCSSWROOT/bin:$PATH

Then try running which val_extract again and it should find ~/seadas-7.4/ocssw/bin/val_extract. If it does, then val_extract version=1 should return:
val_extract=2.6.5
val_extract-main=1.3.1
- By cmitchell Date 2017-11-17 11:17
I partly figured it out. When I ran the install_ocssw.py script at the command-line I'd included --git-branch=v7.4. I ran it again without that option and that seemed to update the val_extract binaries. What I don't understand is why it didn't work when I tried to update through the GUI.

I've still to test out the mk_matchup.py script.
Up Topic Products and Algorithms / Field Data - SeaBASS / Satellite validation matchup tool val_extract issues

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill