I have 2 scripts to update the anc data that look like this (one for 7.2 and one for 7.3):

> cat /optics1/virtual_ant/bin/cron/ocssw_7.3-CentOS6.csh

#!/bin/tcsh
#$1 should be satellite
source /optics1/software/seadas/seadas-7.3/ocssw/OCSSW.env
set command = "/optics1/software/python27/bin/python $OCSSWROOT/run/scripts/update_luts.py $1 --verbose"
$command

The only diff is the OCSSW env path. Both used to work fine.

> diff /optics1/virtual_ant/bin/cron/ocssw_7.3-CentOS6.csh /optics1/virtual_ant/bin/cron/ocssw_7.2-CentOS6.csh

3c3
< source /optics1/software/seadas/seadas-7.3/ocssw/OCSSW.env
---

> source /optics1/software/seadas/seadas-7.2/ocssw/OCSSW.env

I updated the OCSSW on both seadas 7.2 and 7.3 (maybe no http to https changes were done to the 7.2 tree?) and the 7.3 script fails. However it only fails when running /optics1/software/python27/bin/python

When I  run it by hand on a command line it works fine with Python 2.6.6?

seahorse 109 :/optics1/home1/oo_processing> echo $OCSSWROOT
/optics1/software/seadas/seadas-7.3/ocssw
seahorse 110 :/optics1/home1/oo_processing> $OCSSWROOT/run/scripts/update_luts.py aqua --verbose
[ MODIS ]
+ leapsec.dat
+ utcpole.dat
[ Sensor: AQUA ]
[ Done ]

seahorse 111 :/optics1/home1/oo_processing> python
Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>>

Please advise, 2 things. Was seadas 7.2 updated http to https?
Why does the older version of python now work?

But not this newer version that used to work?
seahorse 112 :/optics1/home1/oo_processing> /optics1/software/python27/bin/python
Python 2.7.2 (default, Nov  4 2011, 09:59:12)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

Cheers,
Brock

I had to adjust the python version on macOS as the vendor's version gave:

$ update_luts.py -v aqua
[ MODIS ]
Error! could not establish a network connection. Check your network connection.
If you do not find a problem, please try again later.


There can be problems with the library used by Python for secure connections and also certificates.  Make sure that your curl and wget tools work (examples were posted on the forum) .  It would be helpful to to have python script that checks out your network bits.

No, the v7.2 scripts were not updated, but feel free to put the v7.3 versions in their place.

The scripts rely on the urllib module, which in turn relies on the socket module and that must be built with SSL support for the scripts to work.
It could be that the "newer" version on your system lacks SSL support in the socket module, while the older version has it.

You can test this by launching a python shell and entering the following:


import socket
socket.ssl


If you have SSL support, the function address will be printed, e.g.:
<function ssl at 0x7f71fd70b320>

Sean

Sean,
It appears that both have it:
seahorse 250 :/optics1/virtual_ant/bin/seadas_l2_viirs_h5> python
Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import socket
>>> socket.ssl

<function ssl at 0x2b523fbcd488>

>>>

seahorse 251 :/optics1/virtual_ant/bin/seadas_l2_viirs_h5> /optics1/software/python27/bin/python
Python 2.7.2 (default, Nov  4 2011, 09:59:12)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-51)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import socket
>>> socket.ssl

<function ssl at 0x2b091f2b79b0>

Interesting that the newer version of python is on an older version of the OS.
Since it's not in the default location, I'll assume it was an alternate install.
You might try getting a "newer" version of 2.7 for the older OS.  

However, even that might not do it, as the version of the ssl library
the code links to  may be too old.

you can check the ssl version, e.g:
import ssl
ssl.OPENSSL_VERSION

I know this one works:
'OpenSSL 1.0.2g  1 Mar 2016'
But can't tell you a minimum version that would work.
I can tell you that your machine's openssl needs to support at least one of the following ciphers:
ECDHE-ECDSA-CHACHA20-POLY1305
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-ECDSA-AES256-SHA384
ECDHE-ECDSA-AES128-SHA256

You can check that your system provides one of these with the following:
openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"

Sean

The CentOS6 system is fully updated and patched. I went though my old notes, and see that at one point I needed python 2.7 to make things work for SeaDAS 6.3. However, It seems that the latest version of python for CentOS 6 (old but updated) works fine, so I am thinking I can just not use the 2.7 in the 'non-standard' location. Your code snippet returns:

ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256

All my scripts are using https now, and I have not had any other issues. ( YET )

Brock

Great!  You should be ready for the switch tomorrow

Sean

The standard Python provided in Linux Mint 17.3 Rosa (should be the same as Ubuntu 14.04) works for me:

$ /usr/bin/python -c 'import ssl ; print ssl.OPENSSL_VERSION'
OpenSSL 1.0.1f 6 Jan 2014

$ /usr/bin/openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256

MacOS El Capitan Python uses an older OpenSSL that does not work:
$ /usr/bin/python
Python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.



>>> import ssl
>>> ssl.OPENSSL_VERSION

'OpenSSL 0.9.8zh 14 Jan 2016'

$ /usr/bin/openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"
[no output]
This version does not work.  There are several easy ways to get a more up-to-date python with ssl support, including MacPorts and Anaconda Python.

The current macports' version does work:

$ /opt/local/bin/python2.7 -c 'import ssl ; print ssl.OPENSSL_VERSION'
OpenSSL 1.0.2j  26 Sep 2016

$ /opt/local/bin/openssl ciphers -tls -v 'HIGH:!ADH:!MD5:@STRENGTH' | egrep "ECDHE-ECDSA-CHACHA20-POLY1305|ECDHE-ECDSA-AES256-GCM-SHA384|ECDHE-ECDSA-AES128-GCM-SHA256|ECDHE-ECDSA-AES256-SHA384|ECDHE-ECDSA-AES128-SHA256"
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256  Note that macports also has certificates in curl-ca-bundle.

With macOS macports, git use the same openssl library.
On Ubuntu 14.04, however, git uses an older version of gnutls that does not support the above ciphers:

$ gnutls-cli -v
gnutls-cli (GnuTLS) 2.12.23
Packaged by Debian (2.12.23-12ubuntu2.5)

$ git fetch
fatal: unable to access 'https://oceandata.sci.gsfc.nasa.gov/ocssw/scripts.git/': gnutls_handshake() failed: Handshake failed

None of the above cyphers are listed (for gnutls on Ubuntu 14.04):

$ gnutls-cli -l | grep TLS1.2
TLS_ANON_DH_AES_128_CBC_SHA256                      0x00, 0x6c  TLS1.2
TLS_ANON_DH_AES_256_CBC_SHA256                      0x00, 0x6d  TLS1.2
TLS_DHE_DSS_AES_128_CBC_SHA256                      0x00, 0x40  TLS1.2
TLS_DHE_DSS_AES_256_CBC_SHA256                      0x00, 0x6a  TLS1.2
TLS_DHE_RSA_AES_128_CBC_SHA256                      0x00, 0x67  TLS1.2
TLS_DHE_RSA_AES_256_CBC_SHA256                      0x00, 0x6b  TLS1.2
TLS_RSA_NULL_SHA256                                 0x00, 0x3b  TLS1.2
TLS_RSA_AES_128_CBC_SHA256                          0x00, 0x3c  TLS1.2
TLS_RSA_AES_256_CBC_SHA256                          0x00, 0x3d  TLS1.2
Protocols: VERS-SSL3.0, VERS-TLS1.0, VERS-TLS1.1, VERS-TLS1.2


Well Sean, even though:
seahorse 116 :/optics1/home1/oo_processing> python
Python 2.6.6 (r266:84292, Aug 18 2016, 15:13:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

>>> import ssl
>>> ssl.OPENSSL_VERSION

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'OPENSSL_VERSION'

>>>

My subscriptions and 7.3 LUT updates seems to work but 7.2 LUTS update fail (no difference in the python version):
diff /optics1/virtual_ant/bin/cron/ocssw_7.3-CentOS6.csh /optics1/virtual_ant/bin/cron/ocssw_7.2-CentOS6.csh
3c3
< source /optics1/software/seadas/seadas-7.3/ocssw/OCSSW.env
---

> source /optics1/software/seadas/seadas-7.2/ocssw/OCSSW.env

And no difference in the update_luts.py either:

> diff /optics1/software/seadas/seadas-7.3/ocssw/run/scripts/update_luts.py /optics1/software/seadas/seadas-7.2/ocssw/run/scripts/update_luts.py

seahorse 281 :/optics1/virtual_ant/bin/seadas_l2_viirs_h5>

So I suspect there are other scripts that need to change other than just the update_luts.py?

Having said that, what cruel beasts decide that the week before Chirstmas, NASA should change! Some of us are in fact on holidays (but apparently not in deed ) until after the new year.

Well, it's working now right?  Walk out, close the door, shut the lights and forget about it until January

As to who decided the change needed to happen..."top men" (or women) above my pay grade.
The deadline we were given was Friday the 23rd.  I was the one who moved our switch up a couple days so the pain
wouldn't hit everyone on the the Friday before the holiday.

Sean

Sean,
I would but I think that some folks actually still use 7.2 for somethings.
Can all scripts in 7.2 $OCSSWROOT/run/scripts/ be copied over to the 7.2 $OCSSWROOT/run/scripts/ directory without too much fear?

Brock
PS, I want names....

> Can all scripts in 7.2 $OCSSWROOT/run/scripts/ be copied over to the 7.2 $OCSSWROOT/run/scripts/ directory without too much fear?

*probably*  The easiest way to test is this (while in the $OCSSWROOT/run/scripts/ directory):
git checkout v7.3

If all goes well, done.  If not:
git checkout v7.2

> PS, I want names....

This is NASA, how about an acronym: POTUS
https://www.whitehouse.gov/blog/2015/06/08/https-everywhere-government

Ah, so the POTUS June 8 directive somehow became the NASA Christmas directive....

Having said that , is there some way to ensure that the manual changes that I make to scripts like anc_utils.py or configurations I change in L1B_template.pcf are not overwritten with every git update?

Happy New Years!

"Per the issuance of this Memorandum, all publicly accessible Federal websites must meet the HTTPS-Only Standard by December 31st of 2016."

> ...is there some way to ensure that the manual changes that I make...not overwritten with every git update?

Create a branch and commit your changes to it:

git checkout -b "my-spiffy-new-code" v7.3
git add <files I changed>
git commit

Sean