Not logged inOcean Color Forum

The forum is locked.

The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.

Up Topic SeaDAS / SeaDAS - General Questions / error about install processors (locked)
- By gaofei Date 2014-06-13 22:17
I installed the seadas_7.0.2_linux64_installer.sh on Fedora 20.
When install aqua in the install processors, find error.
execution exception: java.io.IOException: install_ocssw.py failed with exit code 1.
Check log for more details.
Installing bundles.sha256sum (1 of 13)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
100  2423  100  2423    0     0    389      0  0:00:06  0:00:06 --:--:--   652
Installing common (2 of 13)
** Resuming transfer from byte position 353153788
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0  918M    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.
Error - Could not run "cd /usr/local/seadas-7.0.2/ocssw; curl -O --retry 5 --retry-delay 5 -C - http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle"
- By ghulampitt Date 2014-06-14 03:18
may be the problem is came from your cURL version, check the requirement on this link http://seadas.gsfc.nasa.gov/requirements.html
- By gaofei Date 2014-06-23 23:31
Dear ghulampitt,
     My curl version is the 7.37.0-2.0, but the question still came.
- By gnwiii Date 2014-06-24 08:16 Edited 2014-06-24 08:29
Try something like:


curl -v -O --retry 5 --retry-delay 5 -C - http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle


This should resume the download but will provide details of the session that may help understand the problem.

I started a download and killed it to get a partial download:


$ cat /etc/os-release
NAME=Fedora
VERSION="20 (Heisenbug)"
ID=fedora
VERSION_ID=20
PRETTY_NAME="Fedora 20 (Heisenbug)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:fedoraproject:fedora:20"
HOME_URL="https://fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=20
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=20
$ ls -s
149528 common.bundle


Attempting to restart gives the same error you found:


$ curl -v -O --retry 5 --retry-delay 5 -C - http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle
* Adding handle: conn: 0xf07c70
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0xf07c70) send_pipe: 1, recv_pipe: 0
** Resuming transfer from byte position 153112576
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to oceandata.sci.gsfc.nasa.gov port 80 (#0)
*   Trying 169.154.128.84...
* Connected to oceandata.sci.gsfc.nasa.gov (169.154.128.84) port 80 (#0)



> GET /ocssw/common.bundle HTTP/1.1
> Range: bytes=153112576-
> User-Agent: curl/7.32.0
> Host: oceandata.sci.gsfc.nasa.gov
> Accept: */*
>


< HTTP/1.1 200 OK
< Date: Tue, 24 Jun 2014 12:08:49 GMT
< Content-Type: application/octet-stream
< Content-Length: 963196648
< Last-Modified: Mon, 16 Sep 2013 13:22:00 GMT
< Connection: keep-alive
< Expires: Wed, 25 Jun 2014 00:08:49 GMT
< Cache-Control: max-age=43200
< Cache-Control: public
< Content-Security-Policy: default-src oceandata.sci.gsfc.nasa.gov oceandata.gsfc.nasa.gov oceandata101.domain.pub oceandata101 'unsafe-inline';
<
* HTTP server doesn't seem to support byte ranges. Cannot resume.
  0  918M    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.

You may have better results using wget with the manual install method from Ocean Data Science Software Repositories

[Edit:] I also tried resuming the download on another system (linux Mint) with curl-7.35.0 and got the same message.
- By seanbailey Date 2014-06-24 19:15
I'm a little confused by this behavior, mostly because I cannot replicate it.  Doing the steps as you outlined I was able to successfully continue an interrupted download using curl directly and
by manually interrupting the install_ocssw.py script and restarting it.  I am using curl v7.36.0, but I can't imagine that small change in version would keep curl from properly continuing.  Our servers ARE configured to accept the range header (and thus properly continue).

If you can provide the IP address for the machine(s) failing to reconnect, along with the date/time of the failure, perhaps our network gurus can see what I cannot...

Sean
- By gnwiii Date 2014-06-25 06:57 Edited 2014-06-28 10:07
I just did the test using curl v7.37.0 and that failed.  It would not be a surprise to find that the problem is with our site's firewall. 
Resuming downloads using curl (v7.35.0 on linux or 7.37.0 on OS X) does work from a different site.

Google says my public IP address is 134.190.3.121.  The latest test (see timestamp) gave:


$ curl -v -O --retry 5 --retry-delay 5 -C - http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle
* Hostname was NOT found in DNS cache
*   Trying 169.154.128.84...
** Resuming transfer from byte position 4087808
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*
Connected to oceandata.sci.gsfc.nasa.gov (169.154.128.84) port 80 (#0)



> GET /ocssw/common.bundle HTTP/1.1
> Range: bytes=4087808-
> User-Agent: curl/7.37.0
> Host: oceandata.sci.gsfc.nasa.gov
> Accept: */*
>


< HTTP/1.1 200 OK
< Date: Wed, 25 Jun 2014 10:38:42 GMT
< Content-Type: application/octet-stream
< Content-Length: 963196648
< Last-Modified: Mon, 16 Sep 2013 13:22:00 GMT
< Connection: keep-alive
< Expires: Wed, 25 Jun 2014 22:38:42 GMT
< Cache-Control: max-age=43200
< Cache-Control: public
< Content-Security-Policy: default-src oceandata.sci.gsfc.nasa.gov oceandata.gsfc.nasa.gov oceandata101.domain.pub oceandata101 'unsafe-inline';
<
* HTTP server doesn't seem to support byte ranges. Cannot resume.
  0  918M    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
curl: (33) HTTP server doesn't seem to support byte ranges. Cannot resume.

It is interesting that wget --continue ... does work with the same URL.
- By seanbailey Date 2014-06-29 14:51
Well, either Google is lying to you, or your firewall setup is interesting...I wasn't able to find any access to our servers from 134.190.3.121
but,  given the timestamp of the curl request found 205.193.112.245 made the connection.  There was a "client timed out" message
in the error logs, but that could have been associated with you manually terminating the connection.  I'll check with the gurus to see
if there is a possibility that the timeout window is too short.

Regards,
Sean
- By gnwiii Date 2014-06-29 15:58 Edited 2014-06-30 09:31
Our network configuration is very complex, so it is quite possible the IP that talks to google.ca  differs from the one that talks to nasa.gov.  

;; ANSWER SECTION: 245.112.193.205.in-addr.arpa. 384 IN  PTR  g-ex245.mar.dfo-mpo.gc.ca.

This looks right since most of our machines have mar.dfo-mpo.gc.ca domains.   There have been many reports over the years of curl failing in similar cases while wget succeeds, so it might be useful to compare the logic in the 2 programs.

Back in the office, using curl to get the http header shows no Accept-Ranges: line:


$ curl -I http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle
HTTP/1.1 200 OK
Date: Mon, 30 Jun 2014 13:23:31 GMT
Content-Type: application/octet-stream
Content-Length: 963196648
Last-Modified: Mon, 16 Sep 2013 13:22:00 GMT
Connection: keep-alive
Expires: Tue, 01 Jul 2014 01:23:31 GMT
Cache-Control: max-age=43200
Cache-Control: public
Content-Security-Policy: default-src oceandata.sci.gsfc.nasa.gov oceandata.gsfc.nasa.gov oceandata101.domain.pub oceandata101 'unsafe-inline';


For a local machine the Accept-Ranges: line does appear.


$ curl -I http://ambrosia/index.html | grep Accept
Accept-Ranges: bytes
- By seanbailey Date 2014-06-30 18:04
Try again.  One of the backend servers was configured differently from the others and requests served by that machine did indeed fail to continue (oddly just for curl).

Sean
- By gnwiii Date 2014-07-02 06:34
Now I do see the Accept-Ranges: bytes

$  curl -I http://oceandata.sci.gsfc.nasa.gov/ocssw/common.bundle
HTTP/1.1 200 OK
Date: Wed, 02 Jul 2014 10:25:56 GMT
Content-Type: application/octet-stream
Content-Length: 963196648
Last-Modified: Mon, 16 Sep 2013 13:22:00 GMT
Connection: keep-alive
Expires: Wed, 02 Jul 2014 22:25:56 GMT
Cache-Control: max-age=43200
Cache-Control: public
Content-Security-Policy: default-src oceandata.sci.gsfc.nasa.gov oceandata.gsfc.nasa.gov oceandata101.domain.pub oceandata101 'unsafe-inline';
Accept-Ranges: bytes


Was that backend server configured to omit the Accept-Ranges header, or did it really not support ranges?  The Accept-Ranges header  is optional (RFC2616 Section 14.5), but maybe curl's logic requires the header (at least when "continuing" a partial transfer).
- By seanbailey Date 2014-07-02 07:58
It was configured to omit the Accept-Ranges header.  It supported ranges, and wget had no trouble with it before it was corrected to report the header.

Sean
- By melliott Date 2017-05-15 13:03 Edited 2017-05-15 13:06
The requirements link above is out of date. Try:
https://seadas.gsfc.nasa.gov/requirements/
Up Topic SeaDAS / SeaDAS - General Questions / error about install processors (locked)

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill