Not logged inOcean Color Forum
Up Topic Products and Algorithms / Satellite Data Access / curling for MERIS/OLCI
- By tsmoore00 Date 2017-10-03 11:35
Hi,

I am having a problem downloading MERIS or OLCI data using curl from the servers.  I can get Modis data ok, so it seems my curl is working (Mac OSX Sierra 10.12.6, curl curl 7.52.1).

I'm guessing it might be something do to with my user:pass, but I feel like I am using it ok:

curl -u username:passwd -L -O <file>

I am using my Earthdata login user and passwd info.

I get a file but its all metadata html style text that I can't decipher.

Any suggestions?

Thanks,
Tim
- By gnwiii Date 2017-10-03 12:37
Sounds familiar.  Registration says:

"For those users who have already created accounts via Earthdata Login, you will not need to reregister but you may be prompted to authorize a particular application/function/service on our website if you have not used it before. When that occurs, you will be redirected to the Earthdata Login page and asked to authorize that service and then once you agree, you will be automatically redirected back to our website to resume your session." 

There is  chance the html page is asking you to authorize a service.  Have you tried loading the page in a browser to see if it can decipher the important message?
- By tsmoore00 Date 2017-10-04 10:27
Thanks.  I get redirected to the 'Agree' page, but I don't know how curl gets past that.

I can enter the first filename into a web browser, which also brings me to the 'Agree' page, which I agree with, and then a download starts.  All subsequent files entered into a browser are downloaded without redirecting to the Agree page, but curl still won't get past it.

If anyone has a way to get past the Agree page with curl, let me know!
Thanks!
Tim
- By SeanBailey Date 2017-10-05 21:02
Tim,

See the FAQ, or look below for the short version:

Configure your username and password for authentication using a .netrc file
  $ touch ~/.netrc
  $ echo "machine urs.earthdata.nasa.gov login <uid> password <password>" > ~/.netrc
  $ chmod 0600 .netrc

where <uid> is your Earthdata Login username and <password> is your Earthdata Login password.

Call cURL with a cookie:
  $ touch ~/.urs_cookies
curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n  https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/MER_RR__1PRLRA20120330_112205_000026183113_00138_52738_8486.N1.bz2 -O MER_RR__1PRLRA20120330_112205_000026183113_00138_52738_8486.N1.bz2

Sean
- By tsmoore00 Date 2017-10-06 08:12
Great!

That seems to do the trick.

Thanks!!
Tim
Up Topic Products and Algorithms / Satellite Data Access / curling for MERIS/OLCI

Powered by mwForum 2.29.7 © 1999-2015 Markus Wichitill