Proxy for ancillary data download

[oneLaker]

Hi Quinten,

As you said, the ancillary data didn't have ano significant impact on the final results, however I would like to use them in processing now.
In China, because of goverment policy, I cannot access directly ocean color data without vpn proxy...

For the SeaDAS, the getanc.py and ProcUtils.py used the proxy from python, so that I can download the ancillary data sucessfully via a proxy of vpn. However, it didn't work for the acolite. I looked over the scripts of ancillary_download.py and download_file.py, they didn't use the proxy.
I would like to know whether there is a plan to fix it in future, for example like the getanc.py?

Best Regards,
Zhigang Cao

[quinten]

Hi Zhigang

Could you run the following script to see if the download succeeds using the system proxies?

Code: Select all

    url = "https://oceandata.sci.gsfc.nasa.gov/cgi/getfile/S20130940009423_TOAST.OZONE"
    file = "S20130940009423_TOAST.OZONE"

    proxies = None
    auth = None
    
    import requests
 
    import os
    file_path = os.path.abspath(file)
    start = time.time()

    if proxies is None:     
        proxies = {}
        for protocol in ['http', 'https']:
            proxies[protocol]=os.environ.get('{}_proxy'.format(protocol))        
        
    r = requests.get(url, stream=True, auth=auth, proxies=proxies)

    if (r.ok):
            with open(file_path, 'wb') as f:
                for chunk in r.iter_content(chunk_size=1024): 
                    if chunk: # filter out keep-alive new chunks
                        f.write(chunk)
    else:
        print(r.text)

Quinten

[oneLaker]

Hi Quinten,

I cannot obtain the ancillary data successfully with this command. Internet is a exactly BIG problem for us. Proxy is a good way to access the web via browser, but it usually does have a problem to access the data with terminal/Python. In fact, it took me much time to make it work in the SeaDAS(for example, getanc.py) processing.

Given that the impact of ancillary data is not very significant, I can process the images with the default meteorological data.

Best Regards,
Zhigang

[quinten]

Dear Zhigang

Thanks for your reply. If you can access the ancillary data through other means (e.g. wget), then you can always download the required (or all) ancillary data files and put them in the Data/MET folder.

Quinten