Query and download Sentinel data from CDSE

[quinten]

The Copernicus Data Hubs will close soon (September 2023) and existing download clients will very likely cease to work. The Copernicus Data Space Environment (CDSE) has taken over the data querying and distribution of Sentinel data. I have implemented a simple query and download system to use CDSE to get Sentinel-2 and Sentinel-3 data that can be used in ACOLITE processing. This system is now integrated in the ACOLITE code (commit) on GitHub and can be freely tested. (Do a git pull first. ) I am sure there will be better packages coming soon.

[Edit 2024-04-27] I have now moved the cdse module into ac.api, so change the call in the examples below from ac.cdse to ac.api.cdse

To use the CDSE please make an account at https://dataspace.copernicus.eu/ and put your credentials in your .netrc file:

Code: Select all

machine cdse
login YOUR_LOGIN
password YOUR_PASSWORD

Make sure your .netrc file has no group or world read access, i.e. -rw-------. You can set this for example by:

Code: Select all

chmod 600 ~/.netrc

Alternatively you can set CDSE_u=YOUR_LOGIN and CDSE_p=YOUR_PASSWORD in either your OS environment variables or in the acolite/config/config.txt file.

After importing ACOLITE:

Code: Select all

import sys, os
user_home = os.path.expanduser("~")
sys.path.append(user_home+'/git/acolite')
import acolite as ac

You can query based on a region of interest (either a WKT string, polygon file or ACOLITE S,W,N,E limit) for a specific satellite (Sentinel-2 or Sentinel-3). The following example lists urls and scene names with <10% cloud cover over the Blue Accelerator platform near Oostende, Belgium using a simple point WKT:

Code: Select all

urls, scenes = ac.api.cdse.query(
    roi='POINT (2.9192711575039305 51.24648601469525)',
    collection = "SENTINEL-2", start_date = '2023-05-01', end_date = '2023-06-01', cloud_cover = 10)

Note that the cloud cover parameter is the scene wide fraction (0-100) as provided by ESA/Copernicus.

These urls/scenes can then be downloaded to a specific directory:

Code: Select all

local_scenes = ac.api.cdse.download(urls, scenes=scenes, output = None)

If output is set to None, scenes will be downloaded in the current working directory, and you may prefer to specify another path to store downloaded scenes. By default the zip files are extracted and deleted after extraction. This can be switched off by passing extract_zip=False and delete_zip=False.

A sample run of the last two examples in a Jupyter notebook:

CDSE_S2.png (182.08 KiB) Viewed 45879 times

The paths to the local scenes can then be passed to ACOLITE processing, i.e. using ac.acolite.acolite_run(), see for example here.

[quinten]

An alternative option is to have ACOLITE download a specific scene directly from CDSE, by specifying the scene name as the inputfile, and setting scene_download to True. If the local path does not exist, CDSE will then be queried for the scene. For example, to download and process a specific scene with ACOLITE, in a text settings file:

Code: Select all

inputfile=S3B_OL_1_EFR____20230530T095249_20230530T095549_20230530T205046_0179_080_079_2160_PS2_O_NT_003.SEN3
output=full path to output location
scene_download=True

or by using ac.acolite.acolite_run():

Code: Select all

import sys, os
user_home = os.path.expanduser("~")
sys.path.append(user_home+'/git/acolite')
import acolite as ac

settings = {}
settings['inputfile'] = 'S3B_OL_1_EFR____20230530T095249_20230530T095549_20230530T205046_0179_080_079_2160_PS2_O_NT_003.SEN3'
settings['output'] = full path to output location
settings['scene_download']=True

ac.acolite.acolite_run(settings)

You can also specify the local storage directory with scene_download_directory if it should be different than the main output directory. If the requested scene (extracted or not) is already present in that directory it should not be downloaded again.

A sample run of the last example in a Jupyter notebook (note the setting cdse_download was renamed to scene_download after this screenshot was made):

CDSE_S3.png (178.43 KiB) Viewed 45883 times

A sample run of the last example for a Sentinel-2 scene, also setting a limit over the Spuikom in Oostende and scene_download_directory in a Jupyter notebook:

scene_download_sentinel2.png (131.09 KiB) Viewed 45712 times

[quinten]

Please note the changes from ac.cdse to ac.api.cdse in the messages above!