ACOLITE Docker / parallel processing
Posted: Thu Jan 21, 2021 7:46 am
You can run ACOLITE in Docker to allow easy cross platform deployment or to run multiple instances in parallel. It is not recommended to run multiple instances of other ACOLITE distributions in parallel.
To start it is assumed you have a working Docker installation. The attached zip file can then be easily used to set up a local copy of the ACOLITE Docker image. This zip file has a Dockerfile which gets the Ubuntu 20.04 image, updates it to the latest version, installs Miniconda and the ACOLITE dependencies from conda-forge. It then clones the ACOLITE GitHub code. Inside the zip file are also two empty directories, "input" and "output" and a default "settings" file. The input and output directories will at runtime be used as mount points for respectively the image to be processed and the output directory. The settings file can also be replaced with other versions at runtime, but note that the output option in your settings file should always start with "output=/output".
Set up the ACOLITE docker image - note this will take some time as software is downloaded and installed:
When the ACOLITE Docker image is complete it can be run specifying the paths to an input file and a target output directory:
Optionally a settings file can be provided - make sure to have the output option in that settings file starting with /output, e.g. "output=/output/custom-settings":
Several processing instances can be started at the same time, e.g. using a list of files and GNU parallel:
You can save the Docker image, this step will take a while:
and transfer it to another machine:
I hope this is useful! Please let me know if you have any comments on this approach!
Quinten
To start it is assumed you have a working Docker installation. The attached zip file can then be easily used to set up a local copy of the ACOLITE Docker image. This zip file has a Dockerfile which gets the Ubuntu 20.04 image, updates it to the latest version, installs Miniconda and the ACOLITE dependencies from conda-forge. It then clones the ACOLITE GitHub code. Inside the zip file are also two empty directories, "input" and "output" and a default "settings" file. The input and output directories will at runtime be used as mount points for respectively the image to be processed and the output directory. The settings file can also be replaced with other versions at runtime, but note that the output option in your settings file should always start with "output=/output".
Set up the ACOLITE docker image - note this will take some time as software is downloaded and installed:
Code: Select all
unzip acolite-docker.zip
cd acolite-docker
docker build -t acolite .
Code: Select all
input=/path/to/scene
output=/path/to/output
docker run -d -v $input:/input -v $output:/output acolite
Code: Select all
input=/path/to/scene
output=/path/to/output
settings=/path/to/settings
docker run -d -v $input:/input -v $output:/output -v $settings:/settings acolite
Code: Select all
parallel -j 150% docker run -d -v {}:/input -v $output:/output -v $settings:/settings acolite ::: `cat files.txt`
Code: Select all
docker save acolite | gzip > acolite-docker.tar.gz
Code: Select all
docker load < acolite-docker.tar.gz
Quinten