Search found 40 matches
- Thu Apr 18, 2024 8:17 pm
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
To address the use case question-- size of AOI and the merging scenario. Ultimately, the AOI is the entire East and Gulf Coast of the U.S. For tiling, are you still recommending 6x6 KM? If so, I assume a dsf_aot_estimate of fixed would be appropriate for this size of an AOI. The merging question is ...
- Thu Apr 18, 2024 3:39 pm
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Hi Quinten, I will follow up with more specifics about the AOIs and question of merging after this post. I tested the commit https://github.com/acolite/acolite/tree/23af717733a6ff1756951c980c4f6a14b5f30b21 with the change to tiles_interpn. I called acolite using my Python 3.12.3 environment. During ...
- Thu Apr 18, 2024 2:59 pm
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Hi Quinten, Your explanation regarding the yaml file makes sense. Yes, I was able to successfully process a full Sentinel-2 scene at 10m res on linux using acolite-20220222.0 as a command line process. Starting now, I will test the more recent versions starting as well as grab your latest commit. Th...
- Thu Apr 18, 2024 2:07 pm
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Hi Quinten, Thank you for investigating this and for committing a potential fix so quickly while always having prompt replies to my questions. Outside of acolite, I've also experienced high memory usage with scipy.interpolate.griddata and, at one point, was considering using an approach to chunk dat...
- Thu Apr 18, 2024 12:11 am
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Following up... I ran a number of different versions of acolite and in different environments. The step around "Created /tmp/S2A_MSIL1C_20230701T155141_N0509_R011_T18SVE_20230701T210549.SAFE/S 2A_MSI_2023_07_01_15_53_26_T18SVE_L1R.nc" was using a large amount of memory on the most recent v...
- Wed Apr 17, 2024 8:00 pm
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Hi Quinten, I've tried various python versions and numpy versions. The latest runs are conducted using: python 3.12.3 (previously ran python 3.10.12) and current numpy version is 1.26.4 Choosing best fitting model: min_drmsd (2 bands) Computing RMSD for model ACOLITE-LUT-202110-MOD1 Computing RMSD f...
- Wed Apr 17, 2024 10:42 am
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Still running into the same issue with the revised settings file. I will continue to investigate.
Thank you.
-Bryan
Thank you.
-Bryan
- Wed Apr 17, 2024 10:11 am
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Re: Memory issues when processing a full Sentinel-2 scene on linux
Hi Quinten, Thank you for the reply. On my Windows laptop, the processing completed successfully with s2_target_res=10, and I did not run into any memory issues. I only seem to be having this trouble on linux. The attempted creation of the 28 GB array was also a big question and red flag for me-- on...
- Wed Apr 17, 2024 6:05 am
- Forum: Support
- Topic: Memory issues when processing a full Sentinel-2 scene on linux
- Replies: 14
- Views: 55852
Memory issues when processing a full Sentinel-2 scene on linux
Hi, I'm attempting to process a full Sentinel-2 scene "S2A_MSIL1C_20230701T155141_N0509_R011_T18SVE_20230701T210549." On my Windows laptop with 16 GB of RAM, I am able to process the scene without trouble. However, when I run the same scene with the same settings in a linux environment usi...
- Tue Dec 12, 2023 3:41 am
- Forum: Results and discussion
- Topic: NEGATIVE rhos_ (bit flag 3 value 8) for entire scene Sentinel-2 DSF
- Replies: 12
- Views: 112213
Re: NEGATIVE rhos_ (bit flag 3 value 8) for entire scene Sentinel-2 DSF
Hi Quinten,
Thank you for the information.
I sincerely apologize for the late reply. I will attempt to get a list of scenes to you by the end of this week.
-Bryan
Thank you for the information.
I sincerely apologize for the late reply. I will attempt to get a list of scenes to you by the end of this week.
-Bryan