Search found 40 matches

by beder
Thu Apr 18, 2024 8:17 pm
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

To address the use case question-- size of AOI and the merging scenario. Ultimately, the AOI is the entire East and Gulf Coast of the U.S. For tiling, are you still recommending 6x6 KM? If so, I assume a dsf_aot_estimate of fixed would be appropriate for this size of an AOI. The merging question is ...
by beder
Thu Apr 18, 2024 3:39 pm
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Hi Quinten, I will follow up with more specifics about the AOIs and question of merging after this post. I tested the commit https://github.com/acolite/acolite/tree/23af717733a6ff1756951c980c4f6a14b5f30b21 with the change to tiles_interpn. I called acolite using my Python 3.12.3 environment. During ...
by beder
Thu Apr 18, 2024 2:59 pm
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Hi Quinten, Your explanation regarding the yaml file makes sense. Yes, I was able to successfully process a full Sentinel-2 scene at 10m res on linux using acolite-20220222.0 as a command line process. Starting now, I will test the more recent versions starting as well as grab your latest commit. Th...
by beder
Thu Apr 18, 2024 2:07 pm
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Hi Quinten, Thank you for investigating this and for committing a potential fix so quickly while always having prompt replies to my questions. Outside of acolite, I've also experienced high memory usage with scipy.interpolate.griddata and, at one point, was considering using an approach to chunk dat...
by beder
Thu Apr 18, 2024 12:11 am
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Following up... I ran a number of different versions of acolite and in different environments. The step around "Created /tmp/S2A_MSIL1C_20230701T155141_N0509_R011_T18SVE_20230701T210549.SAFE/S 2A_MSI_2023_07_01_15_53_26_T18SVE_L1R.nc" was using a large amount of memory on the most recent v...
by beder
Wed Apr 17, 2024 8:00 pm
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Hi Quinten, I've tried various python versions and numpy versions. The latest runs are conducted using: python 3.12.3 (previously ran python 3.10.12) and current numpy version is 1.26.4 Choosing best fitting model: min_drmsd (2 bands) Computing RMSD for model ACOLITE-LUT-202110-MOD1 Computing RMSD f...
by beder
Wed Apr 17, 2024 10:42 am
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Still running into the same issue with the revised settings file. I will continue to investigate.

Thank you.

-Bryan
by beder
Wed Apr 17, 2024 10:11 am
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Re: Memory issues when processing a full Sentinel-2 scene on linux

Hi Quinten, Thank you for the reply. On my Windows laptop, the processing completed successfully with s2_target_res=10, and I did not run into any memory issues. I only seem to be having this trouble on linux. The attempted creation of the 28 GB array was also a big question and red flag for me-- on...
by beder
Wed Apr 17, 2024 6:05 am
Forum: Support
Topic: Memory issues when processing a full Sentinel-2 scene on linux
Replies: 14
Views: 55852

Memory issues when processing a full Sentinel-2 scene on linux

Hi, I'm attempting to process a full Sentinel-2 scene "S2A_MSIL1C_20230701T155141_N0509_R011_T18SVE_20230701T210549." On my Windows laptop with 16 GB of RAM, I am able to process the scene without trouble. However, when I run the same scene with the same settings in a linux environment usi...
by beder
Tue Dec 12, 2023 3:41 am
Forum: Results and discussion
Topic: NEGATIVE rhos_ (bit flag 3 value 8) for entire scene Sentinel-2 DSF
Replies: 12
Views: 112213

Re: NEGATIVE rhos_ (bit flag 3 value 8) for entire scene Sentinel-2 DSF

Hi Quinten,

Thank you for the information.

I sincerely apologize for the late reply. I will attempt to get a list of scenes to you by the end of this week.

-Bryan