ValueError: shape mismatch: objects cannot be broadcast to a single shape

[quinten]

Hi Isabel

Could you also post the settings you used? I can take a closer look at what is causing the issue you and Brian are having.

If you are using a settings file from an earlier version I would recommend only copying the settings that are really needed for your application.

Quinten

[IsabelBrand]

Hi Quinten,

I only changed the following settings:

Attempt1, default.txt options plus map_l2w=True, l2w_export_geotiff=True, dsf_path_reflectance=tiled

Attempt2, default.txt options plus map_l2w=True, l2w_export_geotiff=True, dsf_path_reflectance=fixed

The shape mismatch is showing up only when using the newest version.

cheers,

Isabel

[quinten]

HI Isabel

Could you try the latest commit from this morning?

Is this a Coll2 image with resolved geometry? (You will have the SAA/SZA/VAA/VZA files in the bundle.)

Quinten

[IsabelBrand]

Hi Quinten,

Thanks for your quick reply and support.

Acolite is running without errors, after downloading the version of this morning. Thank you so much

I have two questions:

1. Do you think it is possible to flag cloud shadow (please see attached figure)? Cloud shadow are showing up as low spm concentrations.
2. Any suggestions on how to save geotiff without large values (e.g. something e36) in the areas outside a tile?
I'm using the following:
## convert to GeoTIFF
ac.output.nc_to_geotiff(outputfile)


Isabel

[quinten]

Hi Isabel

Cloud shadows are a known problem! They are not easy to mask spectrally as they just look like darker water. There are some methods that detect the clouds first, and then project the shadow on the surface. These methods are not perfect, as they have issues with distinguishing other bright targets (e.g. sandy beaches) and/or need to assume the cloud height. At the moment there is no attempt to mask cloud shadows in the ACOLITE outputs.

Perhaps setting the NoData Value helps when writing the GeoTIFF files. I have updated ac.output.nc_to_geotiff on GitHub. Does this improve the handling of the masks?

Quinten

[IsabelBrand]

Hi Quinten,

Thanks for your clear explanation about cloud shadows. I guess it is wise to avoid such images.

The new ac.output.nc_to_geotiff solved the problems with high values being saved to NANs.

I saw that you've changed the GitHub code, so my question is with what should I replace ac.acolite.acolite_settings in the following piece of code?

for bundle in bundles:
settings = ac.acolite.acolite_settings(None) # this acolite_settings.py I could not find on the new version.
settings['limit'] = limit
settings['inputfile'] = bundle
settings['output'] = odir
settings['ancillary'] = False
settings['dsf_path_reflectance'] = 'fixed'


ac.acolite.acolite_run(settings=settings)

[quinten]

Hi Isabel

You can replace that acolite_settings call with:

Code: Select all

sensor = None
settings = ac.acolite.settings.parse(sensor)

If you know a priori the sensor you are processing, it is best to load the defaults for that sensor, e.g.:

Code: Select all

sensor = 'L5_TM'
settings = ac.acolite.settings.parse(sensor)

Quinten

[IsabelBrand]

Hi Quinten,

Thanks for your reply.

I tried both ways, but it seems acolite reads the bundles, gives me output files (l1r_settings.txt and log_file.txt) and stops somewhere.

I get these outputs...sorry for the long text.

I attached the file I am using to process these bundles.

Input files
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B1.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B2.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B3.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B4.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B5.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B6.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B7.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_MTL.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091004
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B1.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B1.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091004_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091004
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B2.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B2.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091004_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091004
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B3.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B3.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091004_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091005
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B4.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B4.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091005_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091005
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B5.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B5.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091005_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091005
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B6.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B6.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091005_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091005
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B7.TIF
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B7.TIF not recognized.
E:/rsdata/out/acolite_run_20210702_091005_l1r_settings.txt
Running generic ACOLITE processing - Generic Version Public Beta - April 2021
Run ID - 20210702_091005
E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_MTL.txt
Processed files

[quinten]

Hi Isabel

The inputfile to ACOLITE should be the directory containing the bands, not the individual band files. So in your example "E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410" rather than "E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410\LT05_L1GS_199023_20000410_20171210_01_T2_B1.TIF" and so on.

In your script you can probably change

Code: Select all

bundles = glob.glob('E:/rsdata/L5/unzipped/out2000/LT05_L1GS_199023_20000410/LT05*')

to

Code: Select all

bundles = glob.glob('E:/rsdata/L5/unzipped/out2000/LT05*')

to get all your L5 bundle directories.

Quinten

[IsabelBrand]

Hi Quinten,

Yes, it works now

Thank you so much for your support.

Have a nice day,

Isabel