Bit Masking

Masking No Data Values

datacube.utils.masking.mask_invalid_data(data, keep_attrs=True)[source]

Sets all nodata values to nan.

This will convert numeric data to type float.

Parameters

  • data (Dataset or DataArray) –

  • keep_attrs (bool) – If the attributes of the data should be included in the returned .

Returns

Dataset or DataArray

Masking with Bit-Flag Measurements

One of the common types of data used with the Data Cube contains discrete values stored within a numeric value. These values are often classifications and outputs from tests, and need to be interpreted in specific ways, not as simple scalar values. They are often used as a mask to exclude observations which are deemed unsuitable for a given analysis. For example, we want to exclude observations of clouds when we are interested in what is on the ground.

Several methods are used when encoding these types of variables:

  • On-off bit flags, for a particular binary bit

  • Collections of bits that can indicate more than two possible values

  • Looking for a specific value stored using all available bits in the variable

From prior work, it is very easy to make mistakes when using these types of variables, which can lead to processing the wrong set of observations, and also making it quite difficult to read the code using them, and ensuring that they are used in a consistent way in different places.

Open Data Cube provides a way of describing the meanings that can be encoded in variables, which can then be used to give a readable method when using that variable.

How to Define Meanings on Measurements

How to Create Masks within code

datacube.utils.masking.describe_variable_flags(variable, with_pandas=True)[source]

Returns either a Pandas Dataframe (with_pandas=True - default) or a string (with_pandas=False) describing the available flags for a masking variable

Interprets the flags_definition attribute on the provided variable and returns a Pandas Dataframe or string like:

Bits are listed from the MSB (bit 13) to the LSB (bit 0)
Bit     Value   Flag Name            Description
13      0       cloud_shadow_fmask   Cloud Shadow (Fmask)
12      0       cloud_shadow_acca    Cloud Shadow (ACCA)
11      0       cloud_fmask          Cloud (Fmask)
10      0       cloud_acca           Cloud (ACCA)
Parameters

variable – Masking Dataset or DataArray

Returns

Pandas Dataframe or str

datacube.utils.masking.make_mask(variable, **flags)[source]

Returns a mask array, based on provided flags

When multiple flags are provided, they will be combined in a logical AND fashion.

For example:

>>> make_mask(pqa, cloud_acca=False, cloud_fmask=False, land_obs=True)

OR

>>> make_mask(pqa, **GOOD_PIXEL_FLAGS)

where GOOD_PIXEL_FLAGS is a dict of flag_name to True/False

Parameters
Returns

boolean DataArray or Dataset