Open Data Cube Manual¶
The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth observation satellite and related data from multiple satellite and other acquisition systems.
Documentation¶
See the user guide for installation and usage of the datacube, and for documentation of the API.
Join our Slack if you need help setting up or using the Open Data Cube.
Please help us to keep the Open Data Cube community open and inclusive by reading and following our Code of Conduct.
Developer setup¶
Clone:
git clone https://github.com/opendatacube/datacube-core.git
Create a Python environment for using the ODC. We recommend conda as the easiest way to handle Python dependencies.
conda create -n odc -c conda-forge python=3.8 datacube pre_commit
conda activate odc
Install a develop version of datacube-core.
cd datacube-core
pip install --upgrade -e .
Install the pre-commit hooks to help follow ODC coding conventions when committing with git.
pre-commit install
Run unit tests + PyLint
./check-code.sh
(this script approximates what is run by Travis. You can alternatively run
pytest
yourself). Some test dependencies may need to be installed, attempt to install these using:pip install --upgrade -e '.[test]'
If install for these fails please lodge them as issues.
(or) Run all tests, including integration tests.
./check-code.sh integration_tests
Assumes a password-less Postgres database running on localhost called
agdcintegration
Otherwise copy
integration_tests/agdcintegration.conf
to~/.datacube_integration.conf
and edit to customise.
Alternatively one can use the opendatacube/datacube-tests
docker image to run
tests. This docker includes database server pre-configured for running
integration tests. Add --with-docker
command line option as a first argument
to ./check-code.sh
script.
./check-code.sh --with-docker integration_tests
Developer setup on Ubuntu¶
Building a Python virtual environment on Ubuntu suitable for development work.
Install dependencies:
sudo apt-get update
sudo apt-get install -y \
autoconf automake build-essential make cmake \
graphviz \
python3-venv \
python3-dev \
libpq-dev \
libyaml-dev \
libnetcdf-dev \
libudunits2-dev
Build the python virtual environment:
pyenv="${HOME}/.envs/odc" # Change to suit your needs
mkdir -p "${pyenv}"
python3 -m venv "${pyenv}"
source "${pyenv}/bin/activate"
pip install -U pip wheel cython numpy
pip install -e '.[dev]'
pip install flake8 mypy pylint autoflake black
- About & Core Concepts
- Overview
- Use Cases
- Overview & Use Cases
- Architectural Overview
- Datasets
- Types of Datasets
- Products
- Metadata Types
- Dataset Documents
- Extensions & Related Software
- Existing Deployments
- What’s New
- v1.8.next
- v1.8.6 (30 September 2021)
- v1.8.5 (18 August 2021)
- v1.8.4 (6 August 2021)
- v1.8.3 (18 August 2020)
- v1.8.2 (10 July 2020)
- v1.8.1 (2 July 2020)
- v1.8.0 (21 May 2020)
- v1.7.0 (16 May 2019)
- v1.7.0rc1 (18 April 2019)
- v1.6.1 (27 August 2018)
- v1.6.0 (23 August 2018)
- v1.6rc2 (29 June 2018)
- v1.6rc1 Easter Bilby (10 April 2018)
- v1.5.4 Dingley Dahu (13th December 2017)
- v1.5.3 Purpler Unicorn with Starlight (16 October 2017)
- v1.5.2 Purpler Unicorn with Stars (28 August 2017)
- v1.5.4 Dingley Dahu (13th December 2017)
- v1.5.3 Purpler Unicorn with Starlight (16 October 2017)
- v1.5.2 Purpler Unicorn with Stars (28 August 2017)
- v1.5.1 Purpler Unicorn (13 July 2017)
- v1.5.0 Purple Unicorn (9 July 2017)
- v1.4.1 (25 May 2017)
- v1.4.0 (17 May 2017)
- v1.3.1 (20 April 2017)
- v1.3.0
- v1.2.2
- v1.2.0 Boring as Batman (15 February 2017)
- v1.1.18 Mushroom Milkshake (9 February 2017)
- v1.1.17 Happy Festivus Continues (12 January 2017)
- v1.1.16 Happy Festivus (6 January 2017)
- v1.1.15 Minion Party Hangover (1 December 2016)
- v1.1.14 Minion Party (30 November 2016)
- v1.1.13 Black Goat (15 November 2016)
- v1.1.12 Unnamed Unknown (1 November 2016)
- v1.1.11 Unnamed Unknown (19 October 2016)
- v1.1.10 Rabid Rabbit (5 October 2016)
- v1.1.9 Pest Hippo (20 September 2016)
- v1.1.8 Last Mammoth (5 September 2016)
- v1.1.7 Bit Shift (22 August 2016)
- v1.1.6 Lightning Roll (8 August 2016)
- v1.1.5 Untranslatable Sign (26 July 2016)
- v1.1.4 Imperfect Inspiration (12 July 2016)
- v1.1.3 Speeding Snowball (5 July 2016)
- v1.1.2 Wind Chill (28 June 2016)
- v1.1.1 Good Idea (23 June 2016)
- v1.1.0 No Spoon (3 June 2016)
- 1.0.4 Square Clouds (3 June 2016)
- 1.0.3 (14 April 2016)
- 1.0.2 (23 March 2016)
- 1.0.1 (18 March 2016)
- 1.0.0 (11 March 2016)
- pre-v1 (end 2015)
- Release Process
- ODC License
- Glossary
- Data Access & Analysis
- Connecting to the Open Datacube
- Dataset Searching & Querying
- Product Discovery
- Data Loading
- Virtual Products
- Data Loading in Depth
- Using Juypter Notebooks
- Administration
- Mac OSX Developer Setup
- Ubuntu Developer Setup
- Windows Developer Setup
- Command Line Tools
- Database Setup
- Step-by-step Guide to Indexing Data
- Indexing data from Amazon (AWS S3)
- Extending the Open Data Cube
- Data Preperation Scripts
- Landsat Samples
- 1. Preparing USGS Landsat Collection 1 - LEVEL1
- 2. Preparing USGS Landsat Surface Reflectance - LEDAPS
- 3. Indexing data on AWS, an example using Sentinel-2
- Custom Prepare Scripts
- Ingesting Data
- API Reference
- Datacube Class
- Dataset
- Dataset Type
- Measurement
- Metadata Type
- Range
- Query
- Tile
- Dataset & Product Discovery
- Dataset Querying
- datacube.index._datasets.DatasetResource.get
- datacube.index._datasets.DatasetResource.search
- datacube.index._datasets.DatasetResource.search_by_metadata
- datacube.index._datasets.DatasetResource.search_by_product
- datacube.index._datasets.DatasetResource.search_eager
- datacube.index._datasets.DatasetResource.search_product_duplicates
- datacube.index._datasets.DatasetResource.search_returning
- datacube.index._datasets.DatasetResource.search_summaries
- datacube.index._datasets.DatasetResource.has
- datacube.index._datasets.DatasetResource.bulk_has
- datacube.index._datasets.DatasetResource.can_update
- datacube.index._datasets.DatasetResource.count
- datacube.index._datasets.DatasetResource.count_by_product
- datacube.index._datasets.DatasetResource.count_by_product_through_time
- datacube.index._datasets.DatasetResource.count_product_through_time
- datacube.index._datasets.DatasetResource.get_derived
- datacube.index._datasets.DatasetResource.get_field_names
- datacube.index._datasets.DatasetResource.get_locations
- datacube.index._datasets.DatasetResource.get_archived_locations
- datacube.index._datasets.DatasetResource.get_datasets_for_location
- Product Querying
- datacube.index._products.ProductResource.can_update
- datacube.index._products.ProductResource.get
- datacube.index._products.ProductResource.get_by_name
- datacube.index._products.ProductResource.get_with_fields
- datacube.index._products.ProductResource.search
- datacube.index._products.ProductResource.search_robust
- datacube.index._products.ProductResource.get_all
- Data Loading
- Bit Masking
- Dataset Writing
- datacube.index._datasets.DatasetResource.add
- datacube.index._datasets.DatasetResource.add_location
- datacube.index._datasets.DatasetResource.archive
- datacube.index._datasets.DatasetResource.archive_location
- datacube.index._datasets.DatasetResource.remove_location
- datacube.index._datasets.DatasetResource.restore
- datacube.index._datasets.DatasetResource.restore_location
- datacube.index._datasets.DatasetResource.update
- Product Writing
- Creating Geometries
- datacube.utils.geometry.point
- datacube.utils.geometry.multipoint
- datacube.utils.geometry.line
- datacube.utils.geometry.multiline
- datacube.utils.geometry.polygon
- datacube.utils.geometry.multipolygon
- datacube.utils.geometry.multigeom
- datacube.utils.geometry.box
- datacube.utils.geometry.sides
- datacube.utils.geometry.polygon_from_transform
- Spatial Operations
- datacube.utils.geometry.unary_union
- datacube.utils.geometry.unary_intersection
- datacube.utils.geometry.bbox_union
- datacube.utils.geometry.bbox_intersection
- datacube.utils.geometry.lonlat_bounds
- datacube.utils.geometry.projected_lon
- datacube.utils.geometry.clip_lon180
- datacube.utils.geometry.chop_along_antimeridian
- Utilities
- datacube.utils.geometry.assign_crs
- datacube.utils.geometry.crs_units_per_degree
- datacube.utils.geometry.geobox_union_conservative
- datacube.utils.geometry.geobox_intersection_conservative
- datacube.utils.geometry.scaled_down_geobox
- datacube.utils.geometry.intersects
- datacube.utils.geometry.common_crs
- datacube.utils.geometry.is_affine_st
- datacube.utils.geometry.apply_affine
- datacube.utils.geometry.roi_boundary
- datacube.utils.geometry.roi_is_empty
- datacube.utils.geometry.roi_is_full
- datacube.utils.geometry.roi_intersect
- datacube.utils.geometry.roi_shape
- datacube.utils.geometry.roi_normalise
- datacube.utils.geometry.roi_from_points
- datacube.utils.geometry.roi_center
- datacube.utils.geometry.roi_pad
- datacube.utils.geometry.scaled_down_shape
- datacube.utils.geometry.scaled_down_roi
- datacube.utils.geometry.scaled_up_roi
- datacube.utils.geometry.decompose_rws
- datacube.utils.geometry.affine_from_pts
- datacube.utils.geometry.get_scale_at_point
- datacube.utils.geometry.native_pix_transform
- datacube.utils.geometry.compute_reproject_roi
- datacube.utils.geometry.split_translation
- datacube.utils.geometry.compute_axis_overlap
- datacube.utils.geometry.w_
- datacube.utils.geometry.warp_affine
- datacube.utils.geometry.rio_reproject
- Geometry Classes
- datacube.utils.geometry.Coordinate
- datacube.utils.geometry.BoundingBox
- datacube.utils.geometry.CRS
- datacube.utils.geometry.Geometry
- datacube.utils.geometry.GeoBox
- datacube.utils.geometry.BoundingBox
- datacube.utils.geometry.gbox.GeoboxTiles
- datacube.model.GridSpec
- datacube.utils.geometry.CRSError
- datacube.utils.geometry.CRSMismatchError
- Amazon Web Services (AWS)
- datacube.utils.aws.s3_client
- datacube.utils.aws.s3_open
- datacube.utils.aws.s3_head_object
- datacube.utils.aws.s3_fetch
- datacube.utils.aws.s3_dump
- datacube.utils.aws.s3_url_parse
- datacube.utils.aws.auto_find_region
- datacube.utils.aws.get_aws_settings
- datacube.utils.aws.get_creds_with_retry
- datacube.utils.aws.mk_boto_session
- datacube.utils.aws.ec2_current_region
- datacube.utils.aws.ec2_metadata
- datacube.utils.aws.configure_s3_access
- Writing GeoTIFFs
- Dask
- Grid Workflow