Download

Most of the times, you will use the storage for downloading data to use as input for processing algorithms.

This section describes how to manage the download resource location of a dataset indexed in the catalog and then use the data agency function for different purposes.

Direct download

Every entry in an index of the catalog can have a link to the data that it describes. There is a resource location (URL) also called enclosure that can be associated in the entry metadata. This URL can be extracted easily using opensearch-client. For instance,

opensearch-client -p do= "https://catalog.terradue.com/sentinel1/search" enclosure

Note

Please note the parameter do= that indicates to the catalog to NOT apply any delegate download.

Delegate Download

We just saw how to simply extract the enclosure of an entry. With the parameter downloadOrigin (do=) the catalog do not perform any change on the original enclosure link of the entry. However, by default and if the catalog provides with the downloadOrigin parameter, the opensearch-client set automatically the hostname of the machine in this parameter. If the same command as above is performed like this on machine sandbox1.dev.terradue.int

opensearch-client "https://catalog.terradue.com/sentinel1/search" enclosure

the catalog is queried as follow

https://catalog.terradue.com/sentinel1/search?do=sandbox1.dev.terradue.int

This information allows the catalog to apply a filter on the results and eventually modify the enclosure link of the entries. If the download origin is elligible to the data gateway proxy download, then the enclosure shall point to the closest data gateway to provide the data access via the best route.

Data Gateway Proxy Download

If the resource location returned by the catalog points to a data gateway like

https://store.terradue.com/download/sentinel1/files/v1/S1A_IW_SLC__1SDH_20160915T090555_20160915T090624_013061_014B4B_4793

then the download is performed via the Data Gateway that enables many function such as caching to allow the best download performance of the data requested.

Warning

Please be aware that the Data Gateway Proxy Download may take some time to start depending of the configuration of the repository from which you perform the download. Indeed, the data gateway may be required to perform Implicit caching before delivering the data.