Most of the times, you will use the storage for downloading data to use as input for processing algorithms.
This section describes how to manage the download resource location of a dataset indexed in the catalog and then use the data agency function for different purposes.
Every entry in an index of the catalog can have a link to the data that it describes. There is a resource location (URL) also called enclosure that can be associated in the entry metadata. This URL can be extracted easily using opensearch-client. For instance,
opensearch-client -p do= "https://catalog.terradue.com/sentinel1/search" enclosure
Please note the parameter
do= that indicates to the catalog to NOT apply any delegate download.
We just saw how to simply extract the enclosure of an entry. With the parameter downloadOrigin (
do=) the catalog do not perform any change on the original enclosure link of the entry. However, by default and if the catalog provides with the downloadOrigin parameter, the opensearch-client set automatically the hostname of the machine in this parameter. If the same command as above is performed like this on machine
opensearch-client "https://catalog.terradue.com/sentinel1/search" enclosure
the catalog is queried as follow
This information allows the catalog to apply a filter on the results and eventually modify the enclosure link of the entries. If the download origin is elligible to the data gateway proxy download, then the enclosure shall point to the closest data gateway to provide the data access via the best route.
Data Gateway Proxy Download¶
If the resource location returned by the catalog points to a data gateway like
then the download is performed via the Data Gateway that enables many function such as caching to allow the best download performance of the data requested.
Please be aware that the Data Gateway Proxy Download may take some time to start depending of the configuration of the repository from which you perform the download. Indeed, the data gateway may be required to perform Implicit caching before delivering the data.