ciop.copy¶

Description¶

Copies file from a URL to a local folder.

Usage: public catalogue¶

import cioppy
ciop = cioppy.Cioppy()

ciop.copy(urls, target)

Usage: private catalogue¶

When using JupyterLab, if the HOME environment variable is not declared run this code:

import os
import getpass

vm_user = getpass.getuser()
os.environ['HOME'] = '/home/{}'.format(vm_user)

Insert the credentials string using the form <user>:<password>

import cioppy
ciop = cioppy.Cioppy()

ciop.copy(urls, target, credentials='<user>:<password>')

Arguments¶

import cioppy
ciop = cioppy.Cioppy()

ciop.copy(urls, target, debugMode=False, credentials=None, extract=True, abortOnFirstError=False, quietmode=False,
           forceTransfer=True, createOutputDirectory=False, compressOutput=False, numRetries=None, intervalBetweenRetries=None,
           timeout=None, port=None, recursive=False, noRetryAfterTimeout=False, noHtmlAndRdfTags=False, skipIfSinkPathExists=False,
           tmpDir=None, excludePattern=None, privateMode=False)

urls: the URL of the product to download, or the URL to the OpenSearch catalogue entry in RDF format
target: the destination folder for the downloaded product
debugMode: set debug mode for command output parsing debugging
credentials: force <user> and <password> authentication. Credentials string is of the form: <user>:<password> NOTE: these passwords will be stored in clear text in the ~/.ciop-copy_cred file. NOTE: If username/passowrd is specified it has precedence over the certificate/proxy authentication
extract: if the product is an archive (.tgz, .gz) it is extracted by default
abortOnFirstError: abort on first error without attempting to process further URLs
quietmode: quiet mode, local filenames are not echoed to stdout after transfer
forceTransfer: force transfer of a physical copy of the file in case of nfs URLs
createOutputDirectory: creates the output directory if it does not exist
compressOutput: provide output as a compressed package (.gz for files or .tgz for folders). NOTE: that it will not compress already compressed files (.gz, .tgz or .zip)
numRetries: defines the maximum number of retries (default is 5)
intervalBetweenRetries: define the time (in seconds) between retries (default is 60)
timeout: defines the timer (in seconds) for the watchdog timeout applicable to gridftp, scp, ftp, http, and https schemes (default is 600 seconds)
port: define the port to use (when different from the default one). Implemented only for scp driver
recursive: allows the scp driver to download folders
noRetryAfterTimeout: do not retry transfer after timeout
noHtmlAndRdfTags: do not follow html and rdf tags and .uar archives.
skipIfSinkPathExists: skip download if sink path already exists
tmpDir: set up temporary directory for drivers (default to /tmp)
excludePattern: exclude the files matching the pattern for directory input
privateMode: private mode, disable storing of authentication session in ~/.ciop-copy_sess file and passwords in the ~/.ciop-copy_cred file.

Details¶

This function stages-in remote files to local folders. urls and target must be specified.

Output¶

List with:

exit.code: 0 if succesful, >0 if unsuccessful
output: local path of the downloaded product

Examples¶

input_url = "https://store.terradue.com/download/sentinel2/files/v1/S2B_MSIL1C_20190705T100039_N0207_R122_T33TTG_20190705T121033"
output_path = ciop.copy(input_url, '/workspace')

Author¶

Terradue