rciop.copy {rciop}

Description

---

Copies an URL to a given folder

Usage

rciop.copy (url, target="./", uncompress=TRUE, credentials=NULL, quietmode=FALSE, abortOnFirstError=FALSE, forceTransfer=FALSE, urllist=NULL, createOutputDirectory=FALSE, compressOutput=FALSE, nouzip=FALSE, numRetries=NULL, intervalBetweenRetries=NULL, timeout=NULL, port=NULL, recursive=FALSE, noRetryAfterTimeout=FALSE, debugMode=FALSE, noHtmlAndRdfTags=FALSE,skipIfSinkPathExists=FALSE, tmpDir=NULL, excludePattern=NULL, privateMode=FALSE)

Arguments

url

the URL of the product to download or the URL to the OpenSearch catalogue entry in RDF format

target

the destination folder for the downloaed product

uncompress

if the product is an archive (.tgz, .gz) it is extracted by default

credentials

force <user> and <password> authentication. Credentials string is of the form: <user>:<password>

NOTE: these passwords will be stored in clear text in the ~/.ciop-copy_cred file. NOTE: If username/passowrd is specified it has precedence over the certificate/proxy authentication

quietmode

quiet mode, local filenames are not echoed to stdout after transfer

abortOnFirstError

abort on first error without attempting to process further URLs

forceTransfer

force transfer of a physical copy of the file in case of nfs URLs

urllist

get URLs from the <url-list> file

createOutputDirectory

creates the output directory if it does not exist

compressOutput

provide output as a compressed package (.gz for files or .tgz for folders).

NOTE: that it will not compress already compressed files (.gz, .tgz or .zip)

nouzip

disable file automatic decompression of .gz, .tgz and .zip files.

numRetries

defines the maximum number of retries (default is 5)

intervalBetweenRetries

define the time (in seconds) between retries (default is 60)

timeout

defines the timer (in seconds) for the watchdog timeout applicable to gridftp, scp, ftp, http, and https schemes (default is 600 seconds)

port

define the port to use (when different from the default one). Implemented only for scp driver

recursive

allows the scp driver to download folders

noRetryAfterTimeout

do not retry transfer after timeout

debugMode

set debug mode for command output parsing debugging

noHtmlAndRdfTags

do not follow html and rdf tags and .uar archives.

skipIfSinkPathExists

skip download if sink path already exists

tmpDir

set up temporary directory for drivers (default to /tmp)

excludePattern

exclude the files matching the pattern for directory input

privateMode

private mode, disable storing of authentication session in ~/.ciop-copy_sess file and passwords in the ~/.ciop-copy_cred file.

Details

This function stages-in remote files to local folders

Value

List with:

• exit.code: 0 if succesful, >0 if unsuccessful
• output: local path of the downloaded product

Examples

input <- "https://store.terradue.com/download/sentinel2/files/v1/S2B_MSIL1C_20190705T100039_N0207_R122_T33TTG_20190705T121033"
res <- rciop.copy(input, TMPDIR, uncompress=TRUE)
if (res$exit.code==0) local.url <- res$output

local.prms <- apply(as.data.frame(prm), 1, function(url) { rciop.copy(url, TMPDIR)$output })
tmp.df <- read.table(textConnection(local.prms))

Author

Terradue