rciop.copy {rciop}
==================

Description

-----------

Copies an URL to a given folder

Usage
-----

.. code-block:: bash

  rciop.copy (url, target="./", uncompress=TRUE, credentials=NULL, quietmode=FALSE, abortOnFirstError=FALSE, forceTransfer=FALSE, urllist=NULL, createOutputDirectory=FALSE, compressOutput=FALSE, nouzip=FALSE, numRetries=NULL, intervalBetweenRetries=NULL, timeout=NULL, port=NULL, recursive=FALSE, noRetryAfterTimeout=FALSE, debugMode=FALSE, noHtmlAndRdfTags=FALSE,skipIfSinkPathExists=FALSE, tmpDir=NULL, excludePattern=NULL, privateMode=FALSE)

Arguments
---------

``url``
  the URL of the product to download or the URL to the OpenSearch catalogue entry in RDF format

``target``
  the destination folder for the downloaed product

``uncompress``
  if the product is an archive (.tgz, .gz) it is extracted by default

``credentials``
  force <user> and <password> authentication. Credentials string is of the form: *<user>:<password>*

  NOTE: these passwords will be stored in clear text in the ~/.ciop-copy_cred file.
  NOTE: If username/passowrd is specified it has precedence over the certificate/proxy authentication

``quietmode``
  quiet mode, local filenames are not echoed to stdout after transfer

``abortOnFirstError``
  abort on first error without attempting to process further URLs

``forceTransfer``
  force transfer of a physical copy of the file in case of nfs URLs

``urllist``
  get URLs from the <url-list> file

``createOutputDirectory``
  creates the output directory if it does not exist

``compressOutput``
  provide output as a compressed package (.gz for files or .tgz for folders).
  
  NOTE: that it will not compress already compressed files (.gz, .tgz or .zip)

``nouzip``
  disable file automatic decompression of .gz, .tgz and .zip files.

``numRetries``
  defines the maximum number of retries (default is 5)

``intervalBetweenRetries``
  define the time (in seconds) between retries (default is 60)

``timeout``
  defines the timer (in seconds) for the watchdog timeout applicable to gridftp, scp, ftp, http, and https schemes (default is 600 seconds)

``port``
  define the port to use (when different from the default one). Implemented only for scp driver

``recursive``
  allows the scp driver to download folders

``noRetryAfterTimeout``
  do not retry transfer after timeout

``debugMode``
  set debug mode for command output parsing debugging

``noHtmlAndRdfTags``
  do not follow html and rdf tags and .uar archives.

``skipIfSinkPathExists``
  skip download if sink path already exists

``tmpDir``
  set up temporary directory for drivers (default to /tmp)

``excludePattern``
  exclude the files matching the pattern for directory input

``privateMode``
  private mode, disable storing of authentication session in ~/.ciop-copy_sess file and passwords in the ~/.ciop-copy_cred file.

Details
-------

This function stages-in remote files to local folders 

Value
-----

List with:

* exit.code: 0 if succesful, >0 if unsuccessful 
* output: local path of the downloaded product

Examples
--------

.. code-block:: bash
  
  input <- "https://store.terradue.com/download/sentinel2/files/v1/S2B_MSIL1C_20190705T100039_N0207_R122_T33TTG_20190705T121033" 
  res <- rciop.copy(input, TMPDIR, uncompress=TRUE)
  if (res$exit.code==0) local.url <- res$output
  
.. code-block:: bash

  local.prms <- apply(as.data.frame(prm), 1, function(url) { rciop.copy(url, TMPDIR)$output })
  tmp.df <- read.table(textConnection(local.prms)) 

Author
------

Terradue