ciop.copy
=========

Description
-----------

Copies file from a URL to a local folder.

Usage: public catalogue
-----

.. code-block:: python

  import cioppy
  ciop = cioppy.Cioppy()

  ciop.copy(urls, target)

Usage: private catalogue
-----

When using JupyterLab, if the ``HOME`` environment variable is not declared run this code:

.. code-block:: python

  import os
  import getpass

  vm_user = getpass.getuser()
  os.environ['HOME'] = '/home/{}'.format(vm_user)

Insert the credentials string using the form *<user>:<password>* 

.. code-block:: python

  import cioppy
  ciop = cioppy.Cioppy()

  ciop.copy(urls, target, credentials='<user>:<password>')

Arguments
---------

.. code-block:: python

  import cioppy
  ciop = cioppy.Cioppy()

  ciop.copy(urls, target, debugMode=False, credentials=None, extract=True, abortOnFirstError=False, quietmode=False,
             forceTransfer=True, createOutputDirectory=False, compressOutput=False, numRetries=None, intervalBetweenRetries=None,
             timeout=None, port=None, recursive=False, noRetryAfterTimeout=False, noHtmlAndRdfTags=False, skipIfSinkPathExists=False,
             tmpDir=None, excludePattern=None, privateMode=False)

``urls``
  the URL of the product to download, or the URL to the OpenSearch catalogue entry in RDF format

``target``
  the destination folder for the downloaded product

``debugMode``
  set debug mode for command output parsing debugging

``credentials``
  force <user> and <password> authentication. Credentials string is of the form: *<user>:<password>*
  NOTE: these passwords will be stored in clear text in the ~/.ciop-copy_cred file.
  NOTE: If username/passowrd is specified it has precedence over the certificate/proxy authentication

``extract``
  if the product is an archive (.tgz, .gz) it is extracted by default

``abortOnFirstError``
  abort on first error without attempting to process further URLs

``quietmode``
  quiet mode, local filenames are not echoed to stdout after transfer

``forceTransfer``
  force transfer of a physical copy of the file in case of nfs URLs

``createOutputDirectory``
  creates the output directory if it does not exist

``compressOutput``
  provide output as a compressed package (.gz for files or .tgz for folders).
  NOTE: that it will not compress already compressed files (.gz, .tgz or .zip)

``numRetries``
  defines the maximum number of retries (default is 5)

``intervalBetweenRetries``
  define the time (in seconds) between retries (default is 60)

``timeout``
  defines the timer (in seconds) for the watchdog timeout applicable to gridftp, scp, ftp, http, and https schemes (default is 600 seconds)

``port``
  define the port to use (when different from the default one). Implemented only for scp driver

``recursive``
  allows the scp driver to download folders

``noRetryAfterTimeout``
  do not retry transfer after timeout

``noHtmlAndRdfTags``
  do not follow html and rdf tags and .uar archives.

``skipIfSinkPathExists``
  skip download if sink path already exists

``tmpDir``
  set up temporary directory for drivers (default to /tmp)

``excludePattern``
  exclude the files matching the pattern for directory input

``privateMode``
  private mode, disable storing of authentication session in ~/.ciop-copy_sess file and passwords in the ~/.ciop-copy_cred file.

Details
-------

This function stages-in remote files to local folders. ``urls`` and ``target`` must be specified.

Output
-----

List with:

* exit.code: 0 if succesful, >0 if unsuccessful
* output: local path of the downloaded product

Examples
--------

.. code-block:: python

  input_url = "https://store.terradue.com/download/sentinel2/files/v1/S2B_MSIL1C_20190705T100039_N0207_R122_T33TTG_20190705T121033"
  output_path = ciop.copy(input_url, '/workspace')

Author
------

Terradue