better-wfp-00006 data pipeline results (COPERNICUS Vegetation Indicators Aggregations): loading from pickle file and exploiting¶
This Notebook shows how to: * reload better-wfp-00006 data pipeline results previously stored in a pickle file * Create a new dataframe with data values cropped over the area of interest * Plot an RGB * Create a geotif
Import needed modules¶
In [1]:
import pandas as pd
from geopandas import GeoDataFrame
import gdal
import numpy as np
from shapely.geometry import Point
from shapely.geometry import Polygon
import matplotlib
import matplotlib.pyplot as plt
from PIL import Image
%matplotlib inline
from shapely.wkt import loads
from shapely.geometry import box
from urlparse import urlparse
import requests
Read picke data file¶
In [2]:
pickle_filename = 'better-wfp-00006_validation.pkl'
In [3]:
results = GeoDataFrame(pd.read_pickle(pickle_filename))
In [4]:
results.head(25)
Out[4]:
| enclosure | enddate | identifier | self | startdate | title | wkt | |
|---|---|---|---|---|---|---|---|
| 0 | https://store.terradue.com/better-wfp-00006/_r... | 2017-01-31 | 4C171ECE9E48E2D91F1FD6671774E34BF6F078D9 | https://catalog.terradue.com//better-wfp-00006... | 2017-01-10 | Output LAI_SouthernAfrica_N3_maxvalues_2017-01... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 1 | https://store.terradue.com/better-wfp-00006/_r... | 2017-01-31 | 69E4561CF3E34251D6F2C52E3DEE60873E2BE57E | https://catalog.terradue.com//better-wfp-00006... | 2017-01-10 | Output FAPAR_SouthernAfrica_N3_averages_2017-0... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 2 | https://store.terradue.com/better-wfp-00006/_r... | 2017-01-31 | 7963FF3C50C865EBB01F466263BC92A6645BC36A | https://catalog.terradue.com//better-wfp-00006... | 2017-01-10 | Output LAI_SouthernAfrica_N3_averages_2017-01-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 3 | https://store.terradue.com/better-wfp-00006/_r... | 2017-01-31 | C3AE42343AC02E69EAB0F373716D654E8C22DC96 | https://catalog.terradue.com//better-wfp-00006... | 2017-01-10 | Output FAPAR_SouthernAfrica_N3_maxvalues_2017-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 4 | https://store.terradue.com/better-wfp-00006/_r... | 2016-01-31 | 4644446E4102BA104DC8A694E23491BE786B7B86 | https://catalog.terradue.com//better-wfp-00006... | 2016-01-10 | Output LAI_SouthernAfrica_N3_averages_2016-01-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 5 | https://store.terradue.com/better-wfp-00006/_r... | 2016-01-31 | 5021EC521CBBCAEB02711E2094E7D25623C81B6E | https://catalog.terradue.com//better-wfp-00006... | 2016-01-10 | Output FAPAR_SouthernAfrica_N3_maxvalues_2016-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 6 | https://store.terradue.com/better-wfp-00006/_r... | 2016-01-31 | 8946DCA1AE000235493C94D8FF43C501028043CC | https://catalog.terradue.com//better-wfp-00006... | 2016-01-10 | Output LAI_SouthernAfrica_N3_maxvalues_2016-01... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 7 | https://store.terradue.com/better-wfp-00006/_r... | 2016-01-31 | CD209104828442D962B4EEDCC8A12D22D41390C9 | https://catalog.terradue.com//better-wfp-00006... | 2016-01-10 | Output FAPAR_SouthernAfrica_N3_averages_2016-0... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 8 | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | 01AD7BA22880C3C56DDD246173ED99A19067613D | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output FAPAR_SouthernAfrica_N3_averages_2015-0... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 9 | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | 02D735D2D7DB93C4052FB9955B70523A1C688D25 | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output LAI_SouthernAfrica_N3_maxvalues_2015-01... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 10 | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | 7F29BA329D61BABAB68A5081F44799D1AD78742F | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output FAPAR_SouthernAfrica_N3_maxvalues_2015-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
| 11 | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | 80D61878F122AD704D83A57602A1EB46B13DEDF0 | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output LAI_SouthernAfrica_N3_averages_2015-01-... | POLYGON((11.5030755518998 -11.1141633706909,41... |
Credentials for ellip platform access¶
- Provide here your ellip username and api key for platform access
In [5]:
import getpass
user = getpass.getpass("Enter you ellip username:")
api_key = getpass.getpass("Enter the API key:")
Define filter parameters¶
Time of interest¶
In [27]:
mydates = ['2015-01-10T00:00:00Z', '2015-01-31T23:59:59Z']
Area of interest (two approaches to determine VOI:)¶
- The user can choose to define an AOI selecting a Point and a buffer size used to build a squared polygon around that point
In [ ]:
point_of_interest = Point(27.11,8.149)
In [ ]:
buffer_size = 0.60
In [ ]:
aoi_wkt1 = box(*point_of_interest.buffer(buffer_size).bounds)
In [ ]:
aoi_wkt1.wkt
- Or creating a Polygon from a points list (in this case this is a point inside the intersect of all results’ polygons) or simply using a WKT Polygon:
In [32]:
#aoi_wkt2 = Polygon([(26.832, 9.5136), (28.6843, 9.5136), (28.6843, 7.8009), (26.832, 7.8009), (26.832, 9.5136)])
#aoi_wkt2 = Polygon([(43.721626, 4.577164), (45.956501, 5.041664), (46.261715, 3.533043), (44.031803 ,3.064234), (43.721626, 4.577164)])
#aoi_wkt2 = 'POLYGON((26.832 9.5136, 28.6843 9.5136, 28.6843 7.8009, 26.832 7.8009, 26.832 9.5136))'
aoi_wkt2 = loads('POLYGON ((11.50307555189977 -11.11416337069092, 41.03432555189977 -11.11416337069092, 41.03432555189977 -34.97636566938584, 11.50307555189977 -34.97636566938584, 11.50307555189977 -11.11416337069092))')
In [33]:
aoi_wkt2.wkt
Out[33]:
'POLYGON ((11.50307555189977 -11.11416337069092, 41.03432555189977 -11.11416337069092, 41.03432555189977 -34.97636566938584, 11.50307555189977 -34.97636566938584, 11.50307555189977 -11.11416337069092))'
POLYGON((43.721626 4.577164,45.956501 5.041664,46.261715 3.533043,44.031803 3.064234,43.721626 4.577164))’
Create a new dataframe with data values¶
- Get the subframe data values related to selected tiles and the selected band, and create a new GeoDataFrame having the original metadata stack with the specific band data extended info (data values, size, and geo projection and transformation).
Define auxiliary methods to create new dataframe¶
- The first method gets download reference, band number, cropping flag, AOI cropping area and returns the related band data array with related original geospatial infos
- If crop = False the original band extent is returned
In [34]:
def get_band_as_array(url,band_name,crop,bbox):
output = '/vsimem/clip.tif'
ds = gdal.Open('/vsicurl/%s' % url)
print url
if crop == True:
ulx, uly, lrx, lry = bbox[0], bbox[3], bbox[2], bbox[1]
ds = gdal.Translate(output, ds, projWin = [ulx, uly, lrx, lry], projWinSRS = 'EPSG:4326', noData=-999)
print 'data cropping : DONE'
else:
ds = gdal.Translate(output, ds)
ds = None
ds = gdal.Open(output)
w = ds.GetRasterBand(1).XSize
h = ds.GetRasterBand(1).YSize
geo_transform = ds.GetGeoTransform()
projection = ds.GetProjection()
data = ds.GetRasterBand(1).ReadAsArray(0, 0, w, h).astype(np.float)
data[data==-999] = np.nan
band = None
print 'band %s value got!' %band_name
ds = None
return data,geo_transform,projection,w,h
- The second one selects the data to be managed (only the actual results related to the filtered metadata) and returns a new GeoDataFrame containing the requested bands extended info with the original metadata set
In [35]:
def get_GDF_with_datavalues(row, user, api_key, band_name, crop=False, bbox=None):
bands_name = ['band_1','band_2']
if band_name not in bands_name:
raise ValueError('Selected band is not defined!')
parsed_url = urlparse(row['enclosure'])
url = '%s://%s:%s@%s/api%s' % (list(parsed_url)[0], user, api_key, list(parsed_url)[1], list(parsed_url)[2])
data,geo_transform,projection,w,h = get_band_as_array(url,band_name,crop,bbox)
print(url)
extended_info = dict()
extended_info=dict(row)
extended_info['band'] = band_name
extended_info['data'] = data
extended_info['geo_transform'] = geo_transform
extended_info['projection'] = projection
extended_info['xsize'] = w
extended_info['ysize'] = h
print 'Get data values for %s : DONE!' %row['title']
return extended_info
In [36]:
results.apply(lambda row: 'tif' in row['title'], axis=1)
Out[36]:
0 True
1 True
2 True
3 True
4 True
5 True
6 True
7 True
8 True
9 True
10 True
11 True
dtype: bool
In [37]:
pd.to_datetime(mydates)
Out[37]:
DatetimeIndex(['2015-01-10 00:00:00', '2015-01-31 23:59:59'], dtype='datetime64[ns]', freq=None)
In [38]:
results.apply(lambda row: row['startdate'] in pd.to_datetime(mydates) , axis=1)
Out[38]:
0 False
1 False
2 False
3 False
4 False
5 False
6 False
7 False
8 True
9 True
10 True
11 True
dtype: bool
- Select from metadataframe a subframe containing the geotiff results of the datetimes of interest
In [39]:
mydate_results = results[(results.apply(lambda row: 'tif' in row['title'], axis=1)) & (results.apply(lambda row: row['startdate'] in pd.to_datetime(mydates) , axis=1))]
print(mydate_results)
enclosure enddate \
8 https://store.terradue.com/better-wfp-00006/_r... 2015-01-31
9 https://store.terradue.com/better-wfp-00006/_r... 2015-01-31
10 https://store.terradue.com/better-wfp-00006/_r... 2015-01-31
11 https://store.terradue.com/better-wfp-00006/_r... 2015-01-31
identifier \
8 01AD7BA22880C3C56DDD246173ED99A19067613D
9 02D735D2D7DB93C4052FB9955B70523A1C688D25
10 7F29BA329D61BABAB68A5081F44799D1AD78742F
11 80D61878F122AD704D83A57602A1EB46B13DEDF0
self startdate \
8 https://catalog.terradue.com//better-wfp-00006... 2015-01-10
9 https://catalog.terradue.com//better-wfp-00006... 2015-01-10
10 https://catalog.terradue.com//better-wfp-00006... 2015-01-10
11 https://catalog.terradue.com//better-wfp-00006... 2015-01-10
title \
8 Output FAPAR_SouthernAfrica_N3_averages_2015-0...
9 Output LAI_SouthernAfrica_N3_maxvalues_2015-01...
10 Output FAPAR_SouthernAfrica_N3_maxvalues_2015-...
11 Output LAI_SouthernAfrica_N3_averages_2015-01-...
wkt
8 POLYGON((11.5030755518998 -11.1141633706909,41...
9 POLYGON((11.5030755518998 -11.1141633706909,41...
10 POLYGON((11.5030755518998 -11.1141633706909,41...
11 POLYGON((11.5030755518998 -11.1141633706909,41...
*PARAMETERS*
- bands_list = all|the list of the bands names
For these products we have two bands: band_1 and band_2. band_1 contains the aggregations data, band_2 contains the amount of no_data_values for each region in the image.
- crop = True|False
if crop=False the bbox can be omitted
In [40]:
myGDF = GeoDataFrame()
for row in mydate_results.iterrows():
myGDF = myGDF.append(get_GDF_with_datavalues(row[1], user, api_key, 'band_1', True, bbox=aoi_wkt2.bounds), ignore_index=True)
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/FAPAR_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif
data cropping : DONE
band band_1 value got!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/FAPAR_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif
Get data values for Output FAPAR_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif : DONE!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/LAI_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif
data cropping : DONE
band band_1 value got!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/LAI_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif
Get data values for Output LAI_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif : DONE!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/FAPAR_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif
data cropping : DONE
band band_1 value got!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/FAPAR_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif
Get data values for Output FAPAR_SouthernAfrica_N3_maxvalues_2015-01-10_2015-01-31.tif : DONE!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/LAI_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif
data cropping : DONE
band band_1 value got!
https://better-wfp-00006:AKCp5bBXiiX2aJjWDybUw37zfZwPbjtPZgzUFbf14NegZ7VL2uvQEXqE1xFxbXrzqZgZAaknv@store.terradue.com/api/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/LAI_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif
Get data values for Output LAI_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif : DONE!
In [41]:
myGDF
Out[41]:
| band | data | enclosure | enddate | geo_transform | identifier | projection | self | startdate | title | wkt | xsize | ysize | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | band_1 | [[nan, nan, nan, nan, nan, nan, nan, nan, nan,... | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | (11.5030755519, 0.00892992137889, 0.0, -11.114... | 01AD7BA22880C3C56DDD246173ED99A19067613D | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS... | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output FAPAR_SouthernAfrica_N3_averages_2015-0... | POLYGON((11.5030755518998 -11.1141633706909,41... | 3307.0 | 2673.0 |
| 1 | band_1 | [[25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25... | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | (11.5030755519, 0.00892992137889, 0.0, -11.114... | 02D735D2D7DB93C4052FB9955B70523A1C688D25 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS... | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output LAI_SouthernAfrica_N3_maxvalues_2015-01... | POLYGON((11.5030755518998 -11.1141633706909,41... | 3307.0 | 2673.0 |
| 2 | band_1 | [[25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25... | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | (11.5030755519, 0.00892992137889, 0.0, -11.114... | 7F29BA329D61BABAB68A5081F44799D1AD78742F | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS... | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output FAPAR_SouthernAfrica_N3_maxvalues_2015-... | POLYGON((11.5030755518998 -11.1141633706909,41... | 3307.0 | 2673.0 |
| 3 | band_1 | [[nan, nan, nan, nan, nan, nan, nan, nan, nan,... | https://store.terradue.com/better-wfp-00006/_r... | 2015-01-31 | (11.5030755519, 0.00892992137889, 0.0, -11.114... | 80D61878F122AD704D83A57602A1EB46B13DEDF0 | GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS... | https://catalog.terradue.com//better-wfp-00006... | 2015-01-10 | Output LAI_SouthernAfrica_N3_averages_2015-01-... | POLYGON((11.5030755518998 -11.1141633706909,41... | 3307.0 | 2673.0 |
- Show data values of Beta0_VV_db band
In [42]:
list(myGDF[myGDF['band'] == 'band_1']['data'].values)
Out[42]:
[array([[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
...,
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan]]),
array([[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
...,
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.]]),
array([[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
...,
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.],
[ 25., 25., 25., ..., 25., 25., 25.]]),
array([[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
...,
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan]])]
In [43]:
bands = None
bands = myGDF[(myGDF['band'] == 'band_1')]['data'].values
numbands = bands.size
for i in range(numbands):
bmin = np.nanmin(bands[i])
bmax = np.nanmax(bands[i])
if bmin != bmax:
bands[i][~np.isnan(bands[i])] = (bands[i][~np.isnan(bands[i])] - bmin)/(bmax - bmin) * 255
In [44]:
bands[0]
Out[44]:
array([[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
...,
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan],
[ nan, nan, nan, ..., nan, nan, nan]])
In [45]:
numbands
Out[45]:
4
In [46]:
fig = plt.figure(figsize=(50,50))
for i in range(numbands):
a = fig.add_subplot(1, numbands, i+1)
imgplot = plt.imshow(bands[i],
cmap='gray')
plt.tight_layout()
fig = plt.gcf()
plt.show()
fig.clf()
plt.close()
Exporting geotif¶
In [56]:
band_number = 4
rg_bands=(bands[0],bands[1])
cols = int(myGDF['xsize'].values[0])
rows = int(myGDF['ysize'].values[0])
print (cols,rows)
geo_transform = myGDF['geo_transform'].values[0]
projection = myGDF['projection'].values[0]
drv = gdal.GetDriverByName('GTiff')
ds = drv.Create('export.tif', cols, rows, band_number, gdal.GDT_Float32)
ds.SetGeoTransform(geo_transform)
ds.SetProjection(projection)
ds.GetRasterBand(1).WriteArray(rg_bands[0], 0, 0)
ds.GetRasterBand(2).WriteArray(rg_bands[1], 0, 0)
ds.FlushCache()
(3307, 2673)
Download functionalities¶
Download a product¶
- Define the download function
In [50]:
def get_product(url, dest, api_key):
request_headers = {'X-JFrog-Art-Api': api_key}
r = requests.get(url, headers=request_headers)
open(dest, 'wb').write(r.content)
return r.status_code
- Get the reference download endpoint for the product related to the first date
In [51]:
enclosure = myGDF[(myGDF['startdate'] == mydates[0])]['enclosure'].values[0]
enclosure
Out[51]:
'https://store.terradue.com/better-wfp-00006/_results/workflows/ec_better_wfp_01_03_01_wfp_01_03_01_1_5/run/7757e274-4108-11e9-9a0e-0242ac11000f/0024944-181221095105003-oozie-oozi-W/80a5cdb0-0f7e-4119-b087-3daf43a702e3/FAPAR_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif'
In [52]:
output_name = myGDF[(myGDF['startdate'] == mydates[0])]['title'].values[0]
output_name
Out[52]:
'Output FAPAR_SouthernAfrica_N3_averages_2015-01-10_2015-01-31.tif'
In [53]:
get_product(enclosure,
output_name,
api_key)
Out[53]:
200
Bulk Download¶
- Define the bulk download function
In [54]:
def get_product_bulk(row, api_key):
return get_product(row['enclosure'],
row['title'],
api_key)
- Download all the products related to the chosen dates
In [55]:
myGDF.apply(lambda row: get_product_bulk(row, api_key), axis=1)
Out[55]:
0 200
1 200
2 200
3 200
dtype: int64