better-wfp-00005 data pipeline results (Sentinel-1 Sigma-0 backscatter all polarization): loading from pickle file and exploiting¶
This Notebook shows how to: * reload better-wfp-00005 data pipeline results previously stored in a pickle file * Create a new dataframe with data values cropped over the area of interest * Plot an RGB * Create a geotif
Import needed modules¶
In [1]:
import pandas as pd
from geopandas import GeoDataFrame
import gdal
import numpy as np
from shapely.geometry import Point
from shapely.geometry import Polygon
import matplotlib
import matplotlib.pyplot as plt
from PIL import Image
%matplotlib inline
from shapely.wkt import loads
from shapely.geometry import box
from urlparse import urlparse
import requests
Read picke data file¶
In [2]:
pickle_filename = 'better-wfp-00005_Jun-Sep_2017_Afghanistan.pkl'
In [3]:
results = GeoDataFrame(pd.read_pickle(pickle_filename))
In [4]:
results.head(25)
Out[4]:
| enclosure | enddate | identifier | self | startdate | title | wkt | |
|---|---|---|---|---|---|---|---|
| 0 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-29 13:30:24.210254 | 6a7b74fabadd30ed10879bdcf7cfce24dc336113 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-29 13:29:59.211786 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1159770636222 35.6876804486044,68.... |
| 1 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-29 13:30:24.210254 | 9343fecbd537bc255623cda53c875565a846e9b4 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-29 13:29:59.211786 | Reproducibility notebook used for generating S... | POLYGON((68.1159770636222 35.6876804486044,68.... |
| 2 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-29 13:30:24.210254 | dc32e57be7b0335fb84ab32e55d1e13b13ebcffd | https://catalog.terradue.com//better-wfp-00005... | 2017-09-29 13:29:59.211786 | S1A_IW_GRDH_1SDV_20170929T132959_20170929T1330... | POLYGON((67.6060857657722 35.6923797948525,68.... |
| 3 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-24 13:22:15.073669 | 235f347476174c4bccbb3669a389d9188d61b9ce | https://catalog.terradue.com//better-wfp-00005... | 2017-09-24 13:21:50.075379 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1159748230219 35.6882018821311,68.... |
| 4 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-24 13:22:15.073669 | b9c020654887ad9982d8c9aa8af93c8007020025 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-24 13:21:50.075379 | Reproducibility notebook used for generating S... | POLYGON((68.1159748230219 35.6882018821311,68.... |
| 5 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-24 13:22:15.073669 | c5c9664bb685a93e7bd4f0f3f2ea3c1b0d548fac | https://catalog.terradue.com//better-wfp-00005... | 2017-09-24 13:21:50.075379 | S1A_IW_GRDH_1SDV_20170924T132150_20170924T1322... | POLYGON((67.6571363813005 35.6923359572968,68.... |
| 6 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-17 13:30:23.947846 | 5fe55932997a6287e5070610799b2e3e0d67fcd6 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-17 13:29:58.949468 | Reproducibility notebook used for generating S... | POLYGON((68.1159893268407 35.6876720245373,68.... |
| 7 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-17 13:30:23.947846 | 6fc09bcf07c3e45e692f967fa8b76b7b9d9632b1 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-17 13:29:58.949468 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1159893268407 35.6876720245373,68.... |
| 8 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-17 13:30:23.947846 | 9a8e3659349b0c1db35ba1339837e1002909f91a | https://catalog.terradue.com//better-wfp-00005... | 2017-09-17 13:29:58.949468 | S1A_IW_GRDH_1SDV_20170917T132958_20170917T1330... | POLYGON((67.6060980826322 35.6923713191443,68.... |
| 9 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-12 13:22:14.768049 | 39a7890c6ce46bd4c991729239251498d5c08ffe | https://catalog.terradue.com//better-wfp-00005... | 2017-09-12 13:21:49.769838 | Reproducibility notebook used for generating S... | POLYGON((68.1160036283468 35.6881883630948,68.... |
| 10 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-12 13:22:14.768049 | b4283fad12339b1ba2308a9ca9f145b69304cc70 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-12 13:21:49.769838 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1160036283468 35.6881883630948,68.... |
| 11 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-12 13:22:14.768049 | 2e02d0a14821655ffa7560b65d2eeaba5c19ddf2 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-12 13:21:49.769838 | S1A_IW_GRDH_1SDV_20170912T132149_20170912T1322... | POLYGON((67.6567174188852 35.6923271973424,68.... |
| 12 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:48.910785 | afde54b059ccd550d967f19a9cb264737adc1dcd | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:31:23.911571 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1160161912308 35.6876325086073,68.... |
| 13 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:48.910785 | e4f50e8b1717b36b4992d618960823b0407b2961 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:31:23.911571 | Reproducibility notebook used for generating S... | POLYGON((68.1160161912308 35.6876325086073,68.... |
| 14 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:48.910785 | d96bb8028a13530fe5884668d20f5acf2a5d5fbd | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:31:23.911571 | S1A_IW_GRDH_1SDV_20170906T013123_20170906T0131... | POLYGON((67.6192557660158 35.6922714443056,68.... |
| 15 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:23.910069 | 6ab3078f2df6bb216d03779a46fd08c50e0fdcfe | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:30:58.912351 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1159763441022 35.6876007040857,68.... |
| 16 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:23.910069 | a0889ed5af8f1c5a14368c02aedcbeca6a3b0b48 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:30:58.912351 | Reproducibility notebook used for generating S... | POLYGON((68.1159763441022 35.6876007040857,68.... |
| 17 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:23.910069 | 8ffdabd1faae62a4d9c6ce9017d7fe74540ed286 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:30:58.912351 | S1A_IW_GRDH_1SDV_20170906T013058_20170906T0131... | POLYGON((67.6060849776756 35.6923000390889,68.... |
| 18 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-05 13:30:23.423471 | 855db6da9e60e1b19540e654a067e47f0f86f750 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-05 13:29:58.425182 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1160460513996 35.6876234330004,68.... |
| 19 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-05 13:30:23.423471 | e0f35cdddd6cc6a798c2039db189cfd154fd7ac1 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-05 13:29:58.425182 | Reproducibility notebook used for generating S... | POLYGON((68.1160460513996 35.6876234330004,68.... |
| 20 | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-05 13:30:23.423471 | b8009aaae19d2c4fb1aca1e979d72f6599ed1013 | https://catalog.terradue.com//better-wfp-00005... | 2017-09-05 13:29:58.425182 | S1A_IW_GRDH_1SDV_20170905T132958_20170905T1330... | POLYGON((67.6060425636592 35.6923237461996,68.... |
| 21 | https://store.terradue.com/better-wfp-00005/_r... | 2017-08-31 13:22:14.341220 | a20320c0411e17a32ff2c8623ac68249945fcd60 | https://catalog.terradue.com//better-wfp-00005... | 2017-08-31 13:21:49.343071 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1159622093509 35.6882196615684,68.... |
| 22 | https://store.terradue.com/better-wfp-00005/_r... | 2017-08-31 13:22:14.341220 | d3fd64d17ed5c7deae8967bc3265ee0d89cb753b | https://catalog.terradue.com//better-wfp-00005... | 2017-08-31 13:21:49.343071 | Reproducibility notebook used for generating S... | POLYGON((68.1159622093509 35.6882196615684,68.... |
| 23 | https://store.terradue.com/better-wfp-00005/_r... | 2017-08-31 13:22:14.341220 | 01ef4c35ef59829bf45f7a815bea04c179e49240 | https://catalog.terradue.com//better-wfp-00005... | 2017-08-31 13:21:49.343071 | S1A_IW_GRDH_1SDV_20170831T132149_20170831T1322... | POLYGON((67.6573475857976 35.6923513522827,68.... |
| 24 | https://store.terradue.com/better-wfp-00005/_r... | 2017-08-25 01:31:48.551324 | c3147691e95a6c7d78cf339b8db00c39e684f97e | https://catalog.terradue.com//better-wfp-00005... | 2017-08-25 01:31:23.552092 | Reproducibility stage-in notebook for Sentinel... | POLYGON((68.1160966402204 35.6876279908404,68.... |
Credentials for ellip platform access¶
- Provide here your ellip username and api key for platform access
In [5]:
import getpass
user = getpass.getpass("Enter you ellip username:")
api_key = getpass.getpass("Enter the API key:")
Define filter parameters¶
Time of interest¶
In [6]:
mydates = ['2017-09-29 13:29:59.211786', '2017-09-24 13:21:50.075379','2017-09-17 13:29:58.949468','2017-09-12 13:21:49.769838', '2017-09-06 01:30:58.912351','2017-09-05 13:29:58.425182']
Area of interest¶
- The user can choose to define an AOI selecting a Point and a buffer size used to build a squared polygon around that point
In [7]:
point_of_interest = Point(67.890, 36.182)
In [8]:
buffer_size = 0.07
In [9]:
aoi_wkt1 = box(*point_of_interest.buffer(buffer_size).bounds)
In [10]:
aoi_wkt1.wkt
Out[10]:
'POLYGON ((67.95999999999999 36.112, 67.95999999999999 36.252, 67.82000000000001 36.252, 67.82000000000001 36.112, 67.95999999999999 36.112))'
- Or creating a Polygon from a points list (in this case this is exactly the Afghanistan reference AOI)
In [11]:
aoi_wkt2 = Polygon([(67.62430, 36.7228), (68.116, 36.7228), (68.116, 35.6923), (67.62430, 35.6923), (67.62430, 36.7228)])
In [12]:
aoi_wkt2.wkt
Out[12]:
'POLYGON ((67.62430000000001 36.7228, 68.116 36.7228, 68.116 35.6923, 67.62430000000001 35.6923, 67.62430000000001 36.7228))'
Create a new dataframe with data values¶
- Get the subframe data values related to selected tiles and bands, and create a new GeoDataFrame having the original metadata stack with the specific bands data extended info (data values, size, and geo projection and transformation).
Define auxiliary methods to create new dataframe¶
- The first method gets download reference, bands numbers, cropping flag, AOI cropping area and returns the related bands data array with related original geospatial infos
- If crop = False the original bands extent is returned
In [13]:
def get_bands_as_array(url,band_indices,bands_name,crop,bbox):
output = '/vsimem/clip.tif'
bands=[]
ds = gdal.Open('/vsicurl/%s' % url)
bands_count = ds.RasterCount
if bands_count < len(band_indices):
raise('Selected bands are more than the data ones!')
if crop == True:
ulx, uly, lrx, lry = bbox[0], bbox[3], bbox[2], bbox[1]
ds = gdal.Translate(output, ds, bandList=band_indices, projWin = [ulx, uly, lrx, lry], projWinSRS = 'EPSG:4326')
print 'data cropping : DONE'
else:
ds = gdal.Translate(output, ds, bandList=band_indices)
ds = None
ds = gdal.Open(output)
w = ds.GetRasterBand(1).XSize
h = ds.GetRasterBand(1).YSize
geo_transform = ds.GetGeoTransform()
projection = ds.GetProjection()
for i in range(len(band_indices)):
data = ds.GetRasterBand(i+1).ReadAsArray(0, 0, w, h).astype(np.float)
data[data==-9999] = np.nan
bands.append(data)
band = None
data = None
print 'bands %s values got!' %band_indices
ds = None
return bands,geo_transform,projection,w,h
- The second one selects the data to be managed (only the actual results related to the filtered metadata) and returns a new GeoDataFrame containing the requested bands extended info with the original metadata set
In [14]:
def get_GDF_with_datavalues(row, user, api_key, bands_list='all', crop=False, bbox=None):
bands_name = ['Sigma0_VH_db','Sigma0_VV_db']
band_indices = range(1,len(bands_name)+1)
if bands_list is not 'all':
band_indices = [bands_name.index(x)+1 for x in bands_list ]
parsed_url = urlparse(row['enclosure'])
url = '%s://%s:%s@%s/api%s' % (list(parsed_url)[0], user, api_key, list(parsed_url)[1], list(parsed_url)[2])
bands,geo_transform,projection,w,h = get_bands_as_array(url,band_indices,bands_name,crop,bbox)
extended_info = list()
for i,bi in enumerate(band_indices):
extended_info.append(dict(row))
extended_info[i]['band'] = bands_name[bi-1]
extended_info[i]['data'] = bands[i]
extended_info[i]['geo_transform'] = geo_transform
extended_info[i]['projection'] = projection
extended_info[i]['xsize'] = w
extended_info[i]['ysize'] = h
print 'Get data values for %s : DONE!' %row['title']
return extended_info
- Select from metadataframe a subframe containing the geotiff results of the datetimes of interest
In [15]:
mydate_results = results[(results.apply(lambda row: 'tif' in row['title'], axis=1)) & (results.apply(lambda row: row['startdate'] in pd.to_datetime(mydates) , axis=1))]
*PARAMETERS*
- bands_list = all|the list of the bands names
lit of bands names can be: [‘Sigma0_VH_db’,’Sigma0_VV_db’] or [‘Sigma0_VH_db’] or [‘Sigma0_VV_db’]
- crop = True|False
if crop=False the bbox can be omitted
In [16]:
aoi_wkt = aoi_wkt1
#aoi_wkt = aoi_wkt2
myGDF = GeoDataFrame()
for row in mydate_results.iterrows():
myGDF = myGDF.append(get_GDF_with_datavalues(row[1], user, api_key, 'all', True, bbox=aoi_wkt.bounds), ignore_index=True)
#myGDF = myGDF.append(get_GDF_with_datavalues(row[1], user, api_key, ['Sigma0_VV_db'], True, bbox=aoi_wkt.bounds), ignore_index=True)
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170929T132959_20170929T133024_018591_01F571_F9F4_Sigma0_all_bands.tif : DONE!
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170924T132150_20170924T132215_018518_01F343_D4DD_Sigma0_all_bands.tif : DONE!
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170917T132958_20170917T133023_018416_01F01B_34CC_Sigma0_all_bands.tif : DONE!
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170912T132149_20170912T132214_018343_01EDE8_8071_Sigma0_all_bands.tif : DONE!
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170906T013058_20170906T013123_018248_01EADD_CC2A_Sigma0_all_bands.tif : DONE!
data cropping : DONE
bands [1, 2] values got!
Get data values for S1A_IW_GRDH_1SDV_20170905T132958_20170905T133023_018241_01EAA7_58B6_Sigma0_all_bands.tif : DONE!
In [17]:
myGDF
Out[17]:
| band | data | enclosure | enddate | geo_transform | identifier | projection | self | startdate | title | wkt | xsize | ysize | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Sigma0_VH_db | [[-21.3440952301, -19.659078598, -18.298664093... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-29 13:30:24.210254 | (393982.438116, 10.0, 0.0, 4012550.41088, 0.0,... | dc32e57be7b0335fb84ab32e55d1e13b13ebcffd | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-29 13:29:59.211786 | S1A_IW_GRDH_1SDV_20170929T132959_20170929T1330... | POLYGON((67.6060857657722 35.6923797948525,68.... | 1241 | 1567 |
| 1 | Sigma0_VV_db | [[-12.2571287155, -10.9044475555, -9.897649765... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-29 13:30:24.210254 | (393982.438116, 10.0, 0.0, 4012550.41088, 0.0,... | dc32e57be7b0335fb84ab32e55d1e13b13ebcffd | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-29 13:29:59.211786 | S1A_IW_GRDH_1SDV_20170929T132959_20170929T1330... | POLYGON((67.6060857657722 35.6923797948525,68.... | 1241 | 1567 |
| 2 | Sigma0_VH_db | [[-1.00000000317e-30, -1.00000000317e-30, -1.0... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-24 13:22:15.073669 | (393981.824105, 10.0, 0.0, 4012545.5538, 0.0, ... | c5c9664bb685a93e7bd4f0f3f2ea3c1b0d548fac | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-24 13:21:50.075379 | S1A_IW_GRDH_1SDV_20170924T132150_20170924T1322... | POLYGON((67.6571363813005 35.6923359572968,68.... | 1241 | 1567 |
| 3 | Sigma0_VV_db | [[-1.00000000317e-30, -1.00000000317e-30, -1.0... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-24 13:22:15.073669 | (393981.824105, 10.0, 0.0, 4012545.5538, 0.0, ... | c5c9664bb685a93e7bd4f0f3f2ea3c1b0d548fac | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-24 13:21:50.075379 | S1A_IW_GRDH_1SDV_20170924T132150_20170924T1322... | POLYGON((67.6571363813005 35.6923359572968,68.... | 1241 | 1567 |
| 4 | Sigma0_VH_db | [[-20.6672821045, -20.3851547241, -19.68181419... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-17 13:30:23.947846 | (393983.524496, 10.0, 0.0, 4012549.46109, 0.0,... | 9a8e3659349b0c1db35ba1339837e1002909f91a | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-17 13:29:58.949468 | S1A_IW_GRDH_1SDV_20170917T132958_20170917T1330... | POLYGON((67.6060980826322 35.6923713191443,68.... | 1241 | 1567 |
| 5 | Sigma0_VV_db | [[-11.4198741913, -10.0633506775, -9.540658950... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-17 13:30:23.947846 | (393983.524496, 10.0, 0.0, 4012549.46109, 0.0,... | 9a8e3659349b0c1db35ba1339837e1002909f91a | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-17 13:29:58.949468 | S1A_IW_GRDH_1SDV_20170917T132958_20170917T1330... | POLYGON((67.6060980826322 35.6923713191443,68.... | 1241 | 1567 |
| 6 | Sigma0_VH_db | [[-1.00000000317e-30, -1.00000000317e-30, -1.0... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-12 13:22:14.768049 | (393984.38735, 10.0, 0.0, 4012554.55935, 0.0, ... | 2e02d0a14821655ffa7560b65d2eeaba5c19ddf2 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-12 13:21:49.769838 | S1A_IW_GRDH_1SDV_20170912T132149_20170912T1322... | POLYGON((67.6567174188852 35.6923271973424,68.... | 1241 | 1567 |
| 7 | Sigma0_VV_db | [[-1.00000000317e-30, -1.00000000317e-30, -1.0... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-12 13:22:14.768049 | (393984.38735, 10.0, 0.0, 4012554.55935, 0.0, ... | 2e02d0a14821655ffa7560b65d2eeaba5c19ddf2 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-12 13:21:49.769838 | S1A_IW_GRDH_1SDV_20170912T132149_20170912T1322... | POLYGON((67.6567174188852 35.6923271973424,68.... | 1241 | 1567 |
| 8 | Sigma0_VH_db | [[-19.3490562439, -20.9711437225, -22.19957542... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:23.910069 | (393982.384507, 10.0, 0.0, 4012551.5647, 0.0, ... | 8ffdabd1faae62a4d9c6ce9017d7fe74540ed286 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:30:58.912351 | S1A_IW_GRDH_1SDV_20170906T013058_20170906T0131... | POLYGON((67.6060849776756 35.6923000390889,68.... | 1241 | 1567 |
| 9 | Sigma0_VV_db | [[-8.6659784317, -11.2367334366, -12.851882934... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-06 01:31:23.910069 | (393982.384507, 10.0, 0.0, 4012551.5647, 0.0, ... | 8ffdabd1faae62a4d9c6ce9017d7fe74540ed286 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-06 01:30:58.912351 | S1A_IW_GRDH_1SDV_20170906T013058_20170906T0131... | POLYGON((67.6060849776756 35.6923000390889,68.... | 1241 | 1567 |
| 10 | Sigma0_VH_db | [[-20.8266983032, -19.8164215088, -18.85611152... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-05 13:30:23.423471 | (393978.633342, 10.0, 0.0, 4012554.13882, 0.0,... | b8009aaae19d2c4fb1aca1e979d72f6599ed1013 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-05 13:29:58.425182 | S1A_IW_GRDH_1SDV_20170905T132958_20170905T1330... | POLYGON((67.6060425636592 35.6923237461996,68.... | 1241 | 1567 |
| 11 | Sigma0_VV_db | [[-12.3493747711, -10.4409170151, -9.673035621... | https://store.terradue.com/better-wfp-00005/_r... | 2017-09-05 13:30:23.423471 | (393978.633342, 10.0, 0.0, 4012554.13882, 0.0,... | b8009aaae19d2c4fb1aca1e979d72f6599ed1013 | PROJCS["WGS 84 / UTM zone 42N",GEOGCS["WGS 84"... | https://catalog.terradue.com//better-wfp-00005... | 2017-09-05 13:29:58.425182 | S1A_IW_GRDH_1SDV_20170905T132958_20170905T1330... | POLYGON((67.6060425636592 35.6923237461996,68.... | 1241 | 1567 |
- Show data values of Sigma0_VH_db bands
In [18]:
list(myGDF[myGDF['band'] == 'Sigma0_VH_db']['data'].values)
Out[18]:
[array([[-21.34409523, -19.6590786 , -18.29866409, ..., -24.05504608,
-23.63244629, -22.70774841],
[-21.11527824, -19.38327217, -18.09986115, ..., -23.54561615,
-22.9217205 , -21.78229523],
[-20.51828384, -19.07429504, -17.95887566, ..., -21.87236595,
-21.05905914, -20.31284332],
...,
[-14.5912838 , -14.44718552, -14.2822113 , ..., -18.10773849,
-18.39187431, -18.24716187],
[-13.46404266, -13.10937595, -12.90262604, ..., -18.39934921,
-18.67106056, -18.4901619 ],
[-12.51110554, -12.23794079, -12.17965984, ..., -18.35859871,
-18.41948318, -18.14713478]]),
array([[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-2.48732357e+01, -2.35832596e+01, -2.15002499e+01],
[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-2.29214897e+01, -2.12535706e+01, -1.96754417e+01],
[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-1.98998318e+01, -1.85715809e+01, -1.76683445e+01],
...,
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.59114332e+01, -1.57629490e+01, -1.55539570e+01],
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.64679108e+01, -1.61270657e+01, -1.59523363e+01],
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.66812496e+01, -1.65325356e+01, -1.66489201e+01]]),
array([[-20.6672821 , -20.38515472, -19.68181419, ..., -24.91559982,
-23.91038704, -22.50609779],
[-20.41353416, -20.16723824, -19.55644989, ..., -23.82912445,
-22.9567852 , -21.59866333],
[-20.32748032, -20.00886726, -19.39842987, ..., -21.62311935,
-20.94953346, -20.16966629],
...,
[-14.11012173, -14.08413792, -14.02135372, ..., -18.09975815,
-18.31803513, -18.48630714],
[-13.07145596, -12.96813297, -12.85067654, ..., -18.04652023,
-18.21645164, -18.46287918],
[-12.08418941, -12.29280663, -12.25531197, ..., -17.58804131,
-17.65422249, -17.8218956 ]]),
array([[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-2.46242752e+01, -2.50860939e+01, -2.44579945e+01],
[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-2.50820808e+01, -2.39774704e+01, -2.23076019e+01],
[ -1.00000000e-30, -1.00000000e-30, -1.00000000e-30, ...,
-2.21160851e+01, -2.04810333e+01, -1.93067799e+01],
...,
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.58349953e+01, -1.59019184e+01, -1.59978867e+01],
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.66590843e+01, -1.66118240e+01, -1.65458069e+01],
[ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,
-1.71753292e+01, -1.69774151e+01, -1.67251205e+01]]),
array([[-19.34905624, -20.97114372, -22.19957542, ..., -20.27356148,
-18.03074265, -16.55474281],
[-19.51149178, -20.79325676, -21.75040436, ..., -18.73208809,
-17.36035919, -16.50928307],
[-19.53284073, -20.52659798, -21.28862953, ..., -17.08140373,
-16.65645218, -16.41674423],
...,
[ -6.43351555, -6.71514034, -7.16550112, ..., -20.20985603,
-20.34573555, -20.56829071],
[ -6.57415628, -6.86812687, -7.40815258, ..., -19.48980522,
-19.73376846, -19.92584991],
[ -6.79258823, -7.20976114, -7.82852077, ..., -18.81736565,
-18.99793434, -19.18014717]]),
array([[-20.8266983 , -19.81642151, -18.85611153, ..., -24.15151596,
-24.3005867 , -23.51057816],
[-20.03212357, -19.08052063, -18.13627434, ..., -24.66919518,
-23.59651756, -22.16068459],
[-18.97042465, -18.11985016, -17.35380936, ..., -22.56612968,
-21.41616821, -20.4427681 ],
...,
[-14.57013035, -14.20810032, -14.24384785, ..., -18.84852028,
-18.83483696, -18.71876335],
[-13.6784029 , -13.0143795 , -12.96100426, ..., -19.49032021,
-19.22070885, -18.86196518],
[-12.7422924 , -12.14266682, -12.17653847, ..., -19.09468269,
-18.71100807, -18.37206459]])]
Plotting data and exporting geotif¶
Plot data¶
Choose bands to be plotted
In [19]:
bands = None
bands = myGDF[(myGDF['band'] == 'Sigma0_VH_db')]['data'].values
numbands = bands.size
for i in range(numbands):
bmin = bands[i].min()
bmax = bands[i].max()
if bmin != bmax:
bands[i] = (bands[i] - bmin)/(bmax - bmin) * 255
- Plot the bands separately
In [20]:
fig = plt.figure(figsize=(20,20))
for i in range(numbands):
a = fig.add_subplot(1, numbands, i+1)
imgplot = plt.imshow(bands[i].astype(np.uint8),
cmap='gray')
plt.tight_layout()
fig = plt.gcf()
plt.show()
fig.clf()
plt.close()
- Select 3 bands and plot them as an RGB
In [21]:
rgbbands=(bands[0],bands[2],bands[4])
rgb_uint8 = np.dstack(rgbbands).astype(np.uint8)
width = 10
height = 10
plt.figure(figsize=(width, height))
img = Image.fromarray(rgb_uint8)
imgplot = plt.imshow(img)
Export the Geotiff¶
In [22]:
band_number = 3
cols = myGDF['xsize'].values[0]
rows = myGDF['ysize'].values[0]
print (cols,rows)
geo_transform = myGDF['geo_transform'].values[0]
projection = myGDF['projection'].values[0]
drv = gdal.GetDriverByName('GTiff')
ds = drv.Create('export.tif', cols, rows, band_number, gdal.GDT_Float32)
ds.SetGeoTransform(geo_transform)
ds.SetProjection(projection)
ds.GetRasterBand(1).WriteArray(rgbbands[0], 0, 0)
ds.GetRasterBand(2).WriteArray(rgbbands[1], 0, 0)
ds.GetRasterBand(3).WriteArray(rgbbands[2], 0, 0)
ds.FlushCache()
(1241, 1567)
Download functionalities¶
Download a product¶
- Define the download function
In [23]:
def get_product(url, dest, api_key):
request_headers = {'X-JFrog-Art-Api': api_key}
r = requests.get(url, headers=request_headers)
open(dest, 'wb').write(r.content)
return r.status_code
- Get the reference download endpoint for the product related to the first date
In [24]:
enclosure = myGDF[(myGDF['startdate'] == mydates[0])]['enclosure'].values[0]
enclosure
Out[24]:
'https://store.terradue.com/better-wfp-00005/_results/workflows/ec_better_ewf_wfp_01_02_03_ewf_wfp_01_02_03_0_5/run/0fe626d4-19bf-11e9-adb5-0242ac11000f/0001333-181221095105003-oozie-oozi-W/ff52d47e-a750-4368-a476-55a197fa1980/S1A_IW_GRDH_1SDV_20170929T132959_20170929T133024_018591_01F571_F9F4_Sigma0_all_bands.tif'
In [25]:
output_name = myGDF[(myGDF['startdate'] == mydates[0])]['title'].values[0]
output_name
Out[25]:
'S1A_IW_GRDH_1SDV_20170929T132959_20170929T133024_018591_01F571_F9F4_Sigma0_all_bands.tif'
In [26]:
get_product(enclosure,
output_name,
api_key)
Out[26]:
200
Bulk Download¶
- Define the bulk download function
In [27]:
def get_product_bulk(row, api_key):
return get_product(row['enclosure'],
row['title'],
api_key)
- Download all the products related to the chosen dates
In [28]:
myGDF.apply(lambda row: get_product_bulk(row, api_key), axis=1)
Out[28]:
0 200
1 200
2 200
3 200
4 200
5 200
6 200
7 200
8 200
9 200
10 200
11 200
dtype: int64