Hands-On Exercise 4: using a toolbox¶

In this exercise we will use a toolbox to process the inputs of our workflow, and we will publish its results on HDFS.

Prerequisites¶

You have cloned the Hands-On git repository (see Clone the Hands-On repository),
(Only for python) You have installed the required software (see Prerequisites when using python).

Install the Hands-On¶

Install the Hands-On Exercise 4, just type:

cd
cd dcs-hands-on
mvn clean install -D hands.on=4 -P bash

Install the BEAM Toolbox¶

The BEAM Toolbox [1] is an open-source toolbox and development platform for viewing, analyzing and processing remote sensing raster data. BEAM stands for ‘Basic ERS & Envisat (A) ATSR and Meris’, the satellite sensor measurements handled by the toolbox. To install the BEAM toolbox type:

sudo yum install -y beam-5.0
sudo /usr/sbin/alternatives --config java

This will show on the terminal window:

There are 3 programs which provide 'java'.

 Selection     Command
------------------------------------------------
 + 1           /usr/java/jdk1.6.0_35/jre/bin/java
   2           /usr/lib/jvm/jre-1.5.0-gcj/bin/java
*  3           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java

Enter to keep the current selection[+], or type selection number:

Select java 1.7 out of the menu options by typing the correct number (here it’s 3).

Inspect the application.xml¶

Inspect the application.xml, it is now slightly different from the one used in the previous hands-on exercise:

<?xml version="1.0" encoding="us-ascii"?>
<application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" id="beam_arithm">
  <jobTemplates>
    <jobTemplate id="expression">
      <streamingExecutable>/application/expression/run</streamingExecutable>
    </jobTemplate>
  </jobTemplates>
    <workflow id="hands-on-4" title="Using a toolbox" abstract="Exercise 4, using a toolbox">
    <workflowVersion>1.0</workflowVersion>
    <node id="node_expression">
      <job id="expression"/>
      <sources>
        <source refid="file:urls">/application/inputs/list</source>
      </sources>
    </node>
  </workflow>
</application>

Inspect the run executable¶

The BEAM Toolbox has to be called within the run executable. We can inspect the run executable and see how the BEAM Toolbox is used:

cd $_CIOP_APPLICATION_PATH
more expression/run

Hint

The BEAM Toolbox is executed through the gpt.sh script

For further explanations on the graph processing tool, see “Bulk Processing with GPT” [2] and “Creating a GPF Graph” [3].

Run and debug the workflow¶

Run the node node_expression:

ciop-run node_expression

Check the output of the application by copying the Tracking URL from the ciop-run command, and paste it in a browser (see Exercise 2: make a robust workflow and debug it). You will see an output similar to:

Recap¶

We installed the ESA BEAM toolbox;
We called it from our run executable;
We processed the inputs with the BandMaths Operator provided by the ESA BEAM Toolbox;
We published the results in HDFS.

Footnotes

[1]	ESA BEAM Toolbox BandMaths

[2]	Bulk Processing with GPT

[3]	Creating a GPF Graph