Step 7 - Prepare the application for the Production Centre

The Production Centre uses Docker images and deploys them in a Cloud based environment for large scale production campaigns.

This step will trigger the build of a Docker image containing the data transformation application created in the previous steps

We will invoke components of the Ellip Platform Continuous Integration.

The goal of step 7 is to create a Jupyter Notebook that will read the data transformation application pom file and trigger the build of the Docker image.

Note: For this step, you will use the Continuous Integration username/API key

• Create a new Python 2 notebook under ‘/workspace’ and name it ‘build.ipynb’ and add the cells below.
• Provide the application information required:

app = dict([('artifact_id', 'wfp-01-01-01-n'),
            ('repository', 'Gitlab Groups'),
            ('community', 'ec-better')])

Note: Change ‘n’ with the letter assigned to you for the training

• Provide the Continuous Integration username/API key username and API key:

import getpass

username = raw_input("Provide the Continuous Integration user username? ")
api_key = getpass.getpass('And its associated API key:')

• A few Python modules to do the job:

import jenkins
import time
import lxml.etree as etree
import sys
import requests
import os
import string
import hashlib
import urllib2
import pytz
from datetime import datetime
import time
import owslib
from owslib.wps import monitorExecution
import uuid
from owslib.wps import WebProcessingService
import getpass

• Create two simple functions to submit the build request and to monitor it:

def submit_build_job(server, job, sleepSecs=3):

    last_build = server.get_job_info(job)['lastCompletedBuild']['number']

    server.build_job(job)

    while server.get_job_info(job)['lastBuild']['number'] == last_build:

        time.sleep(sleepSecs)

    return server.get_job_info(job)['lastBuild']['number']

def monitor_build_job(server, job, sleepSecs=15):

    last_build_number = server.get_job_info(job)['lastBuild']['number']

    while server.get_build_info(job, last_build_number)['building']:

        time.sleep(sleepSecs)

    if str(server.get_build_info(job, last_build_number)['result']) == 'SUCCESS':

        return True

    if str(server.get_build_info(job, last_build_number)['result']) == 'FAILURE':

        return False

• Establish a connection with the build component of the Ellip Platform Continuous Integration:

server = jenkins.Jenkins(url='https://build.terradue.com',
                        username=username,
                        password=api_key)

• Create the build request path:

job = 'communities/%s/%s/training/%s/docker' % (app['repository'],
                                           app['community'],
                                           app['artifact_id'])

print(job)

communities/Gitlab Groups/ec-better/wfp-01-01-01/docker

Note: In subsequent activities, update the build request path by removing the /training part

• Submit the build job:

submit_build_job(server, job)

24

Note: This step can be time consuming. Have a break, a short walk, or go back to step 5 and inspect the results…

• Monitor the build request:

if monitor_build_job(server, job):
    print 'Job build completed'
else:
    print 'Job build failed'

At this stage, there is a Docker container with the data transformation ready to be deployed on a Production Centre. This is accomplished in Step 8 - Deploy the application on the Production Centre