Processing nodes and parallelism

As introduced in the workflow section, the processing nodes are the edges of a Directed Acyclic Graph and they can be run in parallel.

This section explains what can be done within a processing node.

The processing node can be executed as several processing tasks or a single processing task.

While the single processing task doesn’t change the understanding of the workflow where a node will process a single task, an example of a node breaking down into several processing tasks is shown below.

Several processing tasks

In the previous workflow example:

 !define DIAG_NAME Workflow example

 !include includes/skins.iuml

 skinparam backgroundColor #FFFFFF
 skinparam componentStyle uml2

start

:Node A;

fork
  :Node B;
fork again
  :Node C;
end fork
  :Node D;

stop

Let’s say Node A and Node C break-down into several processing task.

Node A is executed as:

!define DIAG_NAME Node A breakdown

!include includes/skins.iuml

skinparam backgroundColor #FFFFFF
skinparam componentStyle uml2

fork
  :Task A.1;
fork again
  :Task A.2;
fork again
  :...;
fork again
  :Task A.n;
end fork

The complete workflow is executed as:

!define DIAG_NAME Workflow example

!include includes/skins.iuml

skinparam backgroundColor #FFFFFF
skinparam componentStyle uml2

start

fork
  :Task A.1;
fork again
  :Task A.2;
fork again
  :...;
fork again
  :Task A.n;
end fork

fork
  :Task B;
fork again
  fork
    :Task C.1;
  fork again
    :Task C.2;
  fork again
    :...;
  fork again
    :Task C.n;
  end fork
end fork
  :Task D;

stop

Tip

There two levels of parallelism:
  • The first is at workflow level where two nodes can run concurrently (fork and join)
  • The second is at node level where the execution can be split in several tasks, each processing a subset of the inputs