Processing nodes and parallelism

As introduced in the workflow section, the processing nodes are the edges of a Directed Acyclic Graph and they can be run in parallel.

This section explains what can be done within a processing node.

The processing node can be executed as several processing tasks or a single processing task.

While the single processing task doesn’t change the understanding of the workflow where a node will process a single task, an example of a node breaking down into several processing tasks is shown below.

Several processing tasks

In the previous workflow example:

Let’s say Node A and Node C break-down into several processing task.

Node A is executed as:

The complete workflow is executed as:

Tip

There two levels of parallelism:

• The first is at workflow level where two nodes can run concurrently (fork and join)
• The second is at node level where the execution can be split in several tasks, each processing a subset of the inputs