Application workflow ==================== We need to picture the application workflow as a Data Pipeline that helps you process and move data between different processing steps. The processing steps are the nodes of a Directed Acyclic Graph (DAG). A directed graph may be used to represent a network of processing elements; in this formulation, data enters a processing element through its incoming nodes and leaves the element through its outgoing nodes. Below an example of a Directed Acyclic Graph depicting a classical workflow where: +-----------------------------------------------------------------+-------------------------------+ | * Node A will process the data coming from the workflow sources | .. include:: uml_workflow.rst | | * Node B and Node C will process the data generated by Node A | | | * Node D will process the data generated by Node B and Node C | | +-----------------------------------------------------------------+-------------------------------+ .. tip:: Take the time to carefully think how to structure the workflow by answering the questions: * How many nodes do I need? * Can the node execution be split in several tasks? * What will each node read as inputs? * What will each node write as outputs? * What parameters does each node need? * Is my workflow cost-effective in terms of I/O? Below a few examples of workflows: * :download:`one single node ` * :download:`one single node with parallel processing and one aggregation node `