#What is a “flow”?

The central element of Chioro are “flows”. A flow describes a data flow - from raw source data to prepared product data.

Each flow starts with (at least) one data source and ends with (at least) one data destination. On the way from the data source to the data destination, the product data is transformed, linked, processed, filtered, enriched, … using Chioro’s Flow Operations).

A flow is first created (in the Flows workspace), tested and refined. This can be done step by step. Once it is defined, it is started either manually, time-controlled or via API call. (here an API documentation is created )

#Idempotency

A flow is the conveyor belt that moves product data from one processing station (i.e., one flow operation) to the next. And because these stations are rule-based, a flow can be executed over and over again.

Either on the same data - e.g. after a change to the flow itself. Or on changed data - e.g. on a new version of a product catalog. In any case, the results are always reproducible: the same input data, with an unchanged flow, always leads to the same output data. No matter how often the flow is executed. This property is called idempotency. (By the way, it applies - logically - not only to the whole flow, but to every single operation).

#Data Exploration

A special feature of Chioro is that the product data on its way through the operations (by default) is cached in a searchable database after each operation. This helps enormously in developing flows and debugging. More about this in the section Data Evaluation.

Within one Chioro client several flows can coexist to consolidate e.g. independent product data streams. For example, from different suppliers. Or, in a marketplace, from different marketplace participants.

#Manage flows

Flows are managed in the tabular flow overview. You can get there by …

1. clicking on the Chioro logo in the top left corner of the header, or
2. click twice on the workspace Flows in the header, or
3. a click on “Back to overview” (if available) at the left sidebar

#Overview of all flows

The flow overview shows all flows created in this client. A click on the name of the flow opens a detailed view with all operations of this flow.

1. name of the flow
2. number of existing flow operations
3. number of data sets read in
4. time of the last change at the flow
5. tags of the whole flow
6. triggers present and status of the whole flow (all single operations)
7. actions (execute flow, edit flow, export, remove, …). See below.
8. more information about the flow
9. the list of operations in this flow
10. the name of the operation with the number of records it outputs.
11. the type of the operation.
12. some information about the operation.
13. time of the last execution.
14. time of the last change (of the definition).
15. the status of the last execution (successful, error, …).
16. the actions related to an operation.
17. searchfilter for tags -> display all flows that contains the flags from the searchfilter

Meaning of the sysmbols for actions:

• Execute the entire flow.

• Import a locally stored flow (e.g. to upload a flow created by us, on behalf)

  .

• Export a flow(or single flow operation) to send them to our support for example.

• Customize how many flows are displayed per page.

• Help for each topic.

• Edit the flow. A window opens, description for this below.

• Delete flow or flow operation.

Meaning of the icons for the status of the last execution of an operation:

• The operation was executed without errors.

• The operation was executed with errors.

• The operation has never been executed.

#Create new flow

• Click plus button.
• Assign a unique name for the flow.
• Click on “Save”.

Every flow needs at least one data source. Therefore, after creating the flow, the hint appears that a data source is needed. Clicking on ‘ADD NEW DATA SOURCE’ creates such a source and opens the Flow workspace directly.

#Copy and Paste Flow Operation

It is possible to copy and paste an operation of a flow into the same flow as well as into other flows To enable this, the Copy button must be pressed under “Actions” on a flow operation. A message will appear that the operation has been successfully saved to the clipboard:

This can now be inserted into any flow. Also in the flow where it comes from. It is done by clicking on the “Insert” button that exists in the actions of each flow:

The view is now reloaded and the item is added to the flow. The timestamp is updated and the flow is displayed at the top of the list.

Furthermore there is the possibility to paste these copied operations from the list in the graphical flow designer. The description for this can be found in the chapter directly below this one.

#The graphical flow editor

The graphical flow editor is a central element of the Flows workspace. Here, flows can be visually “stitched together” by dragging flow operations from the left into the canvas and connecting them with arrows. The data then flows along the arrows from the data sources to the data destinations as the flow is executed, using the flow operations in between.

1. Data sources, data destinations and other operations can be added to the flow by drag & drop. The yellow border indicates which operation is currently selected. Its properties and data are displayed in the Operations Editor and Data Evaluation areas below the Flow Editor.
2. The view in the Operations Editor changes depending on the selected flow element.
3. Clicking on a red point (data output) and dragging to a green point (data input) connects two operations. The product data then flows in the direction of the arrow from one operation to the next as it is executed. Some operations have multiple inputs or outputs. For example, to merge data from multiple sources or to send products to different processing branches.
4. There are flow elements with more than one output or input.
5. In the flow editor there are several actions available for the flow or individual operations:

   • Delete operation

     .

   • Clone operation - Operations from other flows can be cloned. This way, identical operations do not have to be created again.

   • Paste Operation - Pastes a previously copied operation from the clipboard into the flow and displays it in the flow designer.

   • Zugriffsrechte konfigurieren.

   • Configure time triggers - Times at which a flow should be executed can be defined, e.g. "Every weekday at 10am".

   • Choice localization.

   • Start flow - The flow will be executed.

   • Start flow from here - The flow will now be executed from the mark.

   • Start selected element

Addendum:

• Copy

• Paste

There are two buttons in the top view called “Copy Flow Operation” and “Paste Flow Operation”. These buttons can be used to paste an operation that is in the clipboard or to copy it to the clipboard. The special feature here is that you can copy and paste flow operations from the Flow Designer, for example, in the Flow View (explained in a previous chapter).

For example, if a user copies a flow operation in the Flow View and then goes to the Flow Designer and presses “Paste”, the flow item will also be pasted. So you are not bound to the views and you can copy and paste back and forth as you like.

#Trigger

It is possible to execute a flow at a specific time. To make this possible, a trigger can be set up to execute the flow cyclically at a fixed time.

To get to the view, click on the “Clock” above the flow editor. This opens this window:

As you can see, there is the possibility to execute the flow every minute, every hour, every day, every week, every month or every year. Furthermore, you can set additional time conditions, depending on which tab you are in.

In addition, the execution of the flow can be made dependent on whether source data has changed since the last execution. For this, one or more data sources can be specified in the trigger configuration. For these data sources, the (change) timestamp of the underlying source file is checked at the trigger times. Only if at least one of these source files has changed since the last execution, the flow will be executed.

#Access Rights

It is possible to configure access rights for a flow. To get to the view, click on the “Lock” icon above the flow editor. After that this window will open:

Further information can be found under Benutzerrechte