When an instrument wishes to emit a value it uses a inlet Pipe to transmit the value to a Queue owned by a Circuit. Here the value is forwarded to outlet Pipes, registered by Subscribers, by the Circuit using a single thread. This occurs automatically when using a Pipe provided by a Channel provided by a Conduit. But there are times when it can be beneficial to have code that interacts with instruments to be already within the scope of the Circuit’s execution (thread) context. Here is where the Queue, Script, and Current interfaces come in.

Queues

The Queue interface serves two primary purposes within the Substrates API. Firstly, it facilitates coordination (synchronization) with executing currently enqueued tasks via the await method. Secondly, it enables Scripts to be posted for execution by the Circuit’s underlying pipeline engine and thread execution context.

Scripts

The Script interface is a functional interface with a single method, exec, and a single parameter of type Current. When the Script is executed, it is provided a Current by the Queue that enables it to access contextual information, such as the Subject. It also allows a Script to enqueue additional decomposed work efficiently.

Currents

As previously mentioned, the Current interface provides access to the Subject, which is the Script’s subject nested within a Circuit’s subject. This functionality is useful for tracking the scheduling and execution of scripts and their steps. This aligns with a fundamental principle in the design of the Substrates API: observability should be observable itself. The post method is generally used to decompose the execution of a Script over multiple execution steps to provide greater fairness in overall processing and to simplify some complex event constructs.

Substrates

The Substrates API separates the concerns of task definition, scheduling, and execution, while simultaneously enabling robust patterns for composing asynchronous workflows. The capability to post scripts, coupled with control over execution order facilitates adaptable and maintainable event-driven systems. The design aims to strike a balance between simplicity and sophistication. The architecture achieves power through the composition of simple primitives rather than through complex individual interfaces. Each concept has a clear, singular responsibility that naturally integrates with the others to enable sophisticated use cases. This minimalist yet capable design is characteristic of many of the interfaces and contracts within the Substrates API.

Sample

Here is a simple example littered with some console print statements to record execution order. In the first line of the Script, a runnable step is posted which also posts a runnable within its execution. Then within the outermost scope of the Script, a print statement is made to indicate execution, and finally, another print runnable is posted. Outside of this context, two prints are made to indicate the awaiting and completion of the enqueued work.

Here is the output of the code. It does not match exactly unless we consider every script and step is enqueued.

In the next post, we move back into the data pipeline processing arena where we will focus on windowing.