Circuits, not Pipelines!

The first official release of the Substrates API is nearing completion. With that, it is time to explore one of the toolkit’s most significant differences compared to other approaches, such as OpenTelemetry.

Instead of a data pipeline that moves measurements from within the process outwards to some remote endpoint, typically in the cloud, the toolkit encourages local accessing, aggregating, actuating, and augmenting of Events sourced by Instruments that feed Conduits attached to a Circuit.

A Circuit in the Substrates API is a network of one or more Conduits. A Circuit has a single current (flow) shared by all attached Conduits, where a current is, in effect, a single-threaded processing Executor.

Each Conduit, linked to a particular Instrument type, manages Inlets (channels) wrapped by an Instrument (producer) such as a Counter or Gauge. It is via an Inlet that measurement data, an Event’s emittance, is relayed to the circuitry and routed to one or more Outlets (consumers) that can themselves be Instruments.

With this approach, a single Event can result in many more Events flowing through the network as Instruments, other than the source of the initial Event, receive notifications via Outlets and emit an Event themselves.

Because all measurement data flows through the same underlying circuitry code, the Fabric (runtime) can augment all Instruments with new capabilities uniformly and efficiently, such as retracing, rewinding, reordering, recording, replicating, replaying, retrying, etc. Here an Instrument becomes merely a decoration of an Inlet, with the interface of the Instrument being the protocol for safe and secure data generation and transmission.