Pipelines and Projections


An objective of the Substrates API is that it should be possible for developers, whether building Instruments or analytical Plugins via the Subscriber and Outlet interfaces, to be location independent in that code can execute without change within the local process itself or a remote server that is being served the data via a Relay. That is not the say the code is not location-aware, but that there should not be an entirely different interface programming model when executing on some backend; the case for every open-source project and commercial offering we have worked on or with within the monitoring and observability solution space.

Query Languages

Why would you need a custom query language if custom code can be plugged into the simulated replay of one or more application event pipeline projections? Query languages, especially those based mainly on a derivative of SQL, will never be able to reconstruct a comprehensive representation of a complex and composite situation via a SELECT and WHERE clause. Scientists in other domains know this, so they turn to complex-event processing and event-driven simulations to construct better models and capture learnings.


The growing interest in Cyber-Physical Systems (CPS) hints at a better future for Site Reliability Engineering (SRE) than what Prometheus and similar Metric storage systems offer. But it will not happen soon while the engineering community keeps blindly building and naively standardizing on many smaller legacy pieces (tracing) without a grand vision of what effective and efficient management of Systems of Systems (SoS) should look beyond a narrow, shallow perspective of something like CNCF‘s OpenTelemetry.