The observability community should move away from traditional metaphors like pillars and pipelines and adopt new ones like substrates and circuits. By doing this, we can gain a new and innovative outlook on tools and techniques, leaving behind outdated thinking that prioritizes data over decisions and content over control. It is time for a holistic, dynamic, and interconnected approach that is both practical and contextual, reminding us that intelligence needs to permeate everything, locally and globally.

Substrates

With the substrates metaphor, we see the community developing and building on top of an ontological foundational framework and runtime that provides a base for understanding and interpreting sensed system behavior, states, and situations. A framework that offers a unified model for representing complex systems, composite components, and contextual interactions.

Looking at various proposed telemetry standards and toolkits, we see very little commonality, sharing, or reuse across the various data-siloed pillars of traces, metrics, and logs within such standards and toolkits.

Instead of pillars, we will have a general model of representation at runtime that is very extensible and extremely versatile in modeling diverse systems, behaviors, and contexts.

The extensibility of the framework will allow for ease of design, fast development, and better delivery of new instruments for sensing and steering system behavior.

We will have a foundational layer of phenomenological constructs that support the development of diverse instruments operating dynamically and integrated at various scales and contexts.

New percepts, such as sensors and instruments, as well as measurements need not be silos.

Circuits

The circuits metaphor introduces the idea of localized processing, where signals are not just collected and stored but actively consumed and acted upon in real-time. This would lead to observability tools that are not only passive collectors of data but active participants in system management.

Circuits imply the presence of feedback loops, where signals are continuously monitored, processed, and used to inform immediate actions. This could lead to the development of observability systems that are more responsive and capable of dynamic adjustments, improving system resilience and performance.

The circuit metaphor encourages a modular approach to observability, where different circuits can be composed to create complex monitoring and control systems. This modularity would allow organizations to tailor their observability solutions to specific needs, combining different circuits to achieve desired outcomes, whether it’s real-time alerting, automated remediation, or deep system introspection.

We can expect the development of more sophisticated, autonomous observability platforms that can take proactive measures, such as predicting and preventing outages, optimizing resource allocation, or automatically tuning system parameters based on observed behavior.

Observability would shift from a reactive to a proactive discipline.

Future-Ready

Both substrates and circuits offer a compelling alternative to the outdated pillars and pipelines approach. They encourage a more holistic, action-oriented view of observability. As systems become more complex and distributed, the need for observability tools to adapt to new challenges will only grow.

The substrates and circuits metaphors provide a flexible and scalable framework that can evolve with technological advancements, ensuring that observability tools remain effective in the face of changing requirements. By focusing on a unified foundational framework (substrates) and dynamic, localized processing and action (circuits), the observability community could create tools that are more powerful, user-friendly, and future-proof, ultimately leading to more resilient and efficient systems.

We need to shift the emphasis from individual data points to overall system behavior. The circuit metaphor promotes a decentralized approach, where observability is built into the system from the ground up.

The focus on controllability via actions strongly emphasizes turning observations into actions, aligning with the ultimate goal of observability. Transitioning to this new way of thinking will require a cultural shift within the observability community, especially where vendors are dominant. While there are challenges to overcome, the potential benefits make this an exciting and promising direction for the future.