Context in Context

In our pursuit of comprehending complex systems through observability, we’ve developed increasingly convoluted tools that focus on capturing the structural context—the static backdrop of components, configurations, and infrastructure against which our systems function. However, these tools overlook the rich tapestry of behavioral and situational contexts that lend meaning to system events.

While we can trace requests, monitor metrics, and map dependencies, we struggle to grasp the more fundamental dynamic interplay of multiple situations unfolding simultaneously across our systems.

Structural Faults

This fixation on structural elements mirrors our tendency to reduce complex systems to their constituent parts, rather than comprehending them as living entities exhibiting emergent behaviors and evolving situations. Accordingly, we face a strange blind spot in our observability practices: we collect vast amounts of data about the stage on which acts are played out, yet fail to capture the essence of the performance.

As our systems become increasingly complex and interconnected, this disparity between what our tools can represent and how systems behave becomes increasingly problematic and ultimately fatal.

Nearly all observability tooling focuses on structural aspects partly because it’s easier to collect, track, and index data like process IDs, hostnames, IP addresses, and configurations. This data is concrete and objective, making it easier to work with using traditional databases, dashboards, and monitoring tools.

Structural data gives a clear, stable base to build interfaces and queries on, which fits well with how many engineering teams have thought about systems in the past. In contrast, behavioral or situational aspects introduce a layer of abstraction that is indeed more nuanced and harder to quantify.

Decision Intelligence

Behavioral data captures patterns, phases, projections, sequences, synchronicity, and system dynamics over time, which can be difficult to model and query in traditional observability systems.

Situational data is particularly crucial in decision-making processes that necessitate adaptive responses, as it accurately reflects the immediate and frequently changing conditions that the system must navigate.

Situational awareness requires considering scenarios, settings, and scenes, concepts that align with human and social cognition rather than the discrete, transactional data points collected by instruments.

In many ways, the structural emphasis reflects the most straightforward path within current technical and cognitive constraints. Behavioral and situational contexts provide more valuable insights for understanding systems, particularly in dynamic, distributed environments where interdependencies shape outcomes.

To acquire such insights, observability needs to adopt models more akin to how we comprehend human and social behavior—tracking not only states but also transitions, patterns, and contexts that unfold over time, thereby creating a narrative of system behavior. A possible model is one based on cybersemiotics.

Contextualizing Sources

In the Substrates API, the concept of context is represented by the Context interface, which contains a method named “source” that returns an instance of the Source interface. The conceptualization of context is a source of information. The Context interface has a single direct subclass, Component.

The toolkit currently has three types of components: Circuit, Conduit, and Clock. You can use Circuits and Conduits to create multiple nested contexts. These contexts can be used to model different types of things, like structural and stable constructs, or dynamic and transient ones like situations, scenes, or scenarios.

As a developer, you can create a multi-contextual dynamical model that fits the complexity of your domain and its system of systems (SoS) and services. This can be done in both online and offline modes.

In the next post in this series, we examine the Source interface as well as Subscriber and Subscription.