The overemphasis on data instead of signals and states has created a great fog. This data fog leads to many organizations losing their way and overindulging in data exploration instead of exploiting acquired knowledge and understanding. This has come about with the community still somewhat unconcerned with a steering process such as monitoring or cybernetics.
Author: William David Louth
Observability in Perspective
There are many perspectives one could take in considering the observability and monitoring of software services and systems of services, but here below are a few perspectives, stacked in layers, that would be included.
Change in Observability
Observability is effectively a process of tracking change. At the level of a measurement device, software or hardware-based, change is the difference in the value of two observations taken at distinct points in time. This change detection via differencing is sometimes called static or happened change. Observability is all about happenings.
A Story of Observability
Once upon a time, there was a period in the world where humans watched over applications and services by proxy via dashboards housed on multiple screens hoisted in front of them – a typical mission control center. The interaction between humans and machines was relatively static and straightforward, like the environment and systems enclosed.
Bi-directional Observability Pipelines
Substrates changed everything by introducing the concept of a Circuit consisting of multiple Conduits fed by Instruments that allowed Observers to subscribe to Events and, in processing such Events, generate further Events by way of calling into another Instrument. But with the introduction of Percept and Adjunct, it is now possible for Observers attached to Circuit and its locally registered Sources to process Events that have come from a far-off Circuit within another process.
The Evolution of Substrates
With the latest update to the Substrates API, the metamorphosis to a general-purpose event-driven data flow library interface supporting the capture, collection, communication, conversion, and compression of perceptual data through a network of circuits and conduits has begun.
A Situational Control Tower
It is time for new direction closer aligned to goals, focused more on the dynamics of systems that humans are already highly adapted to with their social intelligence, within which situation is a crucial conceptual element of the cognitive model. Understanding and appropriately responding to different social situations is fundamental to social cognition and effective interpersonal interactions.
Observability: Disruptions
Disruptions are one factor affecting the maintenance of service quality levels. A disruption is an interruption in the flow of (work) items through a network that can, for a while, make it inoperable or where the network flow performance is subpar. Depending on the severity of the disruption, a network may need to replan and restructure itself for a period afterward. There are two main categories of disruptions: disturbance and deviation.
Observability: Projecting Ahead
The low-level data captured in volume by observability instruments has closed our eyes to salient change. We've built a giant wall of white noise. The human mind's perception and prediction capabilities evolved to detect significant changes to our survival. Observability has no steering mechanism to guide effective and efficient measurement, modeling, and memory processes. Companies are gorging on ever-growing mounds of observability data collected that should be of secondary concern.
Priming Observability for Situations
The Recognition-Primed Decision (RPD) model asserts that individuals assess the situation, generate a plausible course of action (CoA), and then evaluate it using mental simulation. The authors claim that decision-making is primed by recognizing the situation and not entirely determined by recognition. The model contradicts the common thinking that individuals employ an analytical model in complex time-critical operational contexts.