Traditionally, observability data pipelining operates like a single assembly line, where workers halt, examine items, and consult manuals before proceeding. This approach is functional but slow due to inspections. Instead of thinking of observability as one long pipeline, think of it as a graph with many possible routes. Once we know where something needs to go, we can create a specific route for it, like how a GPS creates a particular path for your destination instead of having to check a map at every intersection.
Category: Observability X
Observability X – Sources
Observability has been reduced to the straightforward processing of data collected and its forwarding to a remote centralized endpoint. While frameworks such as OpenTelemetry have standardized this, they overlook a fundamental aspect of true observability: the dynamic nature of observation itself. A more effective and efficient observability approach establishes live, adaptable connections between information sources and observers, both locally and remotely, as well as online and offline.
Observability X – Contexts
In our pursuit of comprehending complex systems through observability, we’ve developed increasingly convoluted tools that focus on capturing the structural context—the static backdrop of components, configurations, and infrastructure against which our systems function. However, these tools overlook the rich tapestry of behavioral and situational contexts that lend meaning to system events.
Observability X – Naming Percepts
In the realm of observability, naming holds paramount importance. It directly influences measurement tracking, correlation, and interpretation, significantly impacting system visibility and problem-solving capabilities. Names are essential for distinguishing the diverse range of system measurements collected by observability tools.
Observability X – Staging State
Managing state in observability instrumentation presents unique challenges, particularly when dealing with concurrent operations. The Humainary Substrates API offers an elegant solution by offloading state management to pipeline processing. Through its Stage interface and built-in concurrency controls, developers can create custom instruments (percepts) without getting entangled in thread safety issues.
Observability X – eXtensibility
Today's observability toolkits represent a one-size-fits-all approach that fails to capture the nuanced needs of modern systems. This post advocates for extensible observability toolkits that empower teams to construct custom instruments (percepts) tailored to their requirements.
Observability X – Location Agnostic
Our approach to observability has been constrained by the notion that instruments and observers are fundamentally distinct entities. This post offers an alternative perspective: observers are themselves instruments, constructing more comprehensive and insightful observations by integrating data from lower-level instruments.