This article was originally posted in 2020, but its identification of the industry problems still holds, if not more so today.

Two World Views

Two distinct hemispheres seem to form within the application monitoring and observability space – one dominated by measurement, data collection, and decomposition, the other by meaning, system dynamics, and (re)construction of the whole. For now, it seems the data-centric side’s left hemisphere is winning attention in the theater though failing in reality and practice.

The Monitoring Heist

Somewhere along the way, observability, much like monitoring, was hijacked by data-centric vendors peddling their ability and capacity to collect ever-increasing amounts of data without ever having to truly justify the cost and complication, as well as the cognitive burden, for all this inefficient and ineffective data collection, transfer, and storage. Unfortunately, interest in controllability, the primary purpose of observability, did not rise during this same period. This would have probably steered software and systems engineering teams away from an addiction to data, driven out of fear and without much regard for utility and meaning.

Big Data, Small Value

We have worked with and for application (performance) monitoring vendors, now marketing themselves as observability solution providers. In all that time, we’ve never seen a system that could accurately track the value derived from the enormous amounts of data collected. We often ask how much of the data collected was novel (informative) and consumed by a user, directly or indirectly. No one seemed to have an answer or a willingness to begin collecting and calculating such information. They seemed to fear the answer because it would call out so explicitly what many software engineers suspect of tracing, metrics, and logging – signals are primarily lost in big data noise. Each time a proposal for a more direct approach to signal extraction at source was made, it was always shot down by data-centric engineers with exceptional use cases. Rarely did we discuss how to attribute additional meaning, learning, and action to the datasets that surfaced in the product and to a user.

Our Precious Attention

Data collection is easy and lazy, meaning construction is demanding and vigilant. What is left today is a vast wasteland of data sold as an investment in the future of unknown unknowns. Investment is now of a junk status – data junk, that is. We have a new modern California Gold Rush, where merchants (data related) make excessive profits over miners (hapless users). The shuffling of the pan is now being replaced by an ad hoc query and analytics interface used in search of a golden signal without much guidance from and understanding of the whole and dynamics. Our attention, a most precious and protected resource, is now wasted on tiny data deposits swishing around in the pan. Vendors have cleverly moved the costs to users, and users seem content with simplistic but small fruitful forms of data manipulation. One prominent observability vendor goes so far as to emphasize the doing (with data) in their marketing material as opposed to what we expect of information and intelligence – sensing, signifying, and synthesizing. 

Simplistic vs. Simplicity

The left hemisphere is about narrowing things down to a certainty – a fact related to a prior event. In contrast, the right hemisphere accepts uncertainty and, from a systemic perspective, is open to possible dynamics of the process and flow. The left is mostly request and payload oriented, the right more conversational and contextual. The right is more in the present in quickly and intuitively assessing the overall system and process(es) in the now, whereas the left is about creating the past’s splintered (traced) narratives. The left seeks out a single truth, or root cause, through the data. The right embraces the multiple potentials and possibilities in the dynamics it gives attention to. The right looks to reconstruction and meaning for greater understanding. The left prioritizes the collection of data over all else. 

Reductionism vs. Holism

The left is about the What, whereas the right is about the How. The right is more adept at identifying problems; the left, on the other hand, aims to detail the problem when found by the right. For the right, situational awareness is of utmost importance and prominence; for the left, it is event datum mostly devoid of an overarching context. The left preference is sequential, with tracing and logging being the primary examples here, while the right is far more parallel in its processing. The right is concerned with synthesizing information, whereas the left is fixated on analysis and categorization. On the left are the trees, called traces; on the right, the forest, an ecological system. The right is tuned to consistency, and fast approximations of the system’s state and environment, whereas it is the incredibly slow and laborious confirmation the left favors most. The left relies on users to reactively search a data (re)collection, while the right actively formulates suggestions based on pattern recognition.

The Bigger Picture

We need both the left and right, but up to this point, we have all but neglected one side, the right, over the other, the left. We need controllability to follow in hand with observability to keep checks on unsustainable resource consumption and costs. We need componentized bundles of observability and controllability to be distributed to the edges and subnetworks with local autonomy. The right needs to regulate much more the degree of attention and other resources given to the left – the bigger picture must return, but not one consisting of data-laden dashboards or ad hoc querying tools. The days of data excess must end with a return to simplicity and significance; otherwise, we’ll be lost aimlessly walking around in a data fog, clouding judgment and hindering action.