The lack of good API design examples out in the wild is because of the extremely expansive engineering effort required to get a design to hit all the correct tick boxes, even in the smallest of contexts now and into the future of a changing landscape.

An Art of Forms

API design, particularly the design of programming libraries, is an art form that seems to be disappearing, somewhat understandable in that it can be soul-crushing up to the point of the last move when all the pieces come together; even then, one is never sure.

It can be as exhausting as competitive chess as you mentally project board positions and pieces into many different possible simulated paths and evaluate the success of each with severely limited information beyond a vision about what the other(s) are thinking, expecting, and needing. It was hard before the Agile movement was weaponized; now, it seems next to impossible.

Modern API Design

Modern API design, which is versatile and extensible, presents profound engineering challenges. In the past, design primarily focused on making it explicit and exact for the caller to understand how to achieve an outcome and execute the operation efficiently within an application-specific conversational context.

Today, a modern API design for a library must, at minimum, (1) allow for multiple implementations of a library interface via a service provider interface (SPI) contract, (2) offer extensibility points that configure and customize an implementation, (3) adapt dynamically to diverse runtime environments and execution behaviors, (4) provide development testkits and benchmark drivers, (5) employ advance experimental language features wisely, (6) integrate robustly and safely with other external components and systems, and (7) offer observability and controllability, even when the library itself is an observability toolkit, such as Substrates.

Design Iterations

Good design takes time, over many iterations (of converging and diverging design paths), in developing, discovering, discussing, discounting, and occasionally destroying. Engineers often release too early and cement existing design mistakes as the resistance to change grows from the outside once a design has made it into the public domain and external dependencies have been created.