A Loose Federation of API Standards

I am corresponding back and forth with what I will call an open source and API standards therapist for now, helping me make sense of the ocean of API standards I am drowning in and assisting me in finding a way forward with. In my personal style, I need to write my way forward, processing what they have been sharing, and try to find ways I can take meaningful steps forward.At the scale and scope in which I operate in, it is a daily challenge for me to not be overwhelmed by what is happening around me, and even more difficult to herd the API cats forward in any way. This post is a digest of the wisdom this person is sharing with me, organized in a way that helps me process, and then hopefully assist me in taking one or two steps forward with what I am doing with both of my API Commons and APIs.json projects.

A Loose Federation of API Standards

I am working towards API Commons becoming a loose federation of API standards, with APIs.json being the discovery mechanism for these standards and how they are applied in real world situations. I strongly believe that the world of API standards would benefit from having more people involved, but most technical, and especially non-technical folks aren’t even aware of what’s going on with API standards today. I can see all the disparate and meaningful pieces of API standards that exist today but to outsiders and people just driving by it is very difficult to see them and understand how they work together. It is too difficult for insiders even to keep up with best practices in this space, understand where to look for information, making the process of choosing between competing standards is often time consuming, unpleasant, stressful, and rife with misinformation. I struggle, and I am intimate with most of the standards, people, and activities occurring. I can only imagine it is completely overwhelming for others to make their way forward with very simple concepts like should I use REST or GraphQL. I envision the API Commons to be a place where you can observe and discover what is happening, and this shouldn’t take too much effort because there is already a lot going on across all of these communities—-they just don’t always properly project and communicate the value.

No Shared Understanding of the Value of Standardization

We all pay a lot of lip service to standardization and interoperability, but if I’ve learned anything after 12 years steeped in the world of APIs, it is mostly an illusion and theater. Honestly, I struggle with being able to tell if someone is just talking about the importance of standards because they are lying, or they actually don’t understand what it all means. This is one of the things that caught my attention early on when it comes to the passion, dogma, and even venom that is present when we talk about APIs, standards, protocols, and the common ways in which we deliver our technology. My API standards therapist focuses in on three separate dimensions of this dilemma:

Decision-makers don’t grok the business value of standards. Some are even concerned by the potential for negative impact on what they (mistakenly) believe are competitive advantages.
Developers aren’t incentivized to adopt standards, suffer from Not-Invented-Here syndrome, and probably consider that standards folks are sitting in their ivory towers disconnected from the reality of day-to-day software development (they might be onto something about that later point).
Standards folks tend to focus on the technical aspects of the work they’re doing at the expense of broader benefits. For example, by putting consistency between APIs over common practices.

If I didn’t know better, I’d think there was a conspiracy to keep all of us divided and working against each other, but I understand it is more of an Occam’s razor situation. I can point to actual evidence of big tech companies actively dividing and conquering, but it really isn’t some grand conspiracy-—it is just selfishness, greed, and hubris on all our parts. We all right and wrong in the same motion. There are many contributing factors to this lack of shared understanding and forward motion.

Proprietary Standards - Many big companies and even small ones feel that their way of doing things is the right way and thus by default “the” standard, and everyone should just fall in line, and some players have the weight to throw around and dominance in a market to make this so.
Competing Standards - While it’s important to let the market choose winners, it’s also worth acknowledging the value of having an agreed-upon good enough solution over 3 perfect—yet competing—ones.
Lack of Community - The community of folks operating in that space is small and fairly homogeneous, yet it seems very siloed; there isn’t a sense of a common community that bridges the different groups that compose the ecosystem.
Engineering-Centric - While us engineers in the know will default to saying that we are doing this for the greater good and our communities are accessible, we are very good at gatekeeping through complexity and keep all this so far down in the weeds very few can ever get involved.
Cross-Cutting - Everyone wants to be responsible for the standard that fixes all of this and very few are interested in only focusing on or embracing the cross-cutting aspects we need like security, developer experience, internationalization, and other areas—we reinvent the wheel or do not address at all.
Layered Architecture - There doesn’t seem to be a clear and layered architecture for the different standards that compose the ecosystem, and not an XKCD cartoon where you just create a new standard, but develop a layered architecture from what already exists.
Use Cases - There aren’t any obvious use cases that connect the technical work being done to the solutioning of concrete real-world problems, connecting the dots that will matter to business stakeholders, elevating standards to the level necessary.

While there are undoubtedly other areas of deficiency when it comes to the forward motion of API standards, I agree with my API standards therapist that these are some of the top areas that make all of this feel off. While I am sure there is some fuckery occurring because of bad actors in all of this, I feel like most of this is just due to a lack of communication, listening, and understanding what already exists. I regularly encounter well-meaning standards and open-source folks doing battle in these areas, when they actually agree on what they are trying to do for the overall API community.

Operating on Different Planes

There is another aspect of all of this that I feel causes us to talk past each other while we are all trying to head in the same direction. I am regularly working over-time to explain to data folks why APIs are important, educating API specifications folks why tooling matters, and helping tooling makers adopt more of the existing standards that exist out there. All of these planes of existence all tend to operate under a loose circus tent we call open-source, when in reality there are many differences that keep us from finding common ground.

Tools - Most of the work we do gets lumped under “open source”, but in reality, tooling is what we mean, and specifications, standards, and tooling tends to suffer from malnourishment under this tent, while there are still synergy required for tools, specifications, and standards to work together.
Specifications - API specifications like OpenAPI, AsyncAPI, and GraphQL are almost always open source, depend on tooling, and apply industry and data standards, but have much broader applications and needs when it comes to governance and engagement across many different communities.
Standards - Industry level API and data standards may overlap with API specifications, but often reflect the needs of a specific domain, and will almost always get lost in the shuffle of open-source tooling or specification tooling, rendering it a second-class citizen that doesn’t benefit anyone.

We need to acknowledge that there are overlapping concerns between specifications, standards, and tooling, but that they also have different purposes, there are tradeoffs, and things should be handled differently. There are plenty of cross-cutting and shared concerns across these areas, but the educational, cultural, domain, and governance needs can vary widely across these planes, and we need to be more honest about what is needed to move things forward in a meaningful way that benefits everyone.

Getting Everyone on the Same Page

What API Commons needs to do is work to get everyone on the same page, and act as a home that spans all these communities. I’d like to first invest in bringing together information from across all the existing communities, but then also work to bring the people together to learn about each other and establish deeper relationships. Now that I have all of this loaded up in my head, I will ponder some more ideas for what some possible next steps might be, but here are a few off the top of my head in the moment.

Specifications - Identify what the existing specifications out there like OpenAPI, AsyncAPI, JSON Schema, GraphQL, gRPC, and others that are already doing the hard work to specific common approaches to delivering APIs.
Standards - Identify what the existing standards are like PSD2, FHIR, BIAN, and others that are already doing the hard work to specific the standards that industries and domains are needing, often applying the API specifications above.
Tooling - Identify what the critical tools are to support each of the specifications, such as parsers, generators, documentation, and the other core tooling needed to apply specifications and standards when it comes to specific API use cases.
Channels - Identify what channels information is flowing, tracking on Twitter, LinkedIn, GitHub, Forums, Slack, Discord, and other ways that the specifications, standards, and tooling are using to get their work down and engage with their communities.
People - Understand who the people and personalities are surrounding each of the specifications, tracking on who are behind the specifications, standards, and tooling, but also who in the community is making things go around.

I feel like we need to have a clear map of the space. This is what I have attempted to do with the API Specification Toolbox, but I would like to better focus on bridging between the business and technical worlds—not just providing a toolbox for developers. Now that I’ve brought the API Specification Toolbox into the mix, I’ll have to put some more thought into the differences between API Commons, APIs.json, and the API Specification Toolbox. In this moment, I want API Commons to be accessible by anyone, I want APIs.json to be the machine-readable discover of this world, and the API Specification Toolbox to speak to and enable developers—that might change. However, it is all up for grabs in this moment. This has helped me identify some potential next steps and identify what some of the break down in what is currently happening. It helps me clearly see the different planes we are currently operating on, which has the potential to begin forming bridges. 80% of what I have done as the API Evangelist over the last decade has just been bridging divides, sharing information, and shining a light on what is already happening. Over the years I have learned to also lead and contribute the other 20%, but really the base 80% is the most important part of what I am doing. I feel like this gets at what I am looking to do over the next decade. While I want to also get down to the business of defining standards, I feel like there is a lot of business that must happen first. There is a lot of bridge building to happen before we can connect the dots, and I don’t want to be in the business of creating yet another standard or process. I don’t feel like I have the answers, and most of the answers I am looking for area already out there—we just need to bring them to the surface.