This is a question I heard several times this week, but have been consistently hearing it from customers, analysts, and others in my orbit. I think it is an important question, but it is also a very revealing question regarding the difference in how we want the world to be and the way the world is. Gateway policies are simple configurations to API gateways that employ common API capabilities that leadership and sometimes developers are looking to set in motion—-things like rate limiting, caching, CORS, specific types of authentication. The list goes on and on. This is something that is easier when you have a single gateway, but the reality that has unfolded across enterprise organizations is that you have multiple gateways due to multiple objectives, clouds, protocols, acquisitions, and other tribal boundaries that emerge across the sprawling API chaos that is your average enterprise organization today.
The business and technical leadership I am speaking with are asking me for a solution to how they can automate the detection and configuration of policies like rate limiting and authentication across potentially AWS API Gateway, Azure, Apigee, Kong, Tyke, or any of the other commercial or open source API gateways that have emerged across the enterprise landscape. Business leadership bought the messaging of leading API management providers like Apigee and Mulesoft, believing that a single vendor API gateway would take them into the future, when in reality this might cover some layers of our operations, but teams are diligently employing Kong, Tyk, NGINX, and other next generation API gateways like Zuplo and KrakenD to get what they need done. This all results in numerous ways you end up having to audit the state of API configuration and apply the policies you’ve identified as important as part of a federated approach to API management and wider API strategy. Ironically, in a world of APIs, interoperability, and standards, a whole generation of business and IT leadership, as well as vendors have missed the importance of APIs when it comes to doing business.
You see, this is just one of many mis-alignment of incentives that exist within each enterprise, as well as the industries they operate within. I’ve worked hard to see the landscape through as many perspectives as I can over the years, and as a result I find myself less surprised at the state of things. When I put on the first API Strategy & Practice conference (now API Specification Conference) I ended up with five gold sponsors who were all top API management providers. When Mashery found out Apigee was sponsoring them, they backed out. When SOA Software found out Mashery was involved they backed out. Eventually I got all the toddlers sponsoring, but then I ended up with potentially 5 API management keynotes, so I held a vote for who would win. SOA Software did, and I told them to make sure it was a good talk, and not just a product pitch. They did a product pitch. This was all before Mulesoft came to dominate the scene, but the point is that almost all of the API management providers are purely out for their own short-term interests. They do not care about APIs, standards, or interoperability. They care about dominating the category that is defined for them by analysts, and not about interoperability across the lifecycle or different vendors.
You can go back through this blog and probably find more than five posts ranting about API service providers not having APIs. API service providers across any stop along the API lifecycle, but specifically the ones who are anointed as the leaders in their slice of the technology pie are not concerned with having APIs. They will tell you, their customers, that you should have APIs–you must to remain competitive. But many API gateways still do not have first-class APIs. The API gateways that do API have APIs do not always cover the full spectrum of gateway capabilities with their APIs. The next layer of this onion is that even when there are APIs present they are of wildly different designs. Configuring rate limiting on one gateway is very different from configuring rate limiting on another gateway. Even if you can agree on rate limiting being a policy, possessing the vocabulary to describe the policy in a way that can be automated across different API gateways is a challenge. This state of things is due to many conscious and unconscious decisions made by API gateway providers and their investors, and while I’d love to say it is malicious, it really is just Occam’s razor. This reality is due to misaligned incentives, short-term outlooks, and decision makers who just aren’t very imaginative. Ultimately I love hearing folks who haven’t been in the game very long ask why we can’t just have a common interface across API gateways. ;-) Bless your heart.
Ok, let’s stop whining about the state of things and talk a little about how we are going to move forward. Like most of the world of APIs there will be no simple universal solution to the API chaos, but there will be several layers in which we can modularly abstract away the differences between API gateways and define a menu for automating the application of policies across vendors from multiple gateways. The first part of this equation is that we need to ground things with a common vocabulary for what policies are, despite what the nuances of the rules we apply to each individual gateway are. This takes an opinionated approach by someone who is neutral-ish in the space. Postman will take this stance. Next we need a way to articulate the nuances across the messy and sprawling API gateway landscape that exists today. We are doing this through layers of OpenAPI and AsyncAPI extensions, Spectral rules, and executable and automatable Postman collections that can be run manually, on a schedule, or baked into our CI/CD pipelines. To abstract away the API chaos that exists across the API gateways that exist today we need to automate deploying, auditing, and the configuration of our multi-gateway realty using APIs. Imagine that!
The application and integration landscape is APIs. The API operational landscape is APIs. Similar to industry API standards like PSD2/3 and FHIR, we’ll see some industry architectural standards emerge for core capabilities like compute, storage, and maybe gateways. We’ll more likely continue just seeing the misaligned incentivizes, short-term, and unimaginative thinking that we have been seeing to date. It is nice to think that API gateway providers will work together to ensure more interoperability, maybe employing something like Spectral or Open Policy agent as they have been with OpenAPI and AsyncAPI. This will reduce some of the friction that exists. Maybe you have a Terraform or Ansible wizard on your team and can master plan the utopian enterprise landscape, and everyone across teams will bow to the superior design of your landscape planning. But I am predicting that API chaos continues to dominate, and that we’ll have to equip our teams with very scrappy and modular solutions that they can employ as part of their regular work, and invest in more ways we can define and automate around and across teams to do the rest. I am leaning on machine-readable artifacts like OpenAPI, AsyncAPI, Spectral, and Postman collections will be how we abstract away the differences across not just the API gateway landscape, but every other stop along the API lifecycle.
For me, this question represents a very existential moment in my career, but I think also the global economy. You see over my career the power center of the enterprise has migrated from the database to the API gateway. I started my career in 1988 as a database engineer, and every company I worked at for the next twenty years, the power center was the database. But by 2010, this began to shift and externalize to the API gateway, which brings us to this multi-gateway reality we live in. This is less about API gateways and more about the power shifts occurring within the enterprise, but also the distributed nature of that power outside of the enterprise. Making not just APIs, but the API gateway layer one of the most important aspects of how power is welded, but also shifted today. You see, I am playing the long game here. Longer than each wave of API industry evolution, and I see the writing on the wall with API regulation like PSD2 and FHIR, but also the impacts of GDPR and CCP, as well as anti-competitive API ecosystem battles like we are seeing with FTC suing of Facebook for their acquisition of Facebook and Instagram. I am studying up on early waves of regulations of the railroad, telephone, electricity, radio, television, and aerospace to help me understand what is to come for regulation of not the Internet, but more specifically the web. Which will all have to be done at the API layer. Which makes the policies we are negotiating at the gateway layer today across the enterprise, very much resembling the policies we’ll be applying at the municipal, state, and federal levels across almost every industry in the future. Defining policies that govern how digital resources and experiences across all aspects of our online, and increasingly physical worlds is where I am putting all my chips on this API chaos roulette wheel that is spinning in front of me.
