Beyond Just API Discovery: The Technical, Business & Political Decisions Needed At Runtime
29 Aug 2016
I was included in a conversation the other day on Twitter about runtime API discovery which reminded me of some thoughts I was processing before I walked away from work this summer, and before I dive back into the technical work, I wanted to refresh these thoughts and bring them to the surface. Blogging on API Evangelist, and other channels which I publish my work on is how I work through these ideas out in the open, something that saves me expensive time and research bandwidth while I'm down in the trenches doing the coding and API definition work.
The Wider Considerations Of What Is API Discovery
Like APIs themselves, the concept of API discovery means a lot of different things to different people. I find that broadly it means actually finding an API (ie. searching on Google or ProgrammableWeb), but once you talk to a more technical API crowd, it often means the programmatic discovery of APIs. Ideally, this is something that is done using hypermedia suupported discovery, but can also apply a standard like JSON Home, or APIs.json. There are also many folks who are thinking about programmatic API discovery using OpenAPI Spec, API Blueprint, and other common API specification formats.
Some Thoughts On API Discovery At Runtime Today
The conversation I was pulled into was between some of the leading minds in the area of not just defining what APIs are, but also how we truly can scale, and conduct API discovery, consumption, and evolution of our resources in a logical way. This discussion is pushing forward how our web, mobile, and other systems can discover, put to work, and roll with the changes that occur around critical API resources. How a human finds a single API for their use is one thing, but how a system and application finds a single API and puts it to work at runtime is a whole other conversation.
The Hard Work To Define Runtime Discovery of APIs
Separating out the human and programmatic discussions around what is involved with the runtime discovery of APIs is just the first line of challenges we face. The second layer of challenges is often about cutting through dogma and ideology around specific approaches to defining an API. The third layer I'd say is that this is just hard work of separating out the numerous differences between APIs, each often possessing their own nuances and differing approaches to authentication. As with every other aspect of APIs, the challenges are both technical, and human-centered, which slows expectations around the progress we make, but I trust the community will ultimately execute on this properly.
The Even Harder Work To Define Runtime Discovery Of Many APIs
While I'm actively participating in the current discussions around runtime API discovery using both hypermedia, as well as other approaches, I can't help but keep an eye our for the future of how we are going to do the same thing across many APIs--this is what I do as the API Evangelist. We have a lot of work ahead of us to make each individual API is discoverable at runtime, but we also have a significant amount of work to harmonize this at web scale across ALL APIs--which is why so many hypermedia evangelists are so passionate about their work.
The Technical Considerations Of API Discovery At Runtime
98% of the discussions around API discovery at runtime focus on the technical--as it should be at this phase. Hypermedia design constraints, leading API definition specifications like OpenAPI Spec and API Blueprint, and API discovery formats like JSON Home and APIs.json are providing us with vehicles for moving this technical discussion forward. Ideally, our APIs should reflect the web, and when you land on the "home page" of an API, you should be presented with a wealth of links reflecting what the possibilities are (does your API have a navigation). Secondarily, if hypermedia is not desired or feasible, JSON Home and APIs.json should be considered, providing a machine readable index of what APIs are available within any domain, as well as additional details on what is possible using OpenAPISpec and API Blueprint.
The Business Considerations of API Discovery At Runtime
As technologists, we often fail when it comes to considering the business implications of our solutions, ranging from making sure to make money to keep them operational, all the way to industry-wide influences we should be aware of. I see many discussions amongst API specialists fall short in this area, which is why I started API Evangelist in the first place, and which is why I'm pushing these thoughts forward, and sharing with the public, even before they are fully baked.
At runtime, the technical considerations of where an API is, how to authenticate, and what parameters and other details need to be clear. However, when you elevate this process to operate across many APIs, important business criteria also become important--things like what plans are available, what do API resources cost, and are there volume options available. The example I like to use in this scenario is from the world of SMS, and making runtime business decisions across nine separate SMS APIs.
At runtime, I may have different business concerns with each execution, even after I know where the APIs exist. Some SMS blasts I may want to use the cheapest provider, while in other campaigns I may choose to use a higher priced, more trusted provider. These considerations made by a human in 2016 can be difficult, let alone having what we need to do in a programmatic way at runtime--something I've spent some cycles developing schemas and tools to help me sort through the mess. I have been able to establish patterns across some of the more mature API areas like SMS, email, search, and compute, but we are going to have to wait for other areas to evolve before this is even feasible.
There is a reason why I call my research in this area API plans and not simple API pricing. I feel this label reflects the future of business decisions we will have to make at runtime, which won't always be simply about pricing, and hopefully reflect our overall business plans--which are executed in real time at runtime in milliseconds. Sadly old ways of doing business by the enterprise continue to cast a shadow on this area, with companies hiding their pricing page behind firewalls, and not sharing the algorithm behind pricing decisions, let alone looking outward and following common industry patterns--beliefs around intellectual property and what is secret sauce will continue to hinder this all moving forward.
The Political Considerations of API Discovery At Runtime
Another area I have found myself paying attention to as the API Evangelist, beyond just the technology and business of APIs, is what I call the politics of APIs. Alongside the technical and business considerations, these often politically charged areas will have to be considered at runtime. Which API have the terms of service and privacy policies that reflect my companies strategy? Which API is the most reliable and stable? Can I get support if something fails? Is the long-term strategy of an API in alignment with our long-term strategy, or will they be gone within months due to funding and investment decisions (or lack of)? There are many political considerations that will have to be made at the programmatic level and included in runtime discovery and decision making around API integration(s).
Similar to the business considerations I have also invested some cycles into understanding the variability some providers are applying when it comes to the politics of APIs, like variability in terms of service and pricing, and how pricing, plan availability, availability, stability, and other ranking criteria can be made more machine readable and applied at runtime. As with the business concerns around API integration, there are many obstacles present when we are trying to make sense of the political impact at runtime. As more API providers emerge which are not resistant to sharing their API plans, I am able to document the variables at play in these algorithms, and share with the wider industry, but alas, many companies are holding these elements too close to their chest for the conversation to move forward in a healthy manner.
It is easy to think about the political runtime decisions that need to be made around APIs as purely being about terms of service, but there are much more grander considerations emerging, like which country and region we deploy into, and regulatory considerations that will have to be followed when putting API resources to work, or possibly injected at runtime like we are seeing within the drone space. Like terms of service are guiding almost everything we do online today, the politics of APIs will govern the runtime decisions that are made in the future.
Beyond Discovery And Considering The Technical, Business And Political Decisions Needed At Runtime
This is just a glimpse at the long road we have ahead of us when it comes to truly reaching the API economy we all like to talk about in the sector. Unfortunately, there are also many obstacles in the way of us getting to this possible future. We have to increase our investment in hypermedia and web-centric API solutions, and not just vendor-driven API solutions if we are going to move down this road. We have to be more transparent about our API plans, pricing, and the variables that go into the human and algorithmic business decisions that are driving our API platforms. We also have to start having honest discussions about the terms of service, privacy policies, service level agreements, and regulation that are increasingly defining the API space.
I am optimistic that we can move forward on all of this, but current beliefs around what is intellectual property, something that is fueled by venture capital, and further set back by legal struggles like the Oracle v Google API copyright case are seriously hurting us. The definition of your API is not IP or secret sauce. Your pricing and plan variables are not your secret sauce, and should not be hidden behind the firewall in the Internet age--regardless of your enterprise sales belief. The only way that we are going to continue meaningful automation of the growing number of resources being made available via APIs using Internet technology, is to share vital metadata out in the open, so we can make sure we are all making proper, consistent decisions at runtime--not just technically, but also the right business and political decisions that will make the API economy go round.