I have been working on various approaches to rating APIs since about 2012. I have different types of algorithms, even having invested in operating one from about 2013 through 2016, which I used to rank my daily intake of API news. Helping me define what the cream on top of each industry being impacted by APIs, while also not losing site of interesting newcomers to the space. I have also had numerous companies and VCs approach me about establishing a formal API rating system—many of whom concluded they could do fairly easily and went off to try, then failed, and gave up. Rating the quality of APIs is subjective and very hard.
When it comes to rating APIs I have a number of algorithms to help me, but I wanted to step back and think of it from a more simpler human vantage point, and after establishing a new overall relationship with the API industry. What elements do I think should exist within a rating system for APIs:
- Active / Inactive - APIs that have no sign of life need a lower rating.
- Free / Paid - What something costs impacts our decision to use or not.
- Openness - Is an API available to everyone, or is a private thing.
- Reliability - Can you depend on the API being up and available.
- Fast Changing - Does the API change a lot, or remain relatively stable.
- Social Good - Does the API benefit a local, regional, or wider community.
- Exploitative - Does the API exploit its users data, or allow others to do so.
- Secure - Does an API adequately secure its resources and those who use it.
- Privacy - Does an API respect privacy, and have a strategy for platform privacy.
- Monitoring - Does a platform actively monitor its platform and allow others as well.
- Observability - Is there visibility into API platform operations, and its processes.
- Environment - What is the environment footprint or impact of API operations.
- Popular - Is an API popular, and something that gets a large amount of attention.
- Value - What value does an API bring to the table in generalized terms.
These are just a handful of the most relevant data points I’d rank APIs on. Using terms that almost anyone can understand. You might not fully understand the technical details of what an API delivers, but you should be able to walk through these overall concepts that will impact your company, organization, institution, or government agency when putting an API to work. Now the more difficult questions around this API rating system, is how do you make it happen. Gathering data to satisfy all of these areas is easier said than done.
Getting the answer to some of these question will be fairly easy, and we can come up with some low tech ways to handle. However, some of them are near impossible to satisfy, let alone do it continually over time. It isn’t easy to gather the data needed to answer these questions. In my opinion no single entity can deliver what is needed, and it will ultimately need to be a community thing if we are going to provide any satisfactory answer to these API rating questions, regularly satisfy them on a regular schedule, and then honestly acknowledge when an API goes dormant, or away altogether. To properly do this, it would have to be done out in the open, and a few things I’d consider introducing would be:
- YAML Core - I would define the rating system in YAML.
- YAML Store - I would store all data gathered as YAML.
- GitHub Base - Everything would be in a series of repositories.
- Observable - The entire algorithm, process, and results are open.
- Evolvable - It would be essential to evolve and adapt over time.
- Weighted - Anyone can weight the questions that matters to them.
- Completeness - Not all the profiles of APIs will be as complete as others.
- Ephemeral - Understanding that nothing lasts forever in this world.
- Collaborative - It should be a collaboration between key entities and individuals.
- Machine Readable - Able for machines to engage with seamlessly.
- Human Readable - Kept accessible to anyone wanting to understand.
I will stop there. I have other criteria I’d like to consider when defining a ranking system. Specifically, a public ranking system, for public APIs. Actually, I’d consider a company, organizations, institution, and government agency ranking system with an emphasis on APIs. Do they do APIs or not? If they do, how many of the questions can we answer pretty easily, with as few resources as possible, while being able to reliably measure using these data points on into the future. It is something we need. It is something that will be very difficult to setup, taking a significant amount of investment over time. It will also rely on the contribute of other entities and individuals. It is something that won’t be easy to make happen. That shouldn’t stop us from doing it.
That is the basics of my API rating formula. Version 2019. This is NOT an idea for a startup. There is revenue to be generated here, but not if approached through the entrepreneurial playbook. It will fail. I’ve seen it happen over and over. This is a request for the right entities and individuals to come together and make it happen. It is dumb that there is no way of understanding which APIs are good or bad. It is also dumb that there is no healthy and active API search engine after APIs having gone so mainstream—another sign of the ineffectiveness of doing not just API rating, but API discovery as a venture backed startup. Sorry, there are just some infrastructural things we’ll all need to invest in together to make this all work at scale. Otherwise we are going to just end up with a chaotic, unreliable network of API-driven services behind our applications. If you’d like to talk API ratings with me, drop me an email at [email protected], or tweet at @apievangelist.
