I had a customer this morning asking another question I hear a lot-—where do you begin when mapping the existing API landscape within an enterprise. API sprawl is common, and it is rare for an organization to have a handle on all of the APIs they produce for internal and external needs. It is even rarer for enterprises to also have a handle on all of the APIs that they consume, adding another dimension to the API landscape mapping discussion. It is a daunting challenge and work that will never truly be done, but let’s explore where you can start with this important work.
Talking to Teams
Start by talking to teams and asking where APIs are—-see what they share. Start posting to relevant internal destinations, channels, and emailing people who lead different domains, lines of business, and teams. I recommend just simply asking where the APIs are. Don’t be specific about APIs they produce or consume, just ask where their APIs are, and see what they say. Document how each person responds, and what they share. Is it documentation? Is it OpenAPI? Is it Postman collections? Do they point to a portal, Git repository, or maybe a Postman workspace. How and what they share will be as important as the APIs themselves, and will go a long way towards defining how you map the API landscape, and what kind of effort it will take—-as well as who is going to help you do the hard work to properly define your enterprise API operations.
Gather Documentation
As you talk to teams, start searching for API documentation. If you have any internal, partner, or public API portals, start there. Then move to other wikis or places where teams document their work, and search for the word API, REST, SDKs, and other common words and phrases used to describe APIs. Gather the links of all the 3rd-party documentation for the services, infrastructure, and tooling your company already uses, and find their developer portal and documentation. Existing API documentation will be the doorway to mapping the API landscape, but may only be a fraction of the picture regarding your enterprise API landscape. Ideally teams are documenting APIs as they produce them, and including APIs used to develop other Apis, applications, and integrations, but it is common for this work to get lost in the hustle and bustle of project deadlines and shifting business priorities.
Inventory API Artifacts
Some API documentation may be generated using OpenAPI, RAML, and other API specifications. Teams may also publish OpenAPI, Postman Collections, and other machine readable artifacts to GitHub and Postman Workspaces. Find all the API artifacts you can, and collect any you find—-you can dedupe and aggregate them later. Depending on how your enterprise operations teams operate, these artifacts are likely to be spread across common places, but also private stores like locally via Postman, SwaggerHub, or via other tooling and services you depend upon. These machine-readable artifacts are how you will continue to map the API landscape, and the quality and completeness of the artifacts you find will significantly impact how much work you have ahead of you mapping the API landscape. The availability and usage of OpenAPI and JSON Schema, as well as Postman Collections will define how far along your enterprise is in its API journey.
Proxy Applications
As you continue talking to teams, gathering API documentation and inventory API artifacts, you can also use proxies like Postman Interceptor and Charles Proxy to define the APIs used behind your web and mobile applications. Postman Interceptor will generate Collections for the APIs behind any web application you use, and it is easy to click around every part of an application to find the APIs that deliver the resources and capabilities used in each screen. Mobile applications can easily be routed through your Postman to generate Collections or the more industrial-grade Charles Proxy to then output HAR files, which can easily be translated into OpenAPI and JSON Schema, which you will use as the basis for your wider API landscape map. The goal with using proxies is to map out the parts of the API landscape that do not have documentation and are obfuscated by existing web and mobile applications, and outputs more machine-readable artifacts that you can use as part of your overall landscape map.
Look to Log Files
Another place you can look to map the API landscape is the log files for web servers, gateways and other core infrastructure used to deliver APIs. Depending on the industry you operate in, the data stored in log files may contain paths, parameters, and schema you use to generate machine-readable artifacts that continue to fill in gaps in your API landscape map. Like proxying applications, sifting through log files will help you fill in the dark shadows of APIs used to power web and mobile applications. If you are in a heavily regulated industry and have tight security practices you may not be able to see the details or requests and responses, but it will at least give you path, methods, and query parameters for API requests—something you can use to further flesh out your map, and begin looking else for clues regarding the request and response payloads of these APIs. Log files can be tedious and take compute power to properly sift through, but they can be the missing link you need to properly map out the API landscape across the enterprise.
Sift Through Code
The last place I recommend looking when it comes to mapping the API landscape is with the code used to produce and consume APIs. Backend server code, as well as front-end client code have all the technical details you need to define the surface area of APIs. Code annotations can be added to generate OpenAPI from almost any programming language, but the code itself can also be parsed for paths, parameters, and schema. Ideally you are able to accurately map the API landscape by talking to folks, gathering documentation, and inventorying artifacts, but proxying applications, looking through log files, and reverse engineering code should fill in most of the gaps for you. This portion of landscape mapping is going to take the most technical skills and may need some specific language abilities to properly reverse engineer what is happening at the code level with producing and consuming Apis.
Central Landscape Map
Talking to teams, gathering documentation, and inventorying API artifacts is where you want to spend most of your time mapping the API landscape, but to truly find all the shadow and zombie APIs in production you are going to have to proxy applications, look through log files, and probably crack open some code. Ideally your approach lives on beyond just your initial inventorying process, and you are tapping into all of these sources to produce an accurate map of the API landscape in an ongoing way. You should be producing robust OpenAPI and JSON Schema artifacts and your API map, which are augmented by Postman Collections, and organized via Git repositories and Postman Workspaces. Your landscape map is defined as OpenAPI, JSON Schema, and Postman Collections, but this is just the technical details—you will also need to begin mapping out the business landscape that surrounds these APIs, which is something I will write about in a separate post.
To get started I recommend firing up a Git repository in GitHub, GitLab, or Bitbucket. Use the README as the summary of your work, and use the issues as how you track all your work. Map out teams and APIs using this repository, adding links, as well as actual OpenAPI, JSON Schema, and Postman Collections to the repository. Don’t worry about how these artifacts get used early on, just mapping the landscape, and defining the technical details of APIs across operations. As this landscape map comes into focus, then you can consider how you will document, visualize, distribute, and automate around this landscape map. Just focus on mapping the landscape. Everything else can wait until you have a meaningful map of what is happening. You will learn a lot from this process, and you will learn who is your ally when mapping the landscape. Once you have a meaningful representation of the API landscape, then you can get to work governing what is happening and beginning to shift the behavior of teams producing APIs as well as the consumers of APIs, but first you have to get a lay of the land.
