Managing the Scope of Your OpenAPI
18 Nov 2020
Managing the size of an OpenAPI is a common challenge for API development teams. I have regular conversations with teams about the ways in which you can minimize the overall scope of an API, breaking things down into more manageable chunks, while also reducing redundancy—then encouraging reuse. Like other dimensions of the API space there are differing opinions on whether it is better to have one single OpenAPI for all of your APIs, or whether it makes more sense to break things down into well defined bounded contexts. You can see this debate occurring in Monolith vs Microservices, and MonoRepo vs Multi-Repo discussions across the API space. In this game, there are no right or wrong answers, just different approaches that work well for different organizations, teams, and individuals. My goal is to help folks understand the trade-offs by informing them enough to make their own decisions and set into motion API design practices that keep the API factory floor moving along.
The size of your APIs matter. The size of your OpenAPI for your APIs matters. The scope, complexity, and consistency of your APIs will define how easy they are to maintain and put to work. Organizations that do not use OpenAPI struggle with being able to define the API landscape at all, let alone being able to define the scope of individual APIs or groups of APIs. It is common for API providers who are moving into the OpenAPI realm to slam into the brick wall of API scope right away, realizing their OpenAPI definitions are too big to work with in some services and tooling, and become a maze of complexity when it comes to maintenance and consumption. While there are many ways your API design practices can reduce or at least better define the scope of your APIs and resulting APIs, there are a handful of ways you can slice and dice your APIs up to make them easier to manage, or settle in with accepting that there is one monolith OpenAPI to rule them all!
The design of your resource-centric HTTP APIs will help or hinder your ability to spread the scope of your APIs across multiple files. Depending on the underlying schema model, and the types of resources being made available, as well as the API design training and guidance given to teams, the number and scope of API paths being served up will vary. If you have a coherent resource and sub-resource strategy it becomes easier to break things down into more manageable groupings. Helping break your APIs and the OpenAPIs that define them into much more meaningful and reusable chunks. API paths will define your OpenAPI journey, and define the complexity and breakdown of your API lifecycle, so make sure you are thinking about the design of API paths across all teams throughout your organization.
How you use your HTTP methods as part of the API design process will help you downstream when it comes to breaking things down in your OpenAPI. Efficient use of GET, POST, PUT, and DELETE will help your OpenAPIs be more coherent, but it can also dictate the scope of other OpenAPI elements like parameters, headers, and the schema used within your request bodies. Your API design choices impact the overall scope of your API, something you might not realize until you've had to spend the time generating or crafting a complete OpenAPI definition for your API. How you express the methods of your API will help you define the scope of your APIs, the OpenAPIs that define them, as well as all the downstream effects of delivering and consuming an API—make sure you have a plan for this layer.
Adding to the dimensions of how your API design will define the scope of your API and OpenAPI, your usage of path and query parameters, as well as headers will shape how big or small your APIs are. Having an overall organization API design strategy will help lay the groundwork for the scope of the parameter layer of your APIs. Using common parameters for as much of this dimension will help you reuse parameters across APIs, and centralize them as part of your OpenAPI. Each unique parameter will add baggage to the surface area of an API, which is something that will have to be defined in an OpenAPI, adding to the overall scope of each API and resulting OpenAPI. Parameter sprawl is an easy trap to fall into, but once you spend the time laying out an overall API design strategy it is something that will become much more manageable for your team.
OpenAPI helps teams map out the surface area of their APIs so that they can see it. OpenAPI tags then allow you to start defining the bounded contexts in which each API path and method operates in. Providing you with the ability to begin slicing and dicing legacy API infrastructure into more meaningful groupings which can then possibly be segmented into separate OpenAPI files, and then versioned and evolved to become more in alignment with the overall API design strategy. Helping you define, understand, manage, evolve, then sensibly reduce or increase scope across the API landscape. OpenAPI tags are how you are going to do this work, and begin making sense of the API landscape that exists across your organization, but then also be able to become more coherent in how you community across teams when it comes to delivering and consuming APIs.
The components object represents the greatest opportunity for API providers to be moving from Swagger 2.0 to OpenAPI 3.0, and begin to get a handle on the scope of APIs, and your OpenAPI. The OpenAPI components object lets you centralize common schema responses, parameters, examples, request bodies, headers, security schemes, links, and callbacks, and then just reference them across your OpenAPI wherever they are needed with a $ref. This component centralization and reuse will become your biggest ally when it comes to managing the scope of your API. Components provide you with a single reusable toolbox of the components you need to apply across each of your APIs, providing an in-file components library that helps you logically define the scope of your API with well planned lego bricks rather than every part of your OpenAPI being a custom block.
Once you begin getting your API design house in order, and get more organized about how you use the OpenAPI specification to map out your API landscape, including having your APIs well tagged with logical bounded contexts, you will become more efficient with breaking your OpenAPIs up into separate files. Leveraging your tag work you can split up monolith OpenAPI documents into separate OpenAPI files which can then be used independently of each other. Helping you load balance the scope of your OpenAPI across many files, providing a well organized selection of smaller, more precise OpenAPIs rather than relying on everything being in a single file. Of course, there are tradeoffs to having to navigate multiple files, but it depends on whether you like your complexity vertically or horizontally--either way you will have to work through the paths, schema, and other parts and pieces.
Once you have things broken into separate files you can begin to use external references to each of the moving parts. One place to begin is to break out the schema for your APIs as separate JSON Schema files which can be referenced using an external $ref where they are applied within an OpenAPI, but then you can also use this individual JSON Schema files for validation and other purposes across your operations. External references allow you to break things up into separate files, potentially setting up a centralized components library, which can just be referenced from wherever they are needed. In addition to the schema sprawl introduced by this approach, not all OpenAPI enabled tooling will support external references, requiring the rehydration of files before imported or put to work within services and tooling, further balancing out the tradeoffs around API scope.
Start Managing OpenAPI Scope Today
I know that many folks will just opt for the easier route of having a single monolith OpenAPI, but that will just push the technical debt and complexity down the road. At some point you will have to manage OpenAPI scope and you are better off starting as early as you can. As stated several times, having an organizational wide API design strategy will inform how you manage the scope of your API, but thinking about scope at the path, method, and parameter levels, while leveraging tags and components to optimize the scope of each of your OpenAPIs can bring a lot of benefits. You can still remain a mono OpenAPI, but actively work to manage scope. However, if you have thousands of APIs being developed, iterated upon, and delivered across your organization I highly recommend beginning investing in separate files and external references, as this is where you will begin to see the biggest bang for your buck when it comes to managing the scope of your OpenAPIs.
The scope and complexity of your OpenAPI will be a mirror of your API scope and complexity, which is a mirror of your organizational scope and complexity. If creating and managing an OpenAPI seems unwieldy it is because the design and complexity of your API has become unwieldy. Most people who begin on their OpenAPI journey do so to deliver API Documentation, but then they quickly realize how OpenAPI is much for than just documenting what is going on for consumers. Managing your OpenAPI becomes about managing your API operations and the lifecycle you employ to deliver, iterate upon, and support your APIs. OpenAPI scope is often an illusion based upon operating for years without being able to see the entire API landscape--once you generate that first OpenAPI and begin to get intimate with all the moving parts, you quickly begin to see a bigger picture where scope is more about the lack of visibility and manage practices in the past, which is something that becomes radically amplified by the maturing of your API design and governance strategy—but that is another story.