Bryan over at Postman asked me about the “rubric for comparing the different options” when it comes to linting OpenAPI artifacts in service of enterprise scale API governance. I have posted a list of the four open-source listing solutions (Spectral, Redis CLI, Vacuum, and Optic, and my intent is to learn more about what people are using to lint their OpenAPI artifacts. I wouldn’t call it a rubric yet, but I can share more about what my strategy is when it comes to a policy and contract driven approach to API governance.
Let’s start with Spectral rules. I use Spectral because it is the most robust solution out there that is open-source, and I trust the original brains behind it. I also know that the policy engine for my API governance needs to be variable, meaning it supports many engines, specifications, and linting approaches. I am sticking with Spectral as the core of my API governance because it is the one I have the most rules built out in, but also have the deepest understanding of because of its relationship with OpenAPI (and other schema). You hit walls pretty fast trying to lint OpenAPIs using Spectral, which people tend to do two things: 1) shift from default Spectral function to custom functions, and 2) switch to or invent a competing standard. I did the third thing, worked within the constraints of default Spectral function until I intimately understood the surface area of the schema I am linting (primarily OpenAPI), but also the dimensions of whatever “problem” I was trying to lint and govern for—-expanding my approach to applying rules in different ways.
- Polices - The introduction of and grouping of Spectral rules by policy, which defines the business reason behind why we are governing our APIs.
- Lifecycle - The introduction of and grouping of Spectral rules by stage of the life cycle, which tells us the order in which rules (policies) are applied.
- Solutions - The introduction of and grouping of Spectral rules by solution, defining which impact onboarding, documentation, security and other areas.
- Extensions - Weaving in OpenAPI extensions into how you define and apply Spectral rules based upon custom OpenAPI extensions thoughtfully designed.
- Schema - Defining and applying Spectral rules beyond OpenAPI, and realizing that you can have more flexibility over the shape of linting rules in this way.
- Multi-File - The introduction of rules that can run across multiple OpenAPI files or other schema using an APIs.json contract binding it all together.
- Positive - Every Spectral has a negative and a positive rule, so that you don’t just focus on what is wrong, but also reinforcing around what is correct.
There are other dimensions I wrestled with how Spectral rules are defined and applied, augmenting existing discussions around applying Spectral rules during design-time, develop-time, or run-time. There are many different ways to define and apply your rules based upon these dimensions with a variety of motivations. The problem with most of the Spectral rules out there is they are very atomic, technical, and empty of any strategic, operational, and business alignment. The Spectral rules you find on the web via design guides tend to trigger a lot of false positives, and aren’t aligned with the way your teams operate. Like many other aspects of API operations, it takes someone who understands OpenAPI, JSON Schema, JSON Path, Spectral, your API lifecycle, the solutions you need to offer, while also making sure it is aligned with how your teams operate and how they see the world around them. Rules aren’t your salvation, and dumping rules into CI/CD and automation will definitely not save you.
I am looking for variability in defining and applying linting rules across this spectrum I speak of. Like many things in the API space it isn’t about the individual parts, it is about how you stitch it all together and orchestrate with them. Getting outside the OpenAPI realm and applying it to other communities, but also custom schema has really colored my view of things. Another thing that has really shaped my approach is when to use JSON Schema vs. Spectral rules-—there is a lot of nuance here. One important dimension I do not want to lose is the learning opportunity that comes with people being able to define, evolve, and apply their own rules—a big reason YAML Spectral exists and excels. However, as I continue to evangelize governance I am less convinced that people care to learn OpenAPI, JSON Schema, JSON Path, and Spectral rules-—let alone understand operations to better understand why, when, where, and how rules should be applied. This is actually the biggest problem, because if people don’t care I guess we just have to abstract all the rules, policies, lifecycle, and write custom scripts that do things. Why bother with the abstraction? IDK.
When evaluating Spectral, Redis CLI, Vacuum, Optic, as well as the other commercial offerings I am looking for how expansive and extensible these solutions are. I have to be able to apply to multiple specs, and have to be able to focus on what is right as well as what is wrong. I have to be able to orchestrate and prioritize governance, while also shifting left, guiding, and yes enforcing at some stages of the API lifecycle. I am looking to understand how rules get piped into these solutions, and where they can be automated or embedded. I am thinking about how well they travel to where developers live via Git and inside the IDE. I am interested in understanding how well these linting approaches expand to other stages of the API lifecycle, and where Open Policy Agent (OPA) fits into the picture. There are a lot of signals available across the open-source and commercial offerings that speak to what I’d consider to be the technical governance of API design right now, which I am looking to organize as part of a rubric, but I am also expanding that rubric to track API governance needs across the entire API lifecycle, and how it aligns with API platform efforts, but also more importantly with the business side of the equation and speaking to what leadership is interested in in any given moment.
