I am working to better define and shape when and how I apply both JSON Schema and Spectral rules. Both specifications are invaluable when it comes to API governance, and have significant overlap, but there are many ways in which they differ when it comes to trying to define, shape, and ultimately govern the API landscape. I don’t pretend to have the answers here, but I know enough to know these are complex technological specifications seeking to tame a sprawling API landscape, and I enjoy writing about how I am seeing things work in hopes of making just a little more sense of it all.
I do not consider myself an expert with either JSON Schema or Spectral, but I know enough to be dangerous, and I tend to thrive and innovate within an existing set of constraints. This exploration of JSON Schema and Spectral applies to any schema like OpenAPI or AsyncAPI, but I am interested in applying to APIs.json, but then also to any other machine-readable artifact I have defined as part of the APIs indexed within an APIs.json. While JSON Schema is broadly applied across any “JSON schema”, Spectral is very focused on OpenAPI and AsyncAPI. I would like to leave all that baggage at the door, and zoom out to think just about APIs.json, and governing the landscape it maps using JSON Schema and Spectral.
Let’s started with the fundamentals:
- What is JSON Schema? JSON Schema is the vocabulary that enables JSON data consistency, validity, and interoperability at scale.
- What is Spectral? A flexible JSON/YAML linter for creating automated style guides, with support for OpenAPI v3.1, v3.0, and v2.0 as well as AsyncAPI v2.x.
Like I said, I am looking to understand how I use JSON Schema to validate each version of APIs.json to ensure all indexes are consistent, but also Spectral to lint some of same things JSON Schema can be used for, but I have a much wider surface area, with multiple dimensions I am looking to serve with Spectral. I’ve been having trouble separating the reasons why I use each of these specs, and understanding why the overlap between them might matter. Hence the reason why I want to flesh out some of the details before I work on the JSON Schema for v0.18 of APIs.json, and keep strengthening my Spectral rules.
Why JSON Schema Matters
As the description states, JSON Schema gives me the vocabulary to consistently define, but more importantly validate the structure of each version of APIs.json. I only have a JSON Schema for 0.14 of APIs.json, but I am looking to play catch up with my latest 0.18. JSON Schema is the foundation of APIs.json. All the properties of an APIs.json are described and the general shape of each property, with some constraints like min, max, and enums. I use JSON Schema to make sure any APIs.json I produce is valid using common tooling, and provides me with the precise vocabulary I need to describe the shape of any API search index built with APIs.json.
I recently learned more about how VSCode uses JSON Schema to validate JSON files, but specifically about their extensions of JSON Schema for powering intellisense and validation using defaultSnippets and markdownDescription. I am keenly interested in using these two properties as a bridge between JSON Schema and Spectral to help connect the dots, automate, and reduce friction when applying governance at scale across the thousands of APIs.json files I am managing. I am hesitant to extend the JSON Schema because I don’t always know what I am doing in there, and Ben, Jason, and Julian might give me a hard time. ;-) But these properties piqued my interest, and got me thinking more about the relationship between JSON Schema and how I am using Spectral for a richer set of operational API governance guard rails.
Why Spectral Matters
I love exploring and pushing my limits of Spectral. Phil is too busy planting trees to give me a hard time. If I shed all the OAS and AsyncAPI specific language for Spectral, I guess I am left with “automated style guides”. I don’t think this does Spectral justice. I find Spectral, with its use of JSON Path, and built-in functions to be an extremely powerful base for defining, validating, and guiding API and API operational governance. Spectral is very granular. It also provides a rich vocabulary for describing APIs.json, but I’d say it is more about what you can do with APIs.json. If that makes sense. I see Spectral having several key dimensions that work in concert to help me map the public API landscape—let’s see if I can articulate how I am wielding the specification.
- Core Functions - The base capabilities of Spectral rules allow you to do so much to define what is possible with APIs.json, crafting rules to lint for API docs, plans, SDKs, terms of service, and other common building blocks. Using Spectral I am able to lint for all the common properties, which are also defined as JSON Schema enumerators, but Spectral lets me link to docs, expand on messaging, articulate the severity, and other ways I can find tune what is operational API governance.
- Positive / Negative - For over a year I have been exploring how I can establish what I consider to be positive and negative rules, based upon the fact that I do not want to just point out what I have done wrong—I want to also celebrate and showcase what I done well when producing an APIs.json. I will apply severity errors or warnings for the negative rules and I use info and hits across the positive rules, helping me articulate a spectrum of behavior associated with common patterns across API operations. This gives me more than just a boolean pass or fail, and I begin to get more semantic and forgiving when it comes to validation.
- Granularity - Another dimension I am exploiting Spectral rules is when it comes to the granularity of the rule. I widen the scope of the rule for negative, such as applying it across what I don’t want to see when it comes to publishing a road map that I have indexed as an APIs.json property. But I might want to be very precise in articulating the individual positive properties of a road map, allowing me to map the landscape and activity of a road map in very precise ways. This plays out differently depending on if I am governing the road map, change log, or maybe the plans and SDK pages. The granularity of rules matters a lot when it comes to mapping the landscape of APIs, and the details of APIs at scale.
- Modularity - I am also a big fan of the modularity of rules. I tend to group them together. This allows me a lot of flexibility when it comes to how I stitch rulesets together, and in what order I apply them. It is much harder to break up JSON Schema like this. It is just more foundational in nature, but Spectral is a massive buffet. JSON Schema is broad too, where the granularity (or not) and modularity of Spectral rules allows me to paint in different ways that I would not venture doing with JSON Schema.
- Custom - Once the core Spectral begins to fail you, custom rules open up entirely new horizons. Now I would like to share a word of caution. Push the core set of Spectral functions as hard as you can. Bang your head. Think differently before you switch to custom. I’ve learned a lot doing this. You can cluster, and take advantage of positive, negative, granular, and modular rules to expand the surface area of boolean switches. You have to assemble the circuit board for your operational API landscape.
- Schema Type - JSON Schema provides the foundation for any new schema I add to APIs.json, but Spectral rules allow me to develop rulesets that are for any schema type I may be adding as a property to my APIs.json index. As part of my API Commons work I have created schema for getting started, plans, SDKs, change log, road map, and support for API operations, which I use in my developer portal for the APIs.io search engine. This means I have JSON Schema and Spectral rulesets for each of these individual schemas, which I will be publishing as part of each API Common spec. Well, I may keep the ruleset back, but will be sharing the JSON Schema—I am keeping the rules as my secret sauce.
- Multi-File - One thing that APIS.json has pushed my Spectral capabilities is applying across multiple files. APIs.json is designed to index many different schemas into a single index, so I am using custom Spectral rules to stitch schema indexed as part of API properties I am defining for different public APIs. So I am developing rules to work across the getting started and the SDK pages to identify more advanced patterns and the connection of dots across all the artifacts I am using. I am using this to lint for things like the identifier across APIs and their overlays, and the ratings applied across APIs.json, but also to the OpenAPI.
I use JSON Schema as the foundation for APIs.json and each individual property I am trying to define in a machine-readable way. Spectral rules are the words I use to identify patterns and anti-patterns I see across public API portals. I have a JSON Schema that defines what properties are allowed for each API property, but I will have Spectral rules that tell what is possible with those properties across the API lifecycle, walking us through what is possible, identifying patterns, linking to more information about those patterns. I am also looking to understand and lint for groups of patterns applied, or groups of patterns missing. JSON Schema is bedrock, and Spectral rules are the landscaping.
100K Operational Governance
I hesitate calling what I am doing with APIs.json using JSON Schema and Spectral as governance. I am not looking to standardize. I mean, ultimately I am, at scale, but that is the primary focus. I am mapping the landscape, and trying to reserve judgment about what is good or bad, right or wrong, and just describe the patterns I find. I will be working to better understand and define what are patterns or anti-patterns, and apply my own unique brand of API ratings, but the primary objective here is API discovery.
I want to see the landscape. I need JSON Schema to lay the foundation of what I am seeing, and Spectral rules are the trees, bushes, rivers, roads, buildings, bridges, and other infrastructure. I am just getting started with applying JSON Schema and Spectral rules to APIs.json. I enjoy the flexibility of this work. I JSON Schema and Spectral rules to any schema, hanging the results on an APIs.json index with rich APIs.json overlays applied. I didn’t even get into the way I can apply rules differently depending on what overlay they are being applied to, where the JSON Schema is going to be pretty consistent. That will be a good topic for the next round of exploration at this intersection.
APIs.json is all about indexing the properties of API operations, identifying the human and machine-readable properties, while working hard to convert the human-readable into machine-readable artifacts, I am feeling like JSON Schema and Spectral rules are going to be key to doing this safely and consistently at scale. Anyways, I learned a lot writing this. It gave me a clearer view of how I am defining the shape of the API landscape. Next, I will take the complete 300+ API index for Amazon Web Services (AWS), and better articulate what I am laying out here. I want to be able to confidently share a blueprint of how AWS does APIs, while also providing a complex index of AWS APIs with rich metadata to power API discovery. Fun stuff. This was an unexpected by-product of my APIs.json work for APIs.io, but also aligns well with the vision for API Commons I shared this last weekend.
