This is a report for the Department of Veterans Affairs microconsulting project, “Governance Models in Public and Private Sector”. Providing an overview of API governance to help the VA, “understand, with the intention to adopt, best practices from the private and public sector, specifically for prioritizing APIs to build, standards to which to build APIs, and making the APIs usable by external consumers.” Pulling together several years of research conducted by industry analyst API Evangelist, as well as phone interviews with API practitioners from large enterprise organizations who are implementing API governance on the ground across the public and private sector, conducted by Skylight Digital.
We’ve assembled this report to reflect the interview conversations we had with leaders from the space, helping provide a walk through of the types or roles and software architecture being employed to implement governance at large organizations. Then we walk through governance as it pertains to identifying possible APIs, developing standards around the delivery of APIs, how organizations are moving APIs into production, as well as presenting them to their consumers. Wrapping up with an overview of formal API governance details, as well as an acknowledgement that most API governance is rarely ever a fully formed initiative at this point in time. Providing a narrative for API governance, with a wealth of bulleted elements that can be considered, and assembled in the service of helping govern the API efforts across any large enterprise.
Roles Within An Organization
There are many roles being used by organizations who are leading the conversation around the delivery of high quality, industry changing APIs. Defining the personalities that are needed to make change across large organizations when it comes to delivering APIs consistently at scale. While there may be many names for the specific roles leading the charge it is clear that these people are bringing a unique blend of skills to an organization, with an emphasis in a couple of key areas:
- Leadership - Providing leadership for teams when it comes to APIs.
- Innovation - A focus on innovation using APIs across the organization.
- Communication - Facilitating communication across all teams, and projects.
- Advisory - Acting as an advisor to existing leadership and management.
- Strategy - Helping existing teams develop, evolve, and realize their strategy.
- Success - Focusing on helping existing teams be successful when it comes to APIs.
- Architect - Bringing a wide variety of software architectural skills to the table.
- Coaching - Being a coach to existing teams, and decision makers across the organization.
Bringing together a unique set of skills that range from the technical to deep knowledge of the business domain, into a concentrated, although sometimes distributed effort to bring change across an organization using APIs. Along with these roles, many large organizations are investing in new types of structure to help develop talent, take charge of new ideas, and move forward the enterprise wide API strategy, with a handful of common characteristics:
- Labs - Treating API efforts as if it is a laboratory creating new experiments.
- Center - Making it a center for API thinking, ideation, and for access to information.
- Centralized - Keeping all efforts in a single group or organization within larger entity.
- Distributed - Emphasis on keeping API knowledge distributed and not centralized at all.
- Global - Acknowledging that APIs will need to be a global initiative for larger organizations.
- Excellence - Focusing on the organization bring excellence to how APIs are delivered.
- Embedded - Making sure there is API knowledge and expertise embedded in every group.
Combing a unique set of skills and personalities, into a focused organization that takes the reins when it comes to leading API change, and digital transformation at an organization. While many of these efforts emphasize a center, or centralized presence, many are also realizing the importance of embedded and distributed approach, ensuring that talent, and ideas grow within existing teams, and are not seen as just some new, external, isolated group or initiative.
There clearly is not a single role or organization structure that brings success to API efforts at scale across the enterprise, however there are clear patterns being applied in the early stages that can be emulated. Helping ensure that API knowledge and expertise is available and accessible by all groups across an organization, and all its geographic regions, ensuring that the entire enterprise is part of the conversation and moving forward in unison.
Software Architecture Design
Governance is all about shaping and crafting the way we design and architect software, leveraging the web, and specifically web APIs to help drive the web, mobile, device, and network applications we depend on. There are a number of healthy, and not so healthy patterns across the landscape for considering as we look to shape and transform our software architecture, begin honest about the forces that influence what software is, does, and what it will ultimately become.
Software architecture is always a product of its environment, being influenced by a number of factors that already exist within any given domain. We are seeing a number of factors influence how large enterprises are investing and defining their software architecture. Here are a handful of the top areas of consideration when it comes to how the domain an enterprise exists within impact architecture:
- Resources - The types of digital resources an enterprise already possess will drive software architecture, defining how it works, grows, expands, and shifts.
- Schema - Existing schema define how data is stored, and often gathered and syndicated–even if this is abstracted away through other systems, it is still influencing architectural decisions at all levels.
- Process - Existing business process are already in motion, driving current architecture, and is something that cannot immediately be changed, without having echoes of impact on future architectural decisions.
- Industry - External industry factors are always emerging to shift how software architecture is crafted, providing design, development, and operational factors that need to be considered as architecture is refactored.
- Regulatory - Beyond more organic industry influences, there are regulatory, legal, and other government considerations that will shift how software architecture will ultimately operate.
- Definitions - The access and availability of machine readable definitions, schema, process, and other guiding structural elements that can help make software architecture operate more efficiently, or less efficiently in the absence of standardization, and portability.
Domain expertise, awareness, and structure will always shape software architecture, and the decision making process that surrounds it. Making it an imperative for there to be an investment in internal capacity as well as leveraging external expertise and vendors when it comes to shaping the enterprise architectural landscape. Without the proper internal capacity, domain knowledge can be minimized, weakening the overall architecture of the digital infrastructure nutrients an enterprise will need to move forward.
We can never escape the past when it comes to the software architectural decisions we make, and it is important that we don’t just see legacy as a negative, and also view legacy as a historical artifact that should move forward. Maybe not always the same legacy code should be in forward motion, but the wisdom, knowledge, and lessons learned around the enterprise legacy should be on display. Here are a handful of the legacy considerations we’ve identified through our discussions.
- Systems - Existing systems in operation have a significant influence over all current and future architectural decisions, making legacy system consideration a top player when it comes to decision making around software architecture conversations.
- People - Senior staff who operate and sustain legacy system, or were around when they were developed possess a significant amount of power when it comes to influencing any new system architecture, and what gets invested in or not.
- Partners - External partners who have significant history with the enterprise possess a great deal of voting power when it comes to what software architecture gets adopted or not.
- Trauma - Legacy trauma from historical outages, breaches, and bad architectural decisions will continue to influence the future, especially when legacy teams still have influence over future projects.
Systems, people, partners, and bad decisions made in the past will continue to drive, and often times haunt each wave of software architectural shifts. This influence cannot be ignored, abandoned, and needs to be transformed into positive effects on next generation investment in software architecture. Change will be inevitable, and legacy technical and cultural debt needs to be addressed, but not at the cost of repeating the mistakes of the past.
After legacy concerns, we all live in the reality have been given, and it is something that will continue to shape how we define our architecture. Throughout our discussions with companies, institutions, and government agencies regarding the challenges they face, and the current forces that shape their software architecture decisions, we found several recurring themes regarding contemporary considerations that were making the largest impact:
- Talent Available - The talent available for designing, developing, and deploying of API infrastructure dictates what is possible at all stages.
- Offshore Workers - The offshoring of work changes the governance dynamics, and requires strong processes, and a different focus when it comes to execution.
- Mainstream Awareness - Keeping software architectural practices in alignment with mainstream practices helps shape software architecture decisions, allowing them to move forward at a healthier pace.
- Internal Capacity - It has been stated several times that doing APIs at scale across the enterprise would not be possible without investing in internal capacity over outsourcing, or depending on vendor solutions.
Modern practices continue shaping how we deliver our software architecture, defining how we govern the evolution of our infrastructure, and find the resources and talent to make it happen. Keeping software architecture practices in alignment with contemporary approaches helps streamline the road map, how teams work with partners, and can outsource and work with external entities to help get the job done as efficiently as possible.
The technology we adopts helps define and govern how software architecture is delivered, and evolved. There are many evolution trends in software architecture that has moved the conversation forward, allowing teams to be more agile, consistent, and efficient in doing what they do. As we studied the architectural approaches of leading API providers across the landscape, and engaged in conversations with a handful of them, we found several technologically defined views of how software architecture is influencing future generations and iterations.
- Vendors - Specific vendors have their owning guiding principles to how software architecture gets defined, delivered, and governed. Often times given an outsized role in dictating what happens next.
- Frameworks - Software and programming language frameworks dictate specific patterns, and govern how software is delivered, and lead the conversation on how it evolves. Software frameworks can possess a significant amount of dogma that will have a gravity all its own when it comes to evolving into the future.
- Cloud Platforms - Amazon, Google, and Microsoft (Azure) have a strong voice in how software architecture is defined in the current climate, providing us with the services and tooling to govern the lifecycle. This control over how we define our infrastructure is only going to increase with their market dominance in the digital realm.
- Continuous Integration / Deployment - CI/CD services and tooling have established a new way of defining software architecture, and establishing a pipeline approach to moving it forward, building in the hooks needed to govern every step of its evolution. Reducing the cycles from annual down to monthly, weekly, and even daily cycles of change.
- Source Control - Github, Gitlab, and Bitbucket are defining how software is delivered, providing the vehicle for moving code forward, the hooks for governing each commit, and step forward any infrastructure makes as it is versioned, and evolved.
These areas are increasingly governing how we design, develop, deploy, and manage our infrastructure. Providing us with the scaffolding we need to hang our technological infrastructure on, and gives us the knobs and levers we can pull to consistently orchestrate, and move forward increasingly complex and large enterprise software infrastructure, across many teams, and geographic regions. The decisions we make around the technology we use will stick with us for years, and continue to influence decisions even after it is gone.
When it comes to delivering software architecture, not everything is governed by the technical components, and much of what gets delivered and moved forward will defined by the business side of the equation. The amount of investment by a business into its overall IT, as well as more progressive groups, will determine what gets done, and what doesn’t. With the following elements of the business governing software architecture in several cases::
- Budgets - How much money is allocated for a team to work when it comes to defining, deploying, managing, and iterating upon software architecture.
- Investors - Many groups are influenced, driven, and even restricted by outside investors, determining what software architecture is prioritized, and even dictating the decisions around what is put to work.
- Partners - External partners with strong influence over the business discussions that drive software infrastructure decision play a big role in the governance, or lack of governance involved.
- Public Image - Often times the decisions that go into software architecture, and the governance of how it moves forward will be driven by the public image concerns around company, and its stakeholders.
- Culture - The culture of a business will drive decisions being made when it comes to developing, managing, and governing software architecture, which can be more challenging to move forward than the technology in many cases.
The governance of software architecture has to be in alignment with business objectives of an enterprise. Many groups choose to begin their API journeys based upon trends, or the desire of a small group, and have encountered significant friction when trying to bring in alignment with the wider enterprise business objectives. Groups that addressed business considerations earlier on in their strategy have done much better when it came to reducing friction, and eliminating obstacles from their road map.
Almost every discussion we’ve had around governance of software infrastructure has included mentions of the importance of observability across next generation iterations. Software designed, delivered, and supported in the darkness or in isolation either fails, or is destined to become the next generation of technical debt. There were several areas of emphasis when it came to ensuring the API driven infrastructure sees the light of day from day one, and continues to operate in a way that everyone involved can see what is happening.
- Out in Open - Groups who operate out in the open, sharing their progress actively with other teams, and encouraging a transparent process find higher levels of success, adoption, and consistency across their architectural efforts.
- Co-Discovery - Ensuring that before work begins, teams are working together to discovery new ideas, learn about alternative software solutions, and working together to create buy-in, and ultimately make decisions around what gets adopted.
- Collaborative - While identified as sometimes being slower than traditional, more isolated efforts, teams who encouraged cross-team collaboration saw that their architectural decisions were sounder, more stable, and had more longevity.
- Open Source - Following open source software development practices, and working with existing open source solutions helps ensure that enterprise software architecture last longer, has more support, and follows common standards over other more proprietary approaches.
- Publicly - When it makes sense from a privacy and security standpoint, groups often articulate that being public by default helps ensure project teams behave differently, enjoy more accountability, and often attract external talent, domain expertise, and publicly opinion along the way.
Enterprise organizations that push for observability by default find that teams tend to work better together, and have a more open attitude. Attracting the right personalities, encouraging regular communication, and thinking externally by default, not as something that happens down the road. Bringing much needed sunlight and observability into processes that can often be very complex and abstract, and pushing things to speak to a wider audience beyond developer and IT groups.
Having a shared process that can be communicated across teams, going beyond just technical teams, and is something that business groups, partners, 3rd party, and all other stakeholders can follow and participate in, is a regular part of newer, API-centric, software delivery life cycles. Possessing several core elements that help ensure the process for defining, designing, delivering and evolving software architecture is shared by all.
- Contract - Crafting, maintaining, and consistently applying a common machine readable contract that is available in YAML format, is a common approach to ensuring there is a contract that can be used across all architectural projects, defining each solutions as an independent, business service.
- Pipeline - Extending the machine readable service contracts with YAML defined pipeline definitions that ensure all software is delivered in a consistent, reproducible manner across many disparate teams.
- Versioning - Establishing a common approach to versioning code, definitions, and other artifacts, providing a common semantic approach to governing how architecture is evolved in a shared manner that everyone can follow.
Historically the software development and operation lifecycle is owned by IT, and development groups. Modern approaches to delivering software at scale is a shared process, including internal business and technical stakeholders, while also sharing the process with external partners, 3rd party developers, and the public. Bringing software architecture out of the shadows, and conducting it on the open web, making it more inclusive amongst all stakeholders, but done in a way that respects privacy and security along the way.
Identifying Potential APIs
Once the architectural foundations have been laid, there are many ways in which large enterprises begin identifying the potential APIs that should be designed, deployed, and evolved supporting the many applications that will be depending on the underlying platform architecture. Depending on the organization and it’s priorities, the reasons for how new APIs are born will vary, resulting in different lifecycles, and resulting services being delivered across internal groups, partner stakeholders, and 3rd party developers.
Throughout this research, we identified that there is no single approach to identifying which APIs should be delivered, but we did work to understand a variety of approaches in use across the landscape, and by the practitioners interviewed for this project. Establishing some common areas for answering the questions around what should be an API, why are we doing APIs, leading us to the how of doing APIs, and uncovering the pragmatic reasoning behind web, mobile, device, and other applications that APIs are driving across the landscape.
Our existing realities drive the need for APIs, and reflect where we should be looking to provide new services for internal stakeholders, partners, and potentially as new revenue streams for 3rd party developers. While some APIs may be entirely new solutions, it is most likely that APIs will be born out of the realities we are already dealing with on a daily basis, based upon the digital solutions we depend on each day. We identified the most common realities that enterprise group face when it comes to their present day digital transformation challenges.
- Database - The existing databases in operation are the number one place groups are identifying potential resources for deployment of APIs. Exposing historically accumulated digital assets using the web, and making them available for use in new applications, to partners, and driving new types of data products for generating the next generation of revenue streams.
- Website - Our existing websites reflect the last 20 years of the digital evolution of our enterprises, representing the digital resources we’ve accumulated and identified as being important for sharing with partners and the public. HTML representations of our digital assets are always the 101 of API deployment, and understand what should also be available as JSON, and other more sophisticated representations.
- Integrations - Existing software integrations, system to system integrations, are represent a rich area for understanding how digital resources are delivered, accessed, and made available throughout existing applications and systems. Providing an important landscape for mapping out when understand what should be turned into APIs, having a more consistent process applied, eliminating custom integrations, and standardizing how systems speak with one another.
- Applications - Exiting web, mobile, desktop and other applications can have a variety of backend system connectivity solutions, but also might have a more custom, bespoke approach to doing APIs that is off the radar when it comes to governing how infrastructure evolves. Providing another rich area for mapping out the connections behind the common applications in use, understanding the internal, partner, and 3rd party APIs and other connections they use.
- Services - APIs have been around for a while, and exist in a variety of formats. Legacy web services, RPC, FTP, and other API and messaging formats should be mapped out and included as part of the potential API evolutionary landscape. Taking existing services, and evolving them in a consistent manner with all other API-driven services, leveraging web technologies to consistently deliver and manage digital resources.
- Spreadsheets - The majority of business in the world still happens within the spreadsheet. These portable data stores are emailed, shared, and spread around the enterprise, and represent a rich source of information when it comes to understanding which resources should be published as APIs.
You can’t govern what isn’t mapped out and known. It becomes increasingly difficult to govern software infrastructure that exists across many open and proprietary formats, and delivered as custom one-off solutions. Governance begins with a known landscape, and the greatest impedance to functional governance across organizations are the unknowns. Not knowing a solution exists, or its architectural approach being something that isn’t part of the bigger picture, leaving it to be a lone actor, in a larger landscape of known services operating in concert.
Another reason for having an open, very public approach to selecting, delivering, and operating software infrastructure, is that it establishes a public presence, across web properties, social networks, and other platforms where enterprise organizations can build community. There are a number of ways to identify potential new API resources by just being public, engaging with the community, and establishing API delivery life cycles that involve having a public presence.
- User Requests - Actively soliciting web, mobile, and developer feedback, both internally and externally is a great way to learn about potentially new API resource opportunities. Leveraging existing users as a source of insight when it comes to what services would make applications better, and demonstrating the importance of investing in an API literate user base.
- Partner Requests - Actively working with partners, conducting regular meetings regarding digital assets and transformation, seeking feedback on what types of services would improve upon existing solutions, and strengthen partner relations. Investing in existing partners, and using them as a way to evolve the API road map, and increase their dependency on enterprise resources.
- Public Feedback - Engaging with the public via websites, social networks, forums, and at events, to understand what opportunities are out there when it comes to delivering new API resources. Tapping public awareness around specific topics, within particular domains, and considering suggestions for new APIs outside the enterprise firewall and the traditional box.
- Media Coverage - Tuning on to popular tech blogs, mainstream media, and other public outlets to study and understand what opportunities are emerging for the delivery of new API services. Tuning into popular trends when it comes what is happening with APIs, or business sectors that might not have caught up to some of the modern approaches to delivering APIs.
- Feedback Loops - Cultivating trusted feedback loops with existing users, social networks, and private messaging platforms. Investing in long term feedback loops that tap the knowledge and domain expertise of trusted professionals, who can bring fresh ideas to the table when it comes to which new APIs can benefit the enterprise.
- Negative Consequences - One significant thing to note with having a public presence, is that not everything will be positive. There are serious privacy, security, and safety concerns when operating your APIs out in the public, and there should be plenty of consideration for how to do it in a healthy way. Acknowledging that not everyone in the public domain will have the enterprise’s best interest in mind.
It isn’t easy soliciting feedback from the general public when it comes to determining the direction a platform road map should head. However, with some investment, curation, and cultivation, a more reliable source of insight regarding the direction of an API platform can be established. The API community across the public and private sector has grown significantly over the last decade, providing a wealth of knowledge talent that can be tapped, if you know where to look for it.
Moving beyond the “where to look for opportunities for APIs”, and moving into the why to find, as well as how to prioritize which resources should be turned into APIs, we hear a lot about investing in the overall improvement of the enterprise when it came to the motivations behind governance. Looking at many of the common incentives behind doing APIs, but also doing it in a more consistent and scalable way that supports the mission and forward motion of the enterprise.
- Optimization - Seeking to optimize how applications are delivered, and services provided across teams. Providing consistent services that can be used across many internal groups, across partners, and that will improve the lives of 3rd party developers.
- Common Patterns - Doing APIs, and pushing for governance to help identify common patterns across how software is designed, delivered, and managed. Working to extract the existing patterns in use, help standardize and establish the common patterns, and reenforce their usage across teams, and distributed groups.
- Reusability - Encouraging reusability is the number one improvement we heard from different groups, and see across the landscape. Governing how software is not just delivered, but maximized, reused–ensuring the enterprise is maximizing software spend, as well as the revenue from services it delivers.
- Acceleration - Investing in governance to help accelerate how applications are delivered, measuring, standardizing, and optimizing along the way to improve existing efforts, as well as new projects on the horizon. Increasing the speed of not just new services, but how the services are able to be put to work by developers and integrators.
- Efficiency - Setting into motion patterns and processes that increase overall efficiency around how services are delivered, and how they enable teams to deliver on new projects, applications, integrations. Allowing IT, developers, and business users to benefit from an API focus.
- Flexibility - Increasing the flexibility of how applications operate, and teams are able to work together and deliver across the enterprise. Encouraging the design and development of APIs that work together, and flexibly achieve organizational objectives.
Providing a set of criteria that can be used to help prioritize which APIs get identified for delivery, and for evolution and versioning. If API resources help deliver on any of these areas, their benefit to the enterprise is increased, and it should be bumped up the list when looking for API opportunities. Always looking for how the enterprise can be improved upon, while also understanding which specific resources should be targeted for wrapping, and exposing as simple web APIs that can be used for both internal and external use cases.
The API journey is always full of challenges, and these areas of friction should be identified, and incorporated into the criteria for identifying new API solutions, as well as determining which APIs should be invested in and evolved. While some challenges can be minimized and overcome, many can also cause unnecessary friction throughout the API roadmap, making challenges something to be considered when putting together any API release strategy.
- Education - What education is required when it comes to acquiring the resources behind any potential API, developing and standing up a proper API, as well as deploying, managing, and supporting an API. What challenges can be foreseen, and identified early on, helping weigh what investment in education, training, and learning along the API journey for any set of services.
- Maturity - Understanding early on, and putting together a plan on what maturity will look like for any service, acknowledging that every service will begin in a juvenile state, and take time to harden, mature, and become something that is dependable, reliable, and usable in a production environment.
- Isolation - Identifying resources that are being developed, maintained, and operated in isolation, and working to move them out into the mainstream. While also ensuring that any new services being developed avoid isolation, and are developed, evolved, and managed out in the open, ensuring that services never operate alone.
- Management - Including management in discussions around which resources should be developed into APIs, including leadership in all conversations involving the targeting, evolution, and even deprecation of API services. Ensuring that the prioritization of API development is always on the radar of management, and there is a general awareness regarding the delivery of services.
- Consistency - Realizing that while consistency is the goal, it may be an elusive, non-stop chase to actually realize consistency across teams. It should be a goal, but also realistically understanding that it won’t be easy to achieve, and while we want to strive for perfection, sometimes there will be good enough achieved for some services.
- Reusability - Similar to consistency, reusability is an obvious goal, and should be worked towards, but it will also be elusive, and not always reliably achieved over time. There might still be redundancy for some services, and overlapping aspects of delivering services, while some areas reusability will be achievable.
- Build It And They Will Come - There has been a significant amount of reflection regarding targeting, developing, and publishing APIs that were not needed based upon an “if you build it, they will come” mentality–where most often, nobody came, and the work was in vain.
Challenges are a fact of life in the delivery of software, and evolving complex systems using APIs. Identifying challenges should be a natural part of targeting resources for delivery as APIs. Challenges can increase fiction when delivering service, and should be carefully evaluated before tackling the development of any new services. It is easy to identify the potential of new APIs, but it takes a more seasoned eye to understand potential challenges.
New ideas for APIs will become numerous once you begin looking. Based upon existing resources, applications, and the feedback of internal groups, partners, and the public. Along with all of the possibilities that come along, a standardized, pragmatic approach to understanding the potential, value, as well as challenges with each potential idea should be part of the equation.
Defining Data Models & Standards
To help realize and deliver upon governance at scale it will take heavy investment in standardizing data models, and incorporating existing patterns and standards throughout the API delivery lifecycle. Many enterprise API development groups are streamlining and standardization the delivery of APIs through the adoption and development of standards across operations, which is something that is also contributing to adoption, integration and removing friction for application developers.
The adoption of common data models, interfaces, media types, and web standards helps contribute to the delivery of consistent APIs at scale, but they can also prove to be a challenge for some teams, and even been seen as a threat by others. There are a number of ways in which teams are pushing for standardization across their operations, and helping achieve more consistency, reuse, and the desired results across operations. Reflecting one of the strengths of web APIs, in that they employ web standards to achieve wider adoption, and the delivery of valuable resources at web scale.
A suite of approaches have emerged in the last decade for designing, developing, evolving, and applying common API patterns across the API lifecycle. These standardized approaches to defining and delivering APIs, using common machine readable specifications, and widely used patterns, have become central to API governance discussions. Providing the fuel for the growth of the API sector to serve mobile applications, as well as the growth of other emerging channels like voice, bot automation, and the connecting of everyday objects to the net. Helping the enterprise get more organized about how services are delivered across the organization at scale.
- Resource-Defined - RESTful design patterns have provided a simple approach to taking corporate resources and defining them as an intuitive, reusable, potentially web-scale stack of API resources that can be used across a variety of applications. REST provides a philosophy that can be adopted across the enterprise to help organize digital resources as a reusable stack of resources that can be discovered and put to use across many channels.
- Schema-Driven - JSON Schema is being used to take a variety of schema and standardize them for use in RESTful API resource delivery. Providing a reusable blueprint that can be used across the request and response model for all APIs. Deriving, and standardizing existing schema in use, and making available for usage in in newly developed, and evolving APIs allows for teams to achieve many of the objectives set out as part of modern API strategies.
- Domain Driven - The business domain is used across the enterprise for guiding the identification, development, evolution, and standardization of a variety of API definitions in use across the enterprise. Lines of business, industry definitions, and a focus on the domain helps establish areas of concern, and the separation of services, allowing for the decoupling of enterprise resources used across systems, but working in unison to deliver a single set of business objectives.
- Legacy Abstraction - Continue movements to decouple, redefine, and evolve legacy systems is pushing forward the identification of common patterns, and pushing to map, transform, and give them new life as newer web APIs. Taking legacy databases, system interfaces, and distilling the wisdom that exists across them, to help drive the development of common standards.
- Vocabularies - API development teams are establishing common vocabularies based upon the standardized language already in use, but also essentially taking the slang that is used in bespoke systems and helping tame it, and add it to the common lexicon when it makes sense. Providing a standard language that can be used across the enterprise to talk about services, resources, and digital assets.
- Discovery - Many groups expressed challenges around standards not seeing the desired adoption because other teams could not find existing schema, definitions, and other existing standards. Emphasizing the importance of comprehensive, actively maintained, and evangelized catalog of core definitions across the enterprise. Providing a single, or distributed location where everyone can find and publish their common definitions.
The definitions coming out of existing API development efforts are being organized into catalog, and discovery systems that can be used to guide governance efforts. Mapping out the known landscape across the enterprise, and turning it into the common patterns that can be reused across the design, development, and operation of the next generation of APIs. Distilling down the essence of the enterprise so that it can become the building blocks of an API program, while also allowing each stop along the lifecycle to be quantified, measured, and considered as part of a wider governance strategy.
Doing Business On The Web
APis are built on the web. They use, and benefit from over 25 years of the evolution of the web. There are a number of elements to consider when working to identify and define common standards for use across the governance of any API program. While the API strategy should be rooted in definitions derived from the core of the enterprise, secondarily it should be embracing the web, and employing common patterns that make the web work to use as the foundation for delivering APIs.
- Web Standards - The web is the foundation for the delivery of APIs. Most APIs will use HTTP as a transport, and be employing URLs, HTTP verbs, headers, parameters, and other common web standards. Web standards should be part of any governance strategy to help establish common patterns and definitions for use across operations.
- Media Types - Media types are a fundamental part of the web, and help establish message formats that will be widely recognized outside the enterprise, encouraging the reuse and adoption of APIs that employ common media types. Allowing consumers to negotiate the format that makes the most sense to their team, and the types of applications they are looking to develop.
- Industry Schema - Industry level schema are emerging and maturing for use across API operations. Specifications like FHIR, PSD2, and other schema, along with API design patterns are evolving to help support industry focused API operations, while encouraging reuse and interoperability across disparate groups.
- Open Source - The usage of open source software, tooling, specifications, and processes are helping deliver on the API vision across the enterprise. Web APIs reflect the open source ethos, and plays well with the delivery of web APIs. Encouraging reuse, adoption, and bringing the observability necessary to help APIs succeed.
APIs are all about doing business on the web. The web provides the platform in which any API program will operate. When it comes to defining schema, standards, and common patterns for use across API operations, the web is always the beginning of the conversation. While enterprise defined patterns will always be front and center, the standards used to operate the web should always trump localized definitions, and be given priority whenever possible. Don’t reinvent the wheel when it comes to the web, always reuse and implement what is already known.
Under The Influence
When learning about new standards, and considering which standards to adopt, it can be easy to find yourself under the influence of specific vendors, competing standards, programming communities, and other factors. Careful evaluation of standards is important, and an awareness of what some of the common elements are that may shift your opinions one way or another, or even obfuscate what is real and prevent you from achieving objectives.
- Caught in Trends - Avoid getting caught up in the trend cycles that can often make it difficult to understand the hype around specific specifications. Do your research, understand best practices and adoption levels, and make the sensible decisions around the impact to your own efforts.
- External Entities - While engaging with external entities, understand what their priorities are when it comes to standards and specifications. Consider what affinity may exist between enterprise objectives, and any external entity that is engaged with, and makes sure there is the right alignment, and influences are pushing efforts in the right direction.
- Internal Demands - Similar to external entities, understand what the internal teams priorities are and don’t always assume internal requests will have the overall enterprise objectives in mind. Fully understanding what the awareness and motivation are around the implementation of specific standards, and how the fit into the overall strategy.
- Feedback Loops - Ensure that feedback looks are diverse, and provide a wealth of opinions around what types of standards and specifications should be supported, providing the widest possible view of the landscape when it comes to adoption and investment.
- Organic Change - Keeping an eye on vendor induced standards adoption over a more organic approach to the growth of standards, internally, as well as the outside community. Working to understand when a standard is artificially inflated, amplified, for alternative objectives beyond its core mission.
There are plenty of currents to get caught up in when it comes to identifying, defining, and evolving standards. Not all will bear fruit, or realize the type of adoption they need to be successful. Establishing a balanced view of the landscape across internal, and external actors, while keeping counsel with a diverse set of voices can help ensure you understand which API specifications, standards, and definitions will help move the enterprise forward.
Taking The Lead
While there are a number of ready to use standards available for the web, and organically grown out of the API community, these standards won’t always find their way into the enterprise. Leading organizations demonstrate that it takes a structured effort to define, disseminate, educate, and evolve standards across large organizations, with a number of proven tactics for taking the lead when it comes to standardizing API infrastructure across the enterprise.
- Workshops - Organizing, conducting, and growing the number of workshops held to introduce individuals across many teams to a variety of common standards and specifications.
- Discussions - Formalizing discussions around emerging standards, and those that are in use, to help push forward awareness, and adoption of standardized approaches across groups.
- Collaboration - Push teams to work together when it comes to sharing the standards in use, showcasing the investment they’ve made, and working together to understand the tooling, services, and standards being used.
- Event Storming - Putting event storming, a rapid, lightweight group modeling technique to help accelerate the identification, evolution, and adoption of standards that meet specific team’s needs.
- Influencers - Identifying, investing in, and cultivating influencers who exist within current groups, and encourage them to evangelize and help spread the good word about standards across the enterprise.
- Ask Questions - Always be asking questions about the standards, or lack of standards in use across the enterprise, pushing the conversation forward at all times when it comes to standards.
- Challenge Assumptions - Making sure teams don’t get complacent, and the status quo is always being challenged, and that the internal domain should always be rising to a higher level of standardization whenever possible.
It takes standards bodies to move forward common standards at the web and industry levels, and it takes the same approach to push forward the adoption and usage of standards within the enterprise. Leading enterprise organizations are able to quantify, measure and evolve the infrastructure in a more organized way through the adoption of common schemas, specifications, and standards. Providing a common vocabulary for all teams to use when designing, deploying, and managing services that can be used consistently across the enterprise, and its public interests.
Development to Production
After understanding the roles needed to realize governance, more about the underlying platform architecture that is needed, how organizations can identify where the API opportunities are, and making sure groups are putting standards to work, we scrutinized how groups are moving APIs from development to production in a more structured way. Governing how teams are efficiently moving APIs from idea and design, to actually putting services to work in a production environment at scale across large teams. Documenting the lifecycle of a service, and the common elements of how enterprises are getting the job done on a regular basis.
To be able to deliver APIs at scale in a consistent way teams are relying on a well honed, well defined lifecycle that has been defined, proven, and evolved by senior teams. Forcing structure and rigor throughout the evolution of all services, putting governance in front of teams, and forcing them to execute in a consistent way if they expect to reach a production reality with their services. Focusing on a handful of structured formats for imposing governance at the deployment levels.
- Contract - Requiring ALL services begin with a machine readable OpenAPI contract defining the entire surface area of the API and its schema. Leveraging the contract as the central truth for what the service will deliver, and how governance will be measured throughout the lifecycle of the service.
- Process - Providing a well defined process for all developers laying out how any service moves from design to production, with as much detail regarding each step along the way. Helping all developers understand what they will face as they work to move their services forward in the enterprise.
- Scorecard - Having a machine readable checklist and scorecard, with tooling to help each developer fork or establish an instance for their service. Providing a checklist of everything they need to consider, that allows them to check off what has been done, what is left to be done, and providing a definition that can be used to define and report upon governance along the way.
- Cycles - Provide a variety of cycles that every service will need to go to before they will be production worthy, forcing developers to iterate upon their services, harden and mature them before they will be considered ready for production.
- Reviews - Require all services go through a series of lifecycle reviews by other teams, pushing service owners to present their work to each review team, and work with them to satisfy any concerns, and make sure it meets all governance criteria.
- Clinics - Providing developers with a variety of clinics where they can receive feedback on their work, improve upon their service, and improve the health of their work before submitting it for inclusion in a production environment.
Enterprise organizations that provide structure for API development teams find it much easier to realize their governance aspirations. The scaffolding is already there to think about the consistency of services, and the face to face, and virtual scrutiny of services helps provide the environment for governance practices to be executed, enforced, and evolved before any service reaches a production state. A well defined API deployment lifecycle will help contribute to a well defined API governance practice.
One sign of enterprise groups who are further along in their governance journeys is when there are virtualized environments being put to use. Requiring all API developers to mock and iterate upon their APIs in a virtualized environment, presenting them as if they are real, before they ever are given a license to write any code, let alone reach a production state for their services.
- Mocking - Creating mock APIs for all endpoints, virtualizing every aspect of the surface area of an API, allowing a service to be iterated upon early on in its lifecycle.
- Data - Requiring virtualized and synthesized data be present for all mocked APIs, returning realistic data with responses, reflecting behavior encountered in a production environment.
- Sandbox - Providing a complete labs and sandbox environment for developers to publish their mocked APIs into, reflecting what they’ll encounter in a production environment, but done in a much more safer and secure way.
Virtualized environments provide an important phase in the journey for APIs moving from concept to reality. Establishing a safe environment for developers to iterate upon their work, encounter many of the challenges they’ll face in a public environment, without any of the potential for harm to users or the platform. Ensuring that when a service is ready for development, most of the rough edges have been worked out of the service contract, and for the team behind.
One of the most significant ways in which we’ve found enterprise groups governing the evolution of their APIs is through the technology they employ. This technology is providing much of the structure and discipline that organizations are depending on to help ensure that APIs are being developed, and ultimately deployed in a consistent manner. Bringing most of the governance to the table for some organizations who haven’t begun moving their governance strategy forward as a formal approach.
- Authentication - Requiring standard-based approaches to authentication using Basic Auth, API keys, OAuth, and JWT. Ensuring the teams understand when to use which protocol, and how to properly configure, and use as part of larger API governance strategy.
- Framework - Relying on the programming frameworks in use to inject discipline into the process, dictating the governance of how APIs are delivered before they are ready for a production environment.
- Gateway - Applying the policies and structure necessary to govern API services as they are made available in a production environment. Many groups also had a sandbox or development edition of their gateway emulating many of the same elements that will be found in a production world.
- Management - Similar to the gateway, groups are relying on their API management layers to help govern what APIs do, providing transformations, policy templates, and a wide variety of other standardization that occurs prior to being made available in production sense.
- Vendor - The reliance on technology to deliver governance at the API deployment level gives a lot of control to vendors when it comes to governing the API lifecycle. If a vendor doesn’t provide a specific way of doing things, it may not exist within some groups. Dictating what is governance for many enterprise groups.
- Tooling - Most groups have an arsenal of open source, and custom developed tooling for helping push code from development to production, validating, scanning, transforming, shaping, and hardening code and interfaces to be ready for production usage.
- Encryption - Requiring encryption by default for storage, and in transport, using technology to ensure security is a default parameter for everything that is exposed publicly. Reducing the possibility of a breach, and minimizing the damage when one does occur.
Demonstrating how important the technological choices we make, and the architectural planning we discussed earlier is to the overall API governance conversations. The services, tooling, and applications we adopt will either contribute, or will not contribute to our governance practices. Potentially enforcing governance for all APIs as they move from development to production, in a way that teams cannot circumvent, and often times don’t even notice is occurring behind the scenes.
Augmenting the core technology, there are a number of orchestration practices we found that help quantify and enforce governance on the road from development to production. Dictating how code, artifacts, and other elements included as part of the API lifecycle move forward, evolve, or possibly get held back until specific requirements are met to meet wider governance criteria.
- Pipeline - CI/CD services, tooling, and mindset have introduced a pipeline workflow for many teams, standardizing the API delivery process as an executable, repeatable, measurable pipeline that can be realized across any team.
- Stages - The defining of clear stages that exist after development, but before production, requiring quality assurance, security reviews, compliance audits, and other relevant governance practices to be realized.
- Hooks - Well defined pre and post commit hooks for all service repositories, requiring that governance is applied throughout a service’s pipeline, and are default for all services, no matter which organization they emerge from.
- Devops - Pushing that all teams are competent, and skilled enough to execute on behalf of their services from beginning to end, owning and executing at every stage of the life cycle. Reducing the need for reliance on special teams, and eliminating bottlenecks.
- Logging - Identifying the logging capabilities of the entire stack for each service being delivered. Making sure logging is turned on for everything, and all logs are shipped to a central location for auditing, and when possible real time analysis and response.
Orchestration provides some clarity on the automation side of moving services from development to production, while also enforcing governance along the way. Allowing for an assembly line delivery of consistent services, and the iteration of each version, in alignment with the overall governance strategy. Reducing the chance for human error, and increasing the chance for consistent execution of the enterprise API strategy at scale across many different teams.
The Legal Department
Beyond the technology, the legal department should have a significant influence over APIs going from development to production. Providing a structured framework that can generally apply across all services easily, but also providing granular level control over the fine tuning of legal documents for specific services and use cases. With a handful of clear building blocks in use to help govern the delivery of APIs from the legal side of the equation.
- Terms of Services - Have universally applicable, and modular, as well as possibly machine readable, and human readable terms of service governing all services from a central location.
- Security Policy - Provide a comprehensive security policy that governs how services are secured, reflecting the technologies, checklists, tooling, and reviews that are in use by all team members, providing an overview for all stakeholders to understand.
- Licensing - Establish clear code, data, and interface licensing to be used across the entire API stack, allowing developers to properly license their services, as well as understand the provenance for the systems and services they depend on.
- Server Level Agreements - Have universally applicable, modular, as well as possibly machine readable, and human readable service level agreement (SLA) that can be applied across all services, measured, and reported upon as part of wider governance strategy.
- Scopes - Define and publish the OAuth access scopes as part of a formal set of legal guidance regarding what data is accessible via services, and the role users and developers play in managing scope that is defined by the platform.
The legal department will play an important role in governing APIs as they move from development to production, and there needs to be clear guidance for all developers regarding what is required. Similar to the technical, and business elements of delivering services, the legal components need to be easy to access, understand, and apply, but also make sure and protect the interests of everyone involved with the delivery and operation of enterprise services.
Making APIs Available to Consumers
The next step in the life cycle of properly governed APIs is making them available to consumer after they’ve been published to a production environment. The governing of APIs is not limited to the technical side of things, and this is where we begin understanding how to consistently deliver, measure, and understand the impact of API resources across the many consumers who are integrating the valuable resources into their applications. Shining a light on the business and politics of how digital assets are being put to use across the enterprise.
This portion of this governance research is intended to provide a basic list of the building blocks used by enterprise groups to help reduce friction when putting APIs to work, but also make sense of how consumers are using API resources, establishing a feedback loop that guide the road map for the future of the platform. Taking us back to the beginning of this research and informing how we should be targeting the development of new APIs, the evolution of existing services, and in many cases the deprecation and archiving of services. Ensuring governance goes well beyond the technical details, and making sure they are benefitting the platform, as well as consumers.
Making your APIs available to consumers requires doing a lot of research on who you are marketing them to, and positioning yourself to speak to an intended audience. Tailoring not just the design of your APIs, but the overall presentation, messaging, and even portal, documentation, and other building blocks to speak to a particular audience. For many API providers, APIs might be made available to multiple audience, in a variety of ways, based upon knowing their customers, and presenting exactly the resources they are needing to get a specific job done.
- Studies - Conducting regular stories about what internal, partner, and public user groups are using, and needing when it comes to developing applications, and integrating systems.
- Landscape - Establishing an understanding of the industry landscape for the area being targeted by services, and regularly tuning into and refining the understanding of what consumers are using across that landscape.
- B2B - Positioning the API to speak to a B2B audience, providing separate portal, documentation, and other resources to cater to a business audience.
- B2C - Positioning the API to speak to a B2C audience, providing separate portal, documentation, and other resources to cater to a consumer audience.
- Partners - Providing a unique set of resources that speak to partner groups, providing separate portal, documentation, and other resources to cater to exactly what existing partners will be needing.
- Internal - Positioning the API to speak to internal groups, providing separate portal, documentation, and other resources to cater to the needs of development groups within the enterprise.
- Context - Making sure services have knowledge of the context in which they will be delivering resources into. Different patterns, processes, and practices work well within different context, while others will fail, depending on the context that is relevant to the consumer.
- Office Hours - Holding conference call, or virtual office hours for different consumer groups to be available for discussion around what APIs are available, their supporting resources, as well as contributing to the platform road map.
Knowing your existing, and potential API consumers is essential to position your API program to speak to its intended targets. It is difficult to design and present the right set of resources for an audience you do not understand. Demonstrating how knowing your consumers is something that should happen before you begin the development of services, as well as an ongoing features of a platform, and that understanding the challenges of your consumer, and shifting the road map to stay in alignment with your consumer audience is critical to realizing platform-wide governance
While studying the consumer outreach strategies of leading API providers, as well as the ones that were interviewed, there are common patterns at play defining how to best reach your audience. The consistency of governed APIs speaks to how to best reach a wide audience, helps increase the impact APIs will have in the applications they are use in, and reduce overall friction and support required to operate them. There are several common patterns present when looking at how organizations are presenting APIs to their consumers publicly and privately.
- API Design - The design of APIs, using common RESTful resource design patterns, helps present simple, intuitive, and familiar resources that speak to a wide as possible audience.
- Developer Portals - Consistently designed, easy to navigate, well branded portals provide a familiar, known destination in which consumers can discover, onboard, and stay in touch with where an API platform is going.
- Documentation - Using common open source documentation across APIs provides an interactive, hands-on way for developers to learn about APIs, understand how to integrate with them, and be able to regularly check back in for added features and benefits that they provide.
- Definitions - Providing machine readable OpenAPI, and Postman Collections for consumers gives them a portable definition of what an API does, which they can use in their client tooling, to generate code libraries, setup monitors, tests, and generally understand what is possible.
Common design and presentation patterns is one of the reasons many of the leading API providers have established their foothold with their consumers. When you study the approach of Amazon, Google, Twitter, Twilio, Stripe, and other leading API providers, you see that they all use consistent design patterns, as well as provide similar portals, documentation, and other resources for their consumers. Governing the presentation layer for their API driven services, which reflects the consistency consumers are used to when working with multiple API providers across even different business sectors.
The next aspect of presenting production APIs to consumers involves communication, and ensuring that all stakeholders are kept up to speed on what is happening. Keeping a steady stream of information flowing around the platform, blending it with, and encouraging the activity on feedback loops, with an intent to drive the platform road map.
- Updates - Providing updates on a blog, or other mechanism, helping keep consumers up to date on what is going on with the platform, and making sure everyone knows there is someone home.
- Roadmaps - Publishing a public roadmap for API consumers to help them understand what is being planned, and what the future holds. Also maintaining private versions of road map for internal groups, and potentially partners.
- Issues - Being transparent and communicative around what the current known issues are with a platform, and publishing a list of anything current.
- Change Logs - Translated the road map, and issues into a change log for the platform showcasing what has historically occurred via a platform.
- Showcase - A published listing of applications and developers, showcasing the work being done by API consumers, highlighting the value a platform brings to the table.
Maintaining a steady stream of communication around what is happening with an API platform is a clear signal coming from the strongest enterprise API platforms out there. You see regular communication around what is happening, and what is being worked on, with teams reach out to each other, and sharing healthy practices, challenges, and showcasing what is being done with their API resources.
Realizing API Governance
Everything covered so far in this document feeds into what should be considered as part of the overall governance of an API platform, but focuses on the actual delivery of APIs. This is the section where we look at what is needed specifically for governance, and what teams are doing to invest in the governance of APIs across their teams, projects, and the lifecycle of their operations. There are a number of areas we identified that were relevant for groups who are actively realizing governance across their operations.
One key component of API governance at enterprise organizations who have been doing it a while, and have made significant investment in their efforts, is the presence of organized structure and teams dedicated to advancing governance across the enterprise. While these organizational structures are often defined by many different names, they have some common elements worth noting.
- Organization - Establishing a formal organization within the enterprise that is dedicated to API infrastructure, and developing a structured approach to governance, and the shared strategy across all teams.
- Core Team - Beginning with a small, focused, core team within the API strategy and governance, then expanding and growing as it makes sense, and based upon the expansion across the larger enterprise.
- Enablement Team - Providing an enablement team that can go out and work with individual teams to help enable them to realize healthier API lifecycle practices, and achieve governance objectives.
- Advisory Board - Developing an advisory board of internal, and possibly external individuals who can provide regular feedback on the API strategy, and help move forward the governance conversation.
- Legacy Teams - Involving legacy teams in the central API strategy and governance team to make sure the legacy of the enterprise is reflected and understood as efforts evolve and move forward
There were many variations in how enterprise organizations are organizing their API teams, some with more of a centralized approach, with others possessing a decentralized, and more organic feel. Some come straight out of CIO and CTO groups, where others were more bottom-up, organically grown efforts, reflecting the tone of the conversation occurring at different types of enterprise organizations.
While there were a number of approaches used to organize and execute on the API governance vision across different enterprise, there were some common approaches, and advice regarding how to do it in a pragmatic way. Providing some key elements to consider as organizations think about forming their own strategy, and putting it into motion at their own enterprise organizations.
- Start Simple - Keeping things as simple as possible when getting going. Not trying to bite off more than you can chew, and overpromising to the organization. Start with the basics, get involvement, and buy-in, then move forward in a logical fashion.
- Not Heavy Handed - Refrain from being heavy handed with governance policing and enforcement. It is repeatedly stated that heavy handed efforts get an overwhelming amount of pushback, and can set back efforts significantly.
- Inline Defined - Provide guidance, education, and artifacts inline. Do not expect people to read governance guides, and understand what is going on from the start. Feed them information on a regular basis through the channels they are already tuned into include corporate communication channels, their integrated development environments (IDE), and other existing entry points people will stay tuned on.
- Having A Mandate - Think deeply about having a mandate regarding governance. Some people think it is better stated as a mission, rather than a mandate. Considering the negative impact a mandate might have when it comes to adoption, and participation.
- Select Enforcement - Be creative in how you enforce governance across operations. Be very selective about where you enforce and push back on users. Finding inspiring, and motivational ways to enforce governance, being more carrot than stick.
- Build Community - Working to build community around the API strategy and governance organization, building relationships across teams, recruiting advocates, and working to train and education on a regular basis.
- Evangelism - Spending a significant amount of time reaching out to internal stakeholders, partner contacts, also the public when it makes sense, but most importantly, always be evangelizing to management and leadership.
Moving beyond just a group of people in name only and having an structured and planned approach to executing on the API governance across an organization on a daily, weekly, monthly, quarterly, and annual basis. Establishing a deliberate tone to the API governance effort, measuring it’s impact across groups, and adjusting and evolving as required. Developing a strong voice, and measured approach over time, while understanding what works, and what doesn’t work-being agile and flexible in how APIs are governed.
Building upon the technological layers present in every previous section of this report, we wanted to take another look at how technology is used specifically for defining, measuring, and reporting on governance efforts. Tracking on the specific technological solutions that enterprise groups are using to understand, as well as enforce the governance strategy on the ground, in real time.
- Definitions - Use OpenAPI, Postman Collections, JSON Schema, and other machine readable artifacts available for all stages of the API lifecycle to quantify, measure, and report on how well governance efforts are being realized.
- Management - The API management layer provides a number of features that help apply policies, rate limit, log, and track on what is happening with all APIs. Primarily used to understand consumer behavior, but can also be used to understand provider, publisher, and developer behavior as well.
- Gateway - Providing the single point of entry for all services, allowing for transformation, translation, as well as all the features brought to the table by API management solutions. Providing the perfect opportunity to enforce, as well as measure how well governance is being applied across the organization.
- Logging - Logs shipped centrally will be the most important way that governance efforts measures and reports on what is happening across the enterprise platform. Without a central logging strategy, governance will be flying blind, unable to see into all the services it is supposed to be governing.
- Monitoring - Making the monitoring of ALL services the default. Tracking all services from multiple regions, and understanding if they are meeting internal or external SLAs. Providing a key benchmark for whether governance is being effective across services.
- Testing - Getting much more granular and making sure that APIs are doing what they should. Taking plain business assertions, and testing them against APIs using machine-defined tests that can be executed in real-time, and on a schedule.
- Security - Gathering as much data as possible about how security is being handled, and the results of scanning, monitoring, logging, and authentication around security checkpoints.
- Reporting - Leverage management, gateway, monitoring, testing and other technology to produce reports on how well governance benchmarks are being met. Allowing the technology to do the measurement and enforcement, as well as reporting of the numbers that can be aggregated into a single set of reports to understand the impact of governance efforts.
When making decision on what technology to use as part of the delivery of API infrastructure, it’s role in the wider governance strategy should be considered. Having services and tooling inline, that can help execute and report upon governance efforts is an important aspect of being able to move forward a governance program across the enterprise. Built in governance is much more likely to be leveraged, than externally mandated tracking and reporting.
Every organization we talked to shared their frustrations and stories around the challenges they’ve faced. Many of these have been shared as part of specific section above, but we wanted to focus on the challenges with actually implementing governance itself, and look at some of the solutions groups have found for working around challenges and roadblocks.
- Co-Creation - Isolated API governance organizations stayed isolated, and groups who co-created the strategy with other teams, and worked to share ownership over the strategy, execution, and road map had much better success meeting their objectives.
- Buy-In - Getting buy-in from teams is difficult, and many have spoke of the challenges getting buy-in to the need for a centralized, or even distributed governance approach. Making it difficult to move forward when you don’t have the buy-in of some groups or management.
- Standards - While many agreed standards were good, actually getting people on the ground to adopt, use, and realize the benefits of standards across all stops along the API lifecycle has proven elusive.
- Artifacts - There just wasn’t agreement that definitive governance artifacts like guides, prototypes, and other common solutions were necessary. Some teams just disregard these artifacts, questioning the investment of time and energy to create. While others felt strongly that they were necessary to lead through example.
- Difficult Process - Over and over, API teams express that governance, and pushing for consistency across the API lifecycle was a difficult process. It sounds easy when you plan the strategy, but actually doing it on the ground never works out as you envision.
- Refine Process - You will be constantly refining your governance strategy, adjusting, removing, tweaking, and shifting the approach until you find solutions to incremental aspects of delivering APIs.
- Takes Time - All of this will take time. Be patient. Play the long game. Understand it will take much longer than you expected to see the change you envision.
There will be more challenges along the way than there will be wins, when it comes to governance of vast, complex API infrastructure. Challenges, roadblocks, and friction will exist at all stages of standardizing how APIs are delivered across the enterprise. Dealing with failure, and recognizing challenges and the potential for them to be a roadblock is important to being able to keep moving forward at any pace.
The Road To API Governance
There has been a significant uptick in the number of companies, organizations, institutions, and government agencies doing APIs since 2010, to meet the demands of web, mobile, and device applications. A very small percentage of these entities have any sort of formal governance strategy in motion to address how APIs will be delivered across their organizations. Most API providers are living in the moment, realizing they need to be addressing governance, but struggling to overcome a handful of common roadblocks.
- People - A lack of awareness, training, and communication amongst stakeholders is the biggest challenge API governance efforts face. Do not underestimate the people when crafting a technology focused effort, otherwise the people variable will be what brings it down.
- Culture - Plan for how the governance will address the culture within an organization. This is where the studies, outreach, workshops, and planning will come into play. Plan for everything taking 5 to 10 times longer than you anticipate because of the thickness, and resistance of organizational culture.
- Problems - Count on problems coming up everywhere. Dedicate a significant amount of time and resources to identifying, thinking through, and address problems that come up. Do not let problems fester, go ignored, or address.
- Existing - Map API governance efforts to the existing realities. Yes, the objective is to move the delivery of APIs to a specific destination, but the strategy needs to be rooted in what is existing, building a bridge to where we want to be.
Not all organizations will be ready for capital “G” governance, and many will have to accept inline, ongoing, lower case “g” governance. Doing what they can, with what resources they have, evangelizing, building community, and consensus along the way. While an organized, centralized, well funded governance program is ideal, and can achieve a lot, a significant amount can be done with a scrappier approach, until more traction and resources are achieved.
This report pulls together several years of research, combined with a handful of interviews with API professionals who are pushing forward the API governance conversation at their enterprise organizations. It acknowledges that the discipline of API governance is more discussion, than it is a formal discipline as of 2018. There are many ways in which API providers are governing their APIs, but few that have a formalized API governance strategy and program, and even fewer that are sharing their strategy, or lack of one in a public manner.
The objective with this report is to pull together as much information regarding how organizations are governing their APIs, and assemble the findings in the following logical order, reflecting how an organization might approach governance on the ground:
- Roles Within An Organization - Who is needed to make this happen?
- Design Software Architecture - Laying the foundation for governance.
- Identifying Potential APIs - Defining the right resources to expose.
- Defining Data Models & Standards - working to standardize how things are done.
- Development to Production - Moving from idea to reality in a standard way.
- Making APIs Available to Consumers - Exposing resources properly to consumers.
- Realizing API Governance - Moving towards a structured vision of governance.
- The Road To API Governance - Acknowledging governance is more vision than reality.
Not every detail in this report will apply to the VA, or any other enterprise organization looking to establish a wider API governance strategy. It is meant to be educational, enlightening, and show the scope of how enterprise groups are addressing governance. Allowing enterprise API efforts to learn from each other, and hopefully even share more stories regarding the challenges they face, and the success they are finding–no matter how small.
Hopefully this report reflects a patchwork of things that should be considered, rather than a complete list of what has to be done. There is no such thing as the perfect governance strategy for any API program. There is however, a great deal of things that can be done when you have the right team, the right amount of enthusiasm, and a positive outlook on what governance means. Addressing early on some of the negative perceptions that will exist out there about governance, and how it is something that comes from the top, and how it has the potential to not give regular people at the front lines a voice in the process–this is a myth, it doesn’t have to be the reality.
A definition for governance from the Oxford English Dictionary is, “the way in which an organization is managed at the highest level, and the systems for doing this”. Don’t mistake the highest level being about the highest levels of management, and let it be more about the highest levels of strategy across the organization. It is the system for governing a complex machine of API driven gears that make systems and applications work across the enterprise. It is the governance of a machine that has the potential to allow every individual within the enterprise to play an important role in influencing, allowing everyone to contribute, even if they do not work in a technical capacity within the enterprise machine.