State of APIs In The Federal Government
27 Jul 2017
This is my talk from Washington DC with Steve Willmott of 3Scale by Red Hat about transforming enterprise IT with containers, APIs, and integration, where I assess the current state of APIs in the federal government, and the opportunity in the private sector when it comes to working with government data.
My name is Kin Lane. I am the API Evangelist. I have studied the technology, business, and politics of Application Programming Interfaces, or more commonly known as APIs, full time since 2010. I spend time looking through the growing number of APIs available today, as well as the evolving group of service providers selling their solutions to API providers. I take what I learn across the space and publish as over 80 independent research projects that I run on Github, covering a growing number of stops along the API lifecycle.
In 2011, I began studying and writing about federal government APIs. I have long had an interest in politics, and how our government works (or doesn’t work), which was in alignment with thinking about how I could take what I’ve learned about APIs and apply to the federal government. By 2013, my research and storytelling about APIs attracted the attention of folks in government, which led to an invitation to come work on open data and API projects at multiple agencies. This move took my work to new levels, opening up some interesting doors that have opened my eyes to the scope of APIs within federal government.
In the summer of 2013 I was invited to be part of the round two Presidential Innovation Fellowship, and work at the Department of Veterans affairs doing web service and API inventory, as well as assist with the wider open data efforts of the Obama administration. I worked in DC until the government shutdown in October, when I decided to leave my position so that I could continue doing my work around veterans data, and continue meeting my obligations as the API Evangelist.
My time as a Presidential Innovation Fellow (PIF) gave me access to a number of interesting folks, doing interesting things at a variety of federal agencies, including now 18F and USDS. During my work as the API Evangelist I have spent a lot of time expanding on these relationships by helping craft API strategies, telling stories publicly on my blog, and speaking on conference calls and on-site when it makes sense. While the PIF program wasn’t a good fit for my career, the opportunity forever changed how I see government, and how I see APIs and open data making an impact at this scale.
After I left DC, I began working on a project I called Adopt a Federal Agency Dataset, where anyone could choose a single dataset published by a federal agency, fork and “adopt” the dataset using Github. The goal is to get civic minded folks spending time improving the quality of data, converting to another format, or even publishing a simple API using the open data. I got to work building a prototype, and talking with a variety folks about the projects viability.
Eventually my work paid off, and I attracted the attention of the Knight Foundation, and received $35K USD to develop a prototype template for the project. Over the course of the next year I developed a website for the project, a forkable blueprint for adopting datasets that runs 100% on Github, and applied the template to a handful of open data projects including the White House budget, and Department of Education data. I have continued to support and evolve the work, regularly adopting federal agency data sets, and working to develop common schema and API definitions for use across government of all shapes and sizes.
APIs In The Federal Government
Over four years of work, including my fellowship in Washington DC, I have definitely fired up my passion for understanding how government works (or doesn’t). I’ve continued to keep an eye on open data and API efforts across federal agencies, and regularly meeting and talking with folks who are doing things with APIs in government. I spent at least 2-3 hours a week, and sometimes more on playing with government APIs, and studying their approach to understand where I can help move the conversation forward.
I feel like I could just dedicated my career as the API Evangelist to government APIs, but I find what I learn in the private sector is extremely useful in the public sector, and vice versa. There is a symbiotic relationship between the viability of APIs in the federal government, and the viability of APIs at higher educational institutions, the enterprise, small businesses and agencies, all the way to upstarts in Silicon Valley. This relationship has kept me studying the intersection of public and private sector API operations, and tracking on, and contributing where I can to the API momentum in DC.
96 /Developer Portals
As of this week the GSA has 96 developer portals for federal agencies present. A portion of these developer portals are simply data portals, a result of open data efforts in the last administration. There is another portion of them that you can tell do not have a complete strategy, and are more of a landing page for a couple of APIs, and some open data. However, there are some API developer portals present that have some high value datasets and APIs, with robust documentation, code samples, and all the other bells and whistles you’d expect of an API in the private sector.
Even with the inconsistencies in federal agency developer portals, all the top agencies now have an API portal presence, with a growing number of lesser agencies following their lead. This is significant. This goes beyond just these federal agencies being more transparent and machine readable in how they operate. Many of the resources being made available also have the potential to impact markets, and are powering businesses of all shapes and sizes. This momentum is setting the stage for a future where ALL federal agencies have and open data and API portal that businesses, researchers, and developers know they can find the latest data, content, and algorithms across federal agencies.
As of this week the GSA has over 400 APIs listed across the 96 developer programs found across federal agencies. There are numerous consistency and design issues across these APIs, and many of them are simply downloads, which may have some form filter as a gatekeeper. While we don’t have any numbers to support what kind of growth we’ve seen in federal government APIs over the last five years, off the top of my head I’d say we’ve seen up to 50% growth each year in the number of APIs published by agencies.
In my experience APIs breed more APIs. When one group sees another group doing APIs, they often emulate what they see. Folks in the private sector have been emulating the API moves of pioneers like SalesForce, Amazon, Twitter, and Twilio for years, we are seeing this play out at the federal government level. Some agencies are emulating what they see in the private sector, but I’d wager a significant number of agencies will more likely emulate what they see out of other agencies. This is how we are going to take all of this to the next level, building on the hard work of USDS, 18F, and other individual agencies like Census, NASA, and others who are truly pushing the needle forward.
133 Github Orgs
The almost hundred developer portals, and over 400 APIs gives me hope about APIs in the federal government, but one area that leaves me the most optimistic is the number of agencies who are operating on Github. Doing my time at the Department of Veterans of Affairs (VA), I was the one who setup the VA Github organization, which became a lifeline for every one of the API and open data projects I was working on. Github usage across these agencies goes beyond just managing code, and agencies are using it to define policy, provide documentation, solicit feedback, and much more.
This usage of Github sets the stage for the next wave of growth in API operations within federal governments where Github is a central actor at every stage of the API life cycle. Github adoption is the most important signal I use to understand the viability of any API effort in the private sector. The companies who are using Github effectively almost always also possess a strong position in the space, directly influenced by the way they use Github to manage code, schema, definitions, and other machine readable, reusable components of API operations. I’m predicting that the federal agencies using Github possess a similar advantage if the federal API ecosystem, in coming years.
18F and USDS
A significant amount of the API leadership I’m seeing across federal agencies has been stimulated by tech focused groups at the White House and GSA. U.S. Digital Service (USDS) and 18F are working with multiple agencies to deliver API driven solutions, and software development practices. These groups have played a critical role in bringing outside tech practices into the federal government and doing the hard work to begin applying, evolving, and hardening these practices across agencies.
USDS and 18F are doing great work, and while I’m not 100% up to speed on their road map, the last couple of conversations I’ve had, they are growing and expanding–looking to keep taking on new projects. I’d like to see similar models develop, but from the outside-in, beginning to shift the landscape when we think about government consulting and vendors. 18F has done an amazing job at not just rethinking the technology part of the equation, but also thinking about the business and politics of it all, something that I think should be replicated externally, creating a new realm of API opportunity outside the federal government firewall.
Transparency In Federal Government
Open data and APIs provide transparency into how government works (or doesn’t work). It gives us a look into the budgets of federal agencies, and better understand what government does, manages, and the impact it has on society. I fully support the Obama’s administration mandate requiring ALL federal agencies publish their data inventory as machine readable files, as well as APIs when possible. This is how government should operate by default, ensuring that agencies are more accountable to the public on a real-time basis.
I’m looking to continue the trend of transparency with data and APIs across federal government–doing more of what we’ve been doing for the last 5 years. However, in coming years I’d like to elevate this to become more about observability, where things are transparent, but there is also monitoring, testing, communication, service level agreements, and other terms of service that make government more observable by the public, as well as government auditors. This should be the default state of technology in government, where each agency is able to invest in the Venn diagram of operations that can become API observability.
Resources That Help The Private Sector
Publishing data and APIs isn’t all about making the federal government more transparent. Many of the API and data resources I come across have significant commercial value as well. The healthcare industry benefits from pharmaceutical data being open as part of OpenFDA are significant. Every citizen can benefit from the web, mobile, and automobile applications developed on top of the recreational information database (RIDB) which powers Recreation.gov, and the growing ecosystem on top of the RIDB API. I can keep going, talking about APIs at Commerce, NOAA, and Census–these APIs all have significant impact on the way business works.
There are many ways in which the federal government has been stepping up to provide valuable data for use by companies of all shapes and sizes. Census data is baked into many systems. Labor data is getting baked into the business models of startups. Regulations.gov API allow the rules of the road to get exposed for business consideration. This open data and API stuff at the federal government level is getting pretty important when it comes to how our economy and society works. While I do have some serious concern about the current administrations leadership when it comes to APIs I feel the current momentum around opening up data and APIs can be sustained, if approached in the right away.
Considering The API Lifecycle
Alright, that describes the momentum we have with APIs in the federal government. I want to step back and size up this momentum, and take a look at some of the deficiencies I’m seeing right now in federal government developer portal and API implementations. In my regular work as the API Evangelist I look at the entire API space through the lens of almost 85 stops along an API lifecycle, from definition to deprecation. I’m going to take a handful of the core stops and use them as a lens to look at the state of federal government APIs, helping me frame a discussion about how we might move this conversation forward.
I’m looking to see the federal government API space alongside how I look at every other API in the private sector. I’m not saying that everything that gets done by commercial API providers should be emulated by the federal government, I’m just looking to make sure both sides are learning from each other, and somewhat in sync. I’m looking to try and figure out how we can keep the momentum going that I referenced, but also make sure federal agencies are not falling behind in all the key areas commercial API providers are using to dial in their API operations.
Starting with the most important stop along the API lifecycle, I’m beginning to see more common web and API definitions being applied as part of federal government API operations. I’m always on the hunt for agencies using common web standards and specifications as part of their designing, using common definitions for dates, currency, and other key elements. I’m also finding more APIs defined using OpenAPI, API Blueprint, and other commonly used API definition specifications, providing a machine readable definition of the surface area of any government API.
Even with the definitions in play currently, it is still a small portion of the 400+ APIs I’m seeing. Most APIs are custom definitions, without much investment in recycling and reuse. The federal government needs a heavy dose of evangelism around common definitions, and the development of templates, and schema that agencies can leverage in their projects. This is one significant area I see an opportunity for the private sector to step up, and help bring robust toolboxes full of open definitions for APIs, and underlying schema applying the leading web concepts and standards.
Along with the increased presence of common approaches to defining APIs and schema, there is an emergence of more of a design-centric approach to delivering APIs. Sadly, this is only present in a handful of leading federal government APIs. Most of them are still pretty classic IT and developer focused implementations, often reflecting the backend system where they were generated. The current API design practices reflect that of the commercial sector around 2012, where you developed then documented an API, without much consideration for design beyond just RESTful principles.
The last administration worked hard to put forth API design guidelines, something that has been continued by USDS and 18F, but this needs to continue in practice. You can see an example of this in action over at the GSA with their prototype City Pairs API–which is a working example of a well designed API, portal, and documentation that any agency can fork, and reverse engineer their own design compliant API. A significant area of movement in the private sector that will contribute to the evolution of API design at the federal level involves the availability of open source API design editors like Apicurio, which is a sign of the maturing private sector API design space, which the federal government should be able to leverage across agencies.
18F gives us a regular look into how they deploy the APIs behind their projects, going into detail on the technology stack employed, and other detail about how it was done. Sadly, this isn’t a common practice across federal agencies, leaving API deployment something that happens in the dark. This practice prevents agencies from learning through each agencies deployment, keeping every API deployment in a silo. When it makes sense from a security perspective, all server side APIs should be open sourced, and available on an agencies Github organization–making all API deployments available for reverse engineering, and deployment by other agencies.
I have to point out again the City Pairs API prototype from the GSA, which open sources the API on Github, and provides step by step instructions on how to set it up, including the entire stack of technology used to operate it. This is how all government APIs should be, and opens up a significant opportunity for the private sector to step up and provide open source APIs that federal agencies can deploy as part of their stack, opening the door to other service and consulting opportunities. The federal government will benefit from the standardization introduced by the GSA, and will continue to grow with the type of API storytelling out of the 18F, eventually bringing API deployment out of the shadows.
Before I begin the shift from the more technical stops along the life cycle and move into the business, politics, and more operational side I wanted to talk about modularization of APIs in the federal government. I’m talking about how APIs are designed, and deployed, considering a microservices way of doing things. I’m also talking about containerization, and ensuring the deployment of APIs are modular, scalable, and well defined. Something that makes them more deployable in any infrastructure on-premise or in the cloud.
Another area of modularization to consider in my evaluation of federal government APIs can be found over at 18F with their leadership in modular procurement. Identifying that the underlying technology isn’t the only thing we we need to be decoupling, and that the business and politics surrounding each API needs to be modular as well. I am seeing some talk of microservices, containerization, and serverless, mostly out of the GSA, but it will be another area we need to see growth in when it comes to API operations in the federal government. Acknowledgment that we need to be decoupling the technology, business, and politics of government resources.
API management is another area that I’ve seen some significant growth and maturity in federal agency API operations. Not only did we see one of the first open source API management solutions Umbrella emerge out of a federal agency, we have seen over 10 federal agencies adopt this as part of their API operations. We need more of this. API Management should be baked into all federal government APIs, standardizing how APIs are secured, rate limited, logged, and measured. Standardizing how all digital government resources are accessed and metered–establishing a common awareness across all federal agency APIs.
This layer of API operations is what is going to set the stage for the next wave of growth in not just government APIs, but also government revenue. Commercial API providers have been leverage service composition at this layer to meter and charge for access to some resources, charging different rates to the public, partners, and even amongst internal groups within a company. This approach to managing digital government resources will emerge as the standard for generating revenue from public resources similar to how timber leases, mining claims, and other physical resources are managed–except this is all being done on public servers, not public lands.
Wrapping around any APIs you deploy, and providing a gateway to solid API management practices always begins with a proper developers portal. The almost 100 developer portals for federal agencies that now exist are due to the hard work of the GSA, direction of OMB, under the guidance of the Obama administration. A central portal is key to application developers, system integrators, and even other agencies knowing where to find data and API resources. Most of the existing API developer portals leave much to be desired, but when you look at the efforts out of CFPB, 18F, and the GSA, you get a glimpse of what is possible in government.
I have to point out the City Pairs API prototype from the GSA yet again. This approach to providing API design, deployment, management, and portal guidance is how consistent developer portal operations will spread across all federal agencies. Each agency should be focusing on what they do best, and not worrying about having to reinvent the wheel each time they deploy a new agency portal, project, or other implementation. There should be a wealth of open source API portals available on Github for agencies to fork and employ immediately to support any API implementation.
I am seeing Swagger UI, Slate, and a handful of other interesting API documentation solutions as I browse through the 400+ federal government APIs. This evolution in open source documentation is partly responsible to the API definition portion of the conversation, as OpenAPI is the engine of Swagger UI, and other API documentation solutions. This approach to delivering API documentation across government APIs, and available as part of any portal template has encouraged significant reuse, and consistency across government API implementations–something that should spread to every API implementation, at every federal agency.
I want to point out the OpenFDA has moved the needle when it comes to API documentation by providing more interactive documentation, including visualizations as part of their API implementations. We need to make API definition driven documentation the default practice across federal government APIs, but it is also significant that we are seeing the API documentation be evolved as part of the valuable information government agencies are making available. This is a good sign that there are healthy API documentation efforts emerging, as this type of innovation is always a by-product of healthy API ecosystems.
I’m seeing healthy communication around data, API, and developer portals from the Department of Education, Census, 18F, USDS, and a handful of others, but for the most part communication around API operations is non-existent at the federal level. This is a problem. This makes many APIs look dead or lifeless, with very little information about what is happening beyond just the documentation. Communication with integrators and developers is an essential part of API operations, and a big reason growth in API efforts at the federal level are slower than anticipated.
I understand that federal agencies are constrained by rules regarding how they can communicate with the public, and operating a blog isn’t as straightforward as in the private sector. However, it is clear that it can be done. Plenty of agencies have blogs, and active social media accounts, this practice just has to be applied across API operations. I’m guessing once we see blogs, Twitter and Github accounts default across federal government agency API operations, we’ll see a significant uptick in the number of integrations and applications that are putting government resources to use in their operations.
While you can find road maps for agencies like the one over at the GSA that include APIs, communicating the road map for a federal agency API really isn’t a thing. Honestly, I don’t see much evidence of communicating what is coming down the road with federal agency API operations, what is happening currently, or what has happened in the past by providing a change log. The USA Jobs API has a change log, and some API related code have change logs as part of their API operations, but communicating around change in government API platforms isn’t really a common concept.
The absence of road maps, issues, and change logs for API operations at the federal level is a problem. I’m hopeful that the usage of Github by federal agencies might shift this reality of API operations currently, but I’m also hopeful that this could be maybe be automated for agencies as part of their API deployment, management, and portal solutions. Taking another concern off the table for API providers, while still ensuring what API consumers and integrators will need to stay in tune with federal government resources being exposed via APIs.
Moving on to the monitoring, testing, and other awarness aspects of API operations. Sadly, there is no evidence of API monitoring and testing at the federal government levels. I searched my news archives, and looked for evidence across the APIs I’ve been reviewing as part of this research. I am not seeing any organized approaches to monitoring API endpoints, sharing the results with the developer community, or the mention of any common API testing tooling in the space.
This is another critical issue regarding the state of APIs in the federal government. As I was browsing around looking for APIs I came across several portals and APIs that had gone dark, demonstrating what a problem it is that nothing is being monitored even for availability. Ideally federal agencies are monitoring the uptime and availability of APIs already, and should be moving on to more detailed, meaningful testing. The only good news on this front I found, was a single API for monitoring whether or not the government in Washington DC is operating or not–at least we have that, right?
Continuing down into the realm of bad news, let’s talk about what I’m seeing when it comes to API security. I am seeing a small bright spot, if we are talking about acknowledging the need for encryption in transit to be default for APIs, but API security seems to stop there. Sure we are seeing some APIs adopting API umbrella to key up APIs, but this is more API management then it is API security. All leaving me concerned about what is going on behind the scenes, as well as thinking security concerns and lack of healthy practices are probably a significant area of friction for new APIs to get out the door in the first place.
Security is the number one concern of agencies when you begin to talk with them about APIs. It needs to be front and center in all API conversations. I’m talking about formal strategies regarding how to security APIs, with official guidance from API leadership, and published pages sharing security practices with the community. API security begins with healthy API management practices, something I’m seeing across federal government implementations, but it is something that evolves with sensible API monitoring, and testing practices, which I’m not seeing. The absence of this awareness is a problem, and needs to become baked into all API efforts at the federal level as soon as possible.
My personal confidence in the reliability of government APIs is pretty low. The availability of APIs I was depending on for my work during my Presidential Innovation Fellowship during the fall shutdown in 2013 was the beginning of my concern, but the stability, longevity, support, communication, and lack of road map or change logs for APIs all have contributed to my concerns over the years. The transition from the previous administration to the current one has moved up my levels of concern significantly. I’m just not convinced some agencies will always be good stewards of the APIs they are making available.
API reliability is just as much a business and political challenge as it is a technical one. Staffing and budget issues will most likely contribute more to API reliability than actual server or network problems. I know that if I was building any business on top of any data or API from a federal agency I would be building in multiple layers of redundancy wherever possible, and work to always have a plan B in place in case an API goes away completely. I feel that this is an area where the private sector can step up, and not just provide vendor solutions that deliver reliable API solutions, but also help from the outside-in, providing hosting, caching, virtualization, and other approaches to ensuring there is redundancy, and always access to vital resources being made available via federal government APIs.
The GSA, and 18F has done an amazing job aggregating API resources for use by all federal agencies. This includes providing a listing of federal government developer portals, as well as all the known agency APIs, in CSV, XML, JSON, and YAML formats. I use these lists to help maintain my own federal agency API directory, and keep in tune with what is going on.
Another significant area of opportunity for investment in federal government APIs is a discovery engine. I’ve worked to create APIs.json indexes for some federal government APIs including the GSA, but this work is time intensive, and ideally is something that should be done by each API provider, within a government agency. You can see this in action over at Trade.gov, who has published an APIs.json for their APIs, which also includes links to documentation, OpenAPI (swagger) definitions, and terms of service. Ideally, all federal agencies would have an APIs.json index in the root of their website, similar to the data.json file that already exists for federal government data inventory. It will take time, but eventually federal agencies will see the benefits in making their APIs discoverable using APIs.json, making it available for inclusion in API search engines like APis.io.
There are a wealth of SDKs, code libraries, and frameworks emerging across the Github accounts of federal agencies. You can find a list of these government Github accounts on the government section of Github. Some agencies do well at showcasing code as part of their API operations, while others you have to hunt a bit, and many who don’t offer any code at all as part of their API offerings. I’d say that client tooling and code across federal government APIs is a mess right now. There really isn’t a single discovery mechanism beyond Github, or any standard for developing and presentation across agencies, or even just across many APIs in a single federal agency.
The generation, development, maintenance, and integration of code for use as part of federal API integration is a pretty significant opportunity for the private sector. The APIs are publicly available, many with existing API definitions, it wouldn’t take much to develop and maintain OpenAPIs for all federal agency APIs, and regularly update SDKs, code samples, and other client tooling that put agency APIs to use. Client tooling should be another thing agency API providers shouldn’t have to worry about. They should be just focusing on maintaining an OpenAPI, and then leverage private sector tooling and services to generate client code, and put common client solutions like Postman and Restlet to use when working with federal government APIs.
After client and SDK concerns I like to expand and consider wider integration solutions, and particularly the integration platform as a service (iPaaS) question. In 2017, there are open source, and software as a service solutions for integrating with APIs, and providing client tools that anyone, even non-developers can put to use. Service like Zapier, and open source DataFire allow for integration with existing 3rd party APIs, and custom API integration, which can be used by business users, as well as developer and IT groups.
You can find If This Then That (IFTTT) recipes for working with USA.gov, and guidance from Zapier for API providers to make their API terms of service government friendly, but there is not a lot of other movement when it comes to the federal government making use if iPaaS solutions. While not as far along as we should be, I’d say the stage is set for federal agencies to begin thinking through their integration strategies, leveraging solutions like Zapier, because you not only get an integration solution, you get access to the experience that comes with integrating with over 750+ APIs.
Another significant area for API client integration as well as API deployment is the spreadsheet. Both Microsoft and Google Spreadsheets allow for consumption of many data, content, or algorithmic APIs right in the spreadsheet. When you consider spreadsheets across federal agencies, you see how this could significantly change how federal government APIs are put to use internally at the agencies where they are deployed, inter-agency, or even by external 3rd party researchers, partners, and integrators.
You can find quite a few data downloads in spreadsheet format at data.gov. Use the interesting Excel data add-in and Google Sheets add-on over at the U.S. Energy Information Administration (EIA), and see the cool spreadsheet to API work over at 18F to consider a handful of examples of the role spreadsheets play in the federal API game. There needs to be a significant amount of investment in spreadsheet to API and API to spreadsheet integration across the federal government in coming years, to reach beyond the technical community audience.
Now that we are seeing API management practices being adopted by federal agencies, I’d like to plant the seeds for some of the next steps for agencies when it comes to making public resources available for commercial use. API Umbrella, and api.data.gov help manage API key usage for developers and the daily and hourly rate limits on accessing api.data.gov APIs, ensure API resources stay available, and aren’t overused by any single consumer.
Commercial API providers like Amazon, Google, and others often charge different rates for different API resources, while also providing a free level of access for the public, or possibly researchers. There are also many API service providers who provide advanced API management solutions for billing against API usage, charging for high levels of access, or maybe establish unlimited usage levels for a select handful of trusted consumers. It is common to have multiple plans for API access, and charging for API consumption in some cases, helping pay for infrastructure and other hard costs.
I get that this is a controversial topic in my circles–paying for accessing public data? In short, data is valuable, and making sure it is available, and usable creates value. It costs money to make data available as APIs is a reliable and sustainable way. It makes sense to charge for high volume of commercial access to APIs, similar to how government charges commercial operators to access resources on public lands. It may seem like a crazy notion now, but in the future, this is how government agencies will generate much needed revenue from public resources.
Continuing the API management and plans conversation, I’d like to highlight what I’d consider to be an under-realized, and under-utilized partner network of federal government API consumers. You can see what I am talking about over at the Open Data 500 project, which is “the first comprehensive study of U.S. companies that use open government data to generate new business and develop new products and services.” This approach to quantifying the companies using federal government open data, needs to be applied to APIs, establishing a feedback loop with any company who integrates with a federal agency APIs.
The federal government has many partners, but is rarely ever showcasing or leveraging these partners in any meaningful way when it comes to API integration, or application development. The friction of existing laws regarding how government can work with companies causes some of this, however much of this is due to not understanding how 3rd party development works within API ecosystems at the federal level. Some efforts like the Open Data 500 have been established to help quantify how open data is being used across industries, but in my experience much more work is needed to help understand who is using federal government resources, and who might be a fit for heightened levels of engagement to strengthen partner relationships between the public and private sector.
You see the beginning of this in action over at 18F in the GSA where they are drafting, sharing, and managing key aspects of developing software and managing system integrations with APIs out in the open, making the public a first class citizen in these discussions. The legal department for API operations can be conducted via Github, including partners, developers, and the public in the process. There is a precedent, and the benefits for not just making government transparency, but making technology better serve citizens exists. Agencies will learn API design, deployment, and security, privacy, and terms of service by emulating what they see–if there are no examples out there for them to follow, these legal aspects of operating government will remain behind closed doors.
Building on my earlier comments on the lack of communication out of existing federal agencies when it comes to data and API efforts, the advocacy and evangelism of APIs out of federal agencies is near non-existent. You see efforts from a handful of agencies trying to do more outreach and evangelism around their data and API projects, often led by 18F and USDS, but evangelism is something that just isn’t in the DNA of federal government as it currently exists.
Getting the word out about federal APIs is essential to establishing a vital feedback loop that government agencies will need to improve upon systems, data collection practices, and generally serving the public. Without outreach few will know these API resources exists. Without outreach agencies will never know who is using, or who might be looking to use government resources. Evangelism isn’t just about hackathons and conferences, it is also about communications, support, and other essential aspects of developing software and integrating with systems. With no outreach, all of this occurs in a silo, and behind closed doors, limiting what is possible.
There is a significant amount of momentum when it comes to data and APIs at the federal government level coming out of the last administration. With the leadership from the Obama administration we see agencies stepping up to publish their public data inventory, and increasing the number of developer portals and APIs that are available. This momentum augments existing IT Momentum to grow and expand government IT infrastructure, with API investments reflecting expansion in government doing business securely on the open web, leverage the low-cost, scalable benefits of building using existing web technology.
While this momentum is real, it is seriously in danger of slowing, and even reversing course in the current administration. All the leadership available regarding API implementation has gone away, leaving agencies alone in their API strategy development. Without leadership many of these API efforts will wither and die as they are. With a vacuum in leadership there is a significant opportunity for vendors, and outside leadership to step in, take ownership of open source projects, standards, and guidance, and play an important role in keeping projects active, alive, and even growing.
Lots of Work
The most common advice I give agencies regarding what they can be doing on the API front is doing more of the same. Keep opening up data. Keep publishing APIs. Keep supporting and expanding your developer portals. We have lots of redundant work ahead of us to make sure all federal IT systems can securely communicate over the web to serve all constituents, and support all government workers. There are a number of areas like monitoring, testing, security, communication, and evangelism that should be invested in, but we also just need to do the hard work keeping what is already on the table active.
It is unrealistic to expect the federal government to do all this work alone. Which is why ALL federal agencies should be doing APIs, to open up data and internal systems to other federal agencies, vendors, partners, and the general public to help carry the load. It is perfectly acceptable for work around public data and other resources to happen out in the open via API integrations, Github, and other platforms and services already in operation. This is why APIs can’t be shuttered or allowed to go dormant. Any government project data project in operation should be externalized, API-ized, and opened up for a collaborative approach to running government.
Private Sector Opportunity
The most important part of this state of APIs in federal government story is about what the private sector can step up and do. I mentioned several opportunities for companies, organizations, and institutions to step up and help be stewards of federal government data, APIs, and open source code. This means doing business through existing federal government procurement channels, but it also means stepping up and investing in existing federal agency open data and API efforts, engaging with agencies via Github repositories, forums, and social media.
You see companies like Amazon, Microsoft, Zapier and Github stepping up with government focused section of their platform, specialized products and services, and terms of service tailored for government adoption. We need more of this. We need more startups to step up, incentivize, and lead when it comes to APIs in government. Federal government APIs aren’t just about what government is doing with APIs, it is also about what the private sector does to use these APIs, and how they engage with government agencies whenever possible to hep improve the quality of APIs, and software coming out of federal agencies.
This reality is even more dire in a Trump administration. In the current administration any API leadership has been halted, and the two groups (USDS & 18F who are leading the charge are facing friction, and seeing top talent exit back to the private sector, unable to make the impact they had envisioned when they first signed up for government service. The API hard work is continuing across the federal government, within each agency, but in coming years we need the bulk of work to be assumed by the privacy sector, letting dedicated government employees know that we have their back, are here to support, integrate, and build on their hard work, while providing valuable and constructive feedback wherever we can.
Thank you to Red Hat for bringing me out, and gathering folks at Tyson’s Corner for the interesting discussion about APIs–some interesting people showed up including government, vendors, and consulting groups.