Taking Another Look At Where We Are At With Data.json Files For Federal Agencies

I am spending some time, refreshing my look at the open data work, specifically the results of OMB Memorandum M-13-13, which required federal government agencies to publish a data.json file, providing a machine readable inventory of an agencies public data resources. I recently received a Knight Foundation Grant to build a prototype, explore one possible "whats next", for open data in the federal government--allowing me to push this valuable work forward.

To refresh my memory, I pulled from 87 federal agencies I am tracking on as part of my regular API monitoring--raising the bar from 22 federal agencies having published their data.json file, to 27 who have published their data.json file. 

Bureau of Economic Analysis (with data.json returning a 200 status code and 10 datasets)
Consumer Financial Protection Bureau (data.json) returning a 200 status code and 2 datasets)
Department of Agriculture (data.json) ( returning a 200 status code and 524 datasets)
Department of Commerce (data.json) ( returning a 200 status code and 492 datasets)
Department of Defense (data.json returning a 404 status code and 374 datasets - WTF)
Department of Education (data.json returning a 200 status code and 281 datasets)
Department of Energy (data.json returning a 200 status code and 1001 datasets)
Department of Health and Human Services (data.json returning a 301 status code and 1311 datasets)
Department of Homeland Security (data.json returning a 403 status code and 332 datasets - what?)
Department of Housing and Urban Development (data.json returning a 200 status code and 174 datasets)
Department of Justice (data.json returning a 403 status code and 1642 datasets - what?)
Department of Labor (data.json returning a 200 status code and 363 datasets)
Department of State (data.json returning a 200 status code and 93 datasets)
Department of the Interior (data.json returning a 200 status code and 21,870 datasets)
Department of the Treasury (data.json returning a 200 status code and 234 datasets)
Department of Veterans Affairs (data.json returning a 200 status code and 953 datasets)
ENERGY STAR (data.json returning a 200 status code and 53 datasets)
General Services Administration (data.json returning a 200 status code and 164 datasets
HealthData.gov (data.json returning a 200 status code and 1,311 datasets)
National Archives and Records Administration (data.json returning a 200 status code and 62 datasets)
National Science Foundation (data.json returning a 200 status code and 138 datasets)
Nuclear Regulatory Commission (data.json returning a 200 status code and 34 datasets)
Office of Personnel Management (data.json returning a 200 status code and 624 datasets)
Small Business Administration (data.json returning a 302 status code and 36 datasets)
Social Security Administration (data.json returning a 200 status code and 1163 datasets)
U.S. Agency for International Development (data.json returning a 200 status code and 201 datasets)
White House (data.json)returning a 200 status code and 23 datasets)

It isn't the massive adoption I would like to have seen, but I am happy to see three more agencies get on board with opening up their data inventory, and pleased to see the increase from 6,482 to around 33,073. As usual, I am interested in breaking down what is actually happening across these 26 agencies, but I am looking to identify which agencies should be publishing their data.json files, but are not.

While manually checking each federal agencies data.json availability, these are the agencies that I came across that SHOULD have a data.json file present in the root of their domain, there really is not excuse why they shouldn't in 2015.

Bureau of Alcohol, Tobacco, Firearms, and Explosive
Bureau of Justice Statistics
Bureau of Labor Statistics
Centers for Medicare and Medicaid Services
DEA - E-Commerce Program
Federal Aviation Administration
Federal Emergency Management Agency
Federal Register
National Aeronautics and Space Administration
National Climate Data Center
National Highway Traffic Safety Administration
National Institute on Drug Abuse
National Institutes of Health
National Library of Medicine
National Weather Service
Office of Justice Programs
Office of the Historian
Substance Abuse and Mental Health Services Administration
U.S. Energy Information Administration
U.S. Geological Survey
US Census Bureau

I will be focusing on the 26 federal agencies who have published their data.json, but will be keeping an eye on how we can incentivize the additional 24 agencies to do it as well. As I dig into the details of each agencies data-sets, I will be looking for where data-sets exist that aren't present on inventory, and help agencies along in their journey.

As I continue my work as part of my Knight Foundation Prototype, I'll have lots more stories to share, this is just the beginning.

Update: Added Department of Education to list with data.json