02 Sep 2015
I'm always on the hunt for API driven solutions that will help the average business user be more exposed to the power of APIs. It is common for us technologists to kick up dust, and get all excited about what we build, when in reality, it doesn't mean anything to the average person in the real world--after 25 years in business, I still do this regularly.
This is why, having a constant reminder to look for solutions that bridge the technical world of APIs, to the average business users world, helps me break free from my own biased approach. When I lean back in my chair, and think about what solutions are currently available in the space, that have this potential, these are the two companies.
Access web services from spreadsheets. Blockspring is the world's library of functions, accessible from everywhere you do work. Your tables automatically stay up-to-date. Save hours of manual, error-prone exports and copy/paste. Get fresh data from any source whenever your spreadsheet recalculates. Build and share powerful spreadsheet tools.
Form.io is a platform that enables developers to rapidly build modern web and mobile applications. Our intuitive drag & drop interface allows you to create both the forms and the RESTful API's all in one easy step!
Both of these platform speak API, but more importantly they speak spreadsheet and forms, two of the most common dialects spoken by the average business user today. I'm closely working with both groups to better understand how they can communicate what they do to the business world, and repeatedly the examples in play are not focused on API, they are focused on solutions that make sense to the end user, but just happen to be API driven.
While much of the API design, deployment, management, and integration services and tooling that has evolved over the last five years will play a significant role in providing the API base we will need for the API economy, services like Blockspring and Form.io will bridge the critical last mile to the average user. These two companies are making APIs accessible via the most ubiquitous client out there, the spreadsheet, and helping average users understand what an API is, and what they do, through another ubiquitous interface, that almost everyone gets--the form.
I just wanted to take a moment to highlight the importance of these types of services and tools, among the wealth of other companies I am highlighting.
01 Sep 2015
One of the reasons I setup individual research projects, is to provide me with a structure for better defining each aspect of the API world, something I am working hard to jump-start within my API security research. You will notice the project does not have any building blocks defined, which when you compare with one of my oldest research areas, you start to see what I mean.
The blog posts, and other links I curate as part of my API security will help me find companies and tools that are providing value to the space. As I break down each company, and what they offer, I often have to read between the lines, in trying to understand how an API, service, or tool can be used by API providers, as well as potentially API consumers. I am looking for APIs that offer security, but also APIs that offer security to APIs--make sense?
As part of this research, I am playing with Metacert, which bills itself as a security API for mobile application developers, helping them block malicious ads, phishing links & unwanted pornography inside apps, but I think it is so much more. I could see Metacert being pretty valuable to API providers, as well as API consumers building web and mobile apps. Security isn't always about brute force attacks, and could easily just be in a simple link, added with some content, via your API.
I am adding Metacert to my API security research, with a focus on its potential to API providers. I could see API providers seamlessly integrating the Metacert API into their own stack, processing all links that are submitted through regular operations. I will also be adding link screening like this as a building block to my API security research.
If you are looking for a wise investment in the API security space, you should be talking with Metacert. APIs like Metacert provide us with a model for thinking about how we deliver API driven security services for web, mobile, and IoT applications, but it also provides a potential wholesale API layer that other APIs can use to better secure their own APIs. I consider it a strong blueprint, because its API driven, they have all the essential building blocks, which includes a monetization strategy, and they do one thing, and they do it well.
01 Sep 2015
As I make my way across my research projects, I'm learning more about how companies like Metacert can deliver valuable security services to API providers. I'm also getting a better idea of the nuance that goes into monitoring APIs, from API Fortress.
API Fortress has very interesting API monitoring story, derived from the Etsy API. This isn't the API monitoring story you'd think, it isn't about the overall stability of the Etsy API, and whether its up or down, it is about the details of API payloads, and inconsistencies they found scanning the API.
Through a payload toast of the Etsy API, the API Fortress team found an occurrence of NULL values in 3,600 out of 50,100 items scanned--which negatively affected what results that were actually accessible via the API vs the Etsy interface. Something that can result in loss of exposure for products, translating into reduced sales.
While full API outages are definitely front and center when you think about monitoring, things like this could potentially drain revenue over a long period of time, in a way that makes it very difficult to discover. I can only imagine, when you are doing things at the scale Etsy is, this kind of monitoring could plug some pretty big holes in your sales bucket.
API Fortress is already part of my API monitoring research, but just like with API security, I am going to add content analysis, and pattern matching like this as building blocks for API monitoring. After I make my way through all of the current companies I have listed in my API monitoring, I will publish the master list of common building blocks I've collected.
31 Aug 2015
I am always pleased to see API providers publishing Swagger definitions, and using them to drive their interactive documentation. Projects like the Global Change Information System API, are getting on the API definition bandwagon, and this is a good thing. I have been pushing API definition formats like Swagger and API Blueprint since 2012, but in 2015, while I want to keep on-boarding folks to the concept of using API definitions for interactive documentation, but I also want them to also understand that their APIs definition will be used in other areas of API operations as well.
Most people think Swagger is the documentation, and have not been able able to separate the specification from the interactive documentation. I think Apiary has done a better job of this separation with the separation of API Blueprint from Apiary. As an API provider, you may not have evolved to a full API-first level of operation, which is ok, but I encourage you to make your Swagger or API Blueprint definition as accessible as possible, so your API consumers can put to use in other ways--even if you don't have the time.
As soon as I saw that the Global Change Information System API employed Swagger for their documentation, the next thing I wanted to do was also use in my Postman client. While the Swagger UI provides me with a hands-on way of getting to know the GCIS API, I have to come back to the site to play with more, and it doesn't give me as much detail about how the API works, as my Postman client does. All GCIS API team has to do, is publish a text or image link to their Swagger definition in a prominent location, so it is obvious to us consumers.
I get easily reverse engineer the Swagger UI, using my Google Developer tools, but it is a couple of extra steps for me, that stands in between me, and making calls to the GCIS API in my local Postman Client. Interactive API documentation via Swagger and Apiary has significantly moved the API definition conversation forward, but just like we should be thinking beyond just language specific clients, we also need to be enabling our API consumers to speak the formats popular HTTP clients like Postman speak.
31 Aug 2015
When it comes to API security, there are a number of things to consider, something I will be be working to define, and share as part of my ongoing research. However there are three building blocks that are front and center in most security conversations--SSL, API keys, and OAuth. SSL is a must-have, and OAuth is fast becoming a must-have when there is personal data involved, but I still encounter numerous misconceptions around the role API keys actually play in security.
API key, and its accompanying secret are a common way to secure API access. You require developers to register for an account, create a new application, and they are then given an application key, plus a secret that is passed along with each API call. You cannot call the API, without passing in your API key and secret. This act alone, is what people view as the security role API keys are playing, and I get a number of questions if this is truly secure.
No it is not. Looking for two values being passed with each API call, really doesn't do that much. The actual security of your platform requires a much higher level, IT view of security (which I won't go into here), with API keys being just one tool in your security toolbox. Where keys do play a huge role, is around the awarness that is introduced, of who is accessing what, and managing how much they can access (aka mitigating how much damage is done, when there is a breach).
This short sighted views of API keys is why many companies feel they can just roll their own API management solution, and why many API developers and architects think the keys themselves possess some magical security pixie dust. The actual power comes from your awarness of the resources you are exposing via APIs, organizing into coherent service tiers, applying meaningful rate limits on these resources, and evolving your detailed awareness of who is accessing your resources, how much they are using, and when.
An API key and its companion secret offer very litle security, but the awarness they bring when you have a modern API management layer in place, can bring huge security benefits to the table. Something that will not prevent every security breach, but with the right mechanisms in place you can be alerted of breaches in real-time, and dramatically limit the extent of their damage, when they do occur.
31 Aug 2015
While there are numerous examples of APIs that successfully offer more than one option when it comes to the content types their API returns, the concept is missing from the large portion of the APIs I review. When I see good examples of this in the wild, I try to make time to showcase, and share, to help with the HTTP literacy of my readership.
One API which is rocking several fronts when it comes to API design for me, is the Global Change Information System, which interestingly is an alliance between government agencies, not something born out of Silicon Valley. When requesting resources from the GCIS API, the following HTTP accept headers are honored:
When integrating with the GCIS API, it all begins with some basic JSON, but then look at all the other linked data goodness in there! JSON-LD, RDF, Turtle!! Discovering APIs that support robust content type negotiation like this can be hard to find, most APIs don't talk about it, so even if they do, there is no way to know about it. Thankfully the GCIS API shares it via email, as well as describes in the GCIS Swagger definition, which is how I learned about it.
Content-Type negotiation is one of those HTTP literacy areas that need a lot of discussion, helping on-board API newbies to the power and possibilities that are present here, and that you can go beyond just plain ol JSON, and receive much richer API responses.
29 Aug 2015
I'm immersing myself in the Data Sharing and Open Data for Banks, published by the HM Treasury and Cabinet Office in the United Kingdom, "to explore how competition and consumer outcomes in UK banking could be affected by banks, giving customers the ability to share their transaction data with third parties using external Application Programming Interfaces (APIs)."
While I do not think an open API standard will ride into town like a white knight, saving England from all its banking problems, I do think there is a lot of good that will come from this effort. The banks have a lot of technical debt, corruption, and many other issues to deal with before APIs will even work, but I think there is enough pressure on them right now from consumers, and startups, that they will have to change at some point--or die.
As I look for evidence of movement around an open API standard online, I came across the response to the government proposal back in March. I thought there was a lot to learn from in these responses, not just for banks, but for any company that is struggling with a coherent API strategy. You can read the document yourself--here are the specific responses.
Benefits and risks of an open API standard
2.1 The first set of questions in the call for evidence invited views on the benefits and risks of an open API standard.
2.2 The vast majority of the respondents supported the development of an open API standard, and respondents agreed that there are many potential benefits of an open API standard in UK banking. The most commonly cited benefits were increased competition and innovation in banking, a greater degree of consumer choice in financial services, and the development of new and better services tailored to customers that can enhance the overall efficiency of banking. The government also repeatedly heard that an open API standard would result in customers feeling more empowered to engage with their banking or financial services.
2.3 However, many respondents also recognised that there are risks that could arise from an open API standard. One concern that was raised was the risk that an individual (to whom the data belongs) may not have meaningful control over the exchange of the data between third parties and financial institutions. Respondents from the technology industry and the banking industry cited risks around privacy of customers’ data and the potential for fraudulent use of data. In line with this, the need for appropriate security and vetting systems for third parties was regularly mentioned in the submissions received. In addition, responses received from the payments and banking industries explained that an open API standard should avoid ‘locking in’ a set of standards which cannot be enhanced or restrict banks from their own innovations, and that it should be compatible with European legislation.
Development of an open API standard
2.4 The next set of questions in the call for evidence considered the process for the development of an open API standard.
2.5 The government asked what it could do to facilitate the development and adoption of an open API standard. The majority of respondents said that that the government has a key role to play in coordinating between the banking industry and the fintech community, and driving forward the design of the open API standard, particularly in relation to the development of the standards around data security and confidentiality. Respondents predominantly from the fintech community also explained that government has a role in helping to educate customers to understand the privacy and security issues surrounding an open API standard, and the steps that can be taken to mitigate those risks.
2.6 The government questioned who should play a role in the development of an open API standard, who should be able to make use of it and how it should be used. The government received responses suggesting various different institutions could be involved in the development of an open API standard. These include banks and other financial institutions; the British Standards Institute and International Standards Organisation; the Financial Conduct Authority; the Bank of England; the Information Commissioner’s Office; the British Bankers’ Association; the Payments Council; the European Banking Authority; data security experts; software developers; fintech businesses and the government. Submissions from the fintech and technology business said that the open API standard should be available to be used by everyone, subject to the appropriate vetting procedures, and not restricted to those who banks have existing relationships with or can afford to pay for access. The banks agreed that appropriate security vetting procedures should be developed.
2.7 In response to the question on the cost of developing an open API standard, the government received a variety of estimates ranging from negligible costs to tens of millions of pounds. Several respondents from all industries explained that they were unsure of the cost of delivering an open API standard, or that it was difficult to predict how much it would cost, because much of the detail of the design of the open API standard remains to be agreed. Many respondents also highlighted that the costs were likely to vary greatly between institutions. Software and app developers, however, explained that costs may be lower than anticipated because some banks have already developed their own external APIs and so are not starting from a blank sheet of paper.
2.8 The government asked questions on how long it would take to deliver an open API standard. While there was a degree of variation in the responses, broadly respondents were in agreement that 1 to 2 years was a reasonable timescale to develop and deliver an open API standard in UK banking, although more in depth responses explained that the timescale for delivery would be clearer once the design specification of the open API standard is more defined.
2.9 The government also asked what issues would need to be considered in relation to data protection and security. Respondents highlighted that customers must at all times be in control of which third parties they grant access to their bank data and for how long, how that data can be used and the level of granularity of data that can be assessed. The banking industry and many technology firms said that third parties should be appropriately vetted before being able to access customer bank data, and a permissions list of authorisations and authentications should set out all approved third parties. One technology company added that the degree of access to customers’ bank data could be tiered according to the level of security standards met by the third party.
2.10 The government invited views on the technical requirements an open API standard should meet and adhere to. Respondents put forward a number of recommendations, the most popular options involved such requirements as HTTP, JSON, XML, OAuth, REST5 and Cloud based architecture.
I also found the government responses to be equally valuable:
3.1 It is clear from the vast majority of responses received during the call for evidence that the benefits of an open API standard are numerous and widely recognised. The key benefits that were identified by respondents were an increase in consumer choice, more competition in banking and an enhanced process, experience and outcome for consumers. Respondents strongly emphasised their support for the delivery of an open API standard in UK banking.
3.2 While much of the detail of an open API standard is still to be agreed, it is evident that an open API standard can be designed in a way which meets requirements around data protection and security, and at reasonable cost. The majority of respondents also said that developing an open API standard to a timescale of 1 to 2 years would not be unreasonable.
3.3 Respondents from the financial industry and the technology industry were aware of the risks that an open API standard could bring to consumers such as data privacy and the possibility of fraudulent use. It was made clear, however, that these risks are largely addressed through existing data protection laws, and can be mitigated through detailed planning and meticulous scoping of the open API standard design.
3.4 Submissions to the call for evidence also made clear that the publication of more open data in banking can have benefits to customers and financial institutions, and help to boost competition. The government, however, notes that steps will need to be taken to ensure the appropriate degree of data security and protection to customers is maintained.
3.5 The government is therefore committing to deliver an open API standard in UK banking, and will set out a detailed framework for the design of the open API standard by the end of 2015. The government will work closely with banks and financial technology firms to take the design work forward and, as part of those discussions, will also take forward ideas to introduce more open data in banking for the benefit of customers.
3.6 Delivering an open API standard in UK banking will help to drive more competition in banking for the benefit of customers, and enable fintech firms to make use of bank data on behalf of customers in a variety of effective and creative ways. It will ensure that the UK remains a global hub and a world-leader for financial technology and innovation.
I think 3.5 is the most important: The government is therefore committing to deliver an open API standard in UK banking, and will set out a detailed framework for the design of the open API standard by the end of 2015.
Let's get to work. I'm setting up my monitoring to track on things as they roll out, and hopefully provide a blueprint that can be employed here in the U.S. I'd love to see similar work come out of our Department of Comerce, but I guess, as with other aspects of the open data and API movement here in the U.S., we will be following a UK lead.
29 Aug 2015
Late in 2014, the HM Treasury and Cabinet Office in the United Kingdom, published Data Sharing and Open Data for Banks, "to explore how competition and consumer outcomes in UK banking could be affected by banks, giving customers the ability to share their transaction data with third parties using external Application Programming Interfaces (APIs)."
Very similar to the Obama open data mandate in May of 2012, I saw a lot of discussion shortly after the release of the document from the HM Treasury, but not many details on how things are going so far. There was a response to the document published back in March, which holds some interesting perspective, but not much else from the government, or banks, that I can find.
It is only August, I'm sure we'll see more activity this fall and into 2016, when it comes to standardizing banking APIs in the UK. I've tracked on efforts like the Open Banking Project for a while now, but I think what is potentially happening in the UK is worth me taking things up a notch or two. To help me, I will setup a banking research project, which will push up banking and related APIs in my daily and weekly monitoring, something I can review regularly, and maybe push back on some key players to get more information on where things are. Establishing an official research project really helps me find the key players, both individual and companies, which in turns increases the amount of information I gather as part of my research.
If you work for a bank that does business in the UK, or possibly for the government, I'd love to hear your thoughts, even if it is off the record. I'm hopeful for what can occur in the UK, but I'm also eyeballing the effort as a potential blueprint that the rest of the world could follow. I know the banking industry suffers from some crippling legacy technical debt, and are slowed by government regulations, but I'm pumped about the potential after reading through the Data Sharing and Open Data for Banks again--we will see a significant shift in the next five years, when it comes to how we manage our money.
29 Aug 2015
I enjoy what I do, but when I embarked on my API Evangelist journey in 2010, I set myself on a mission to educate people about the business of APIs, and highlight that it isn't just the technology that makes APIs a thing. I have worked hard to distill down what it takes to execute an effective API management strategy into usable advice that anyone can run with when crafting an API strategy.
As I conduct my monitoring of the API space, It makes me feel accomplished, when I find my work cited, influencing important API related projects. This just occurred to me as I was taking a fresh look at the Data Sharing and Open Data for Banks, published by the HM Treasury and Cabinet Office in the United Kingdom. As I was reviewing the document I happened to search for apievangelist (I know, I'm vain)--I was pleased to find a single citation to my work.
Reference 152 - Thirdly, making the most out of external APIs involves providing on-going support to the developers and third party organisations that make use of them. Good API management can involve providing samples of code, a curated support forum, a sandpit with dummy data, an up to date blog, FAQs and more.
It isn't much, but it is a nod to my work, that helps validate what I've been studying for the last five years--the business of APIs. For me, this isn't about recognition, its about knowing I'm making an impact, and I'm influencing policy, and helping companies, institutions, and government agencies understand API best practices. Little mission accomplished, I'm contributing a small piece to potentially improving the UK banking system, which is something that will undoubtedly spread beyond just England.
Ok, celebration is over. Nice job Kin. Back to work. There is too much to be done.
29 Aug 2015
I'm almost to the point with my API stack, where I can start plugging in new APIs I have planned. Up until now, the APIs i have deployed, are of little use to a wider commercial audience. However some of the APIs I have planned for the next year, I'm looking to monetize their usage, and operate as part of a larger commercially viable API stack. (practice what I preach baby!)
To run this stack, I need a plug and play way to define what an API is costing me, and potentially how much revenue I am generating from each API. With this in mind, here is my draft look at an API monetization framework, that I am employing across my API Stack.
Acquisition (One Time or Recurring)
- Discover - What did I spent to find this. I may have had to buy someone dinner or beer to find, as well as time on the Internet searching.
- Negotiate - What time to I have in actually getting access to something. Sometimes its time, and sometimes it costs me.
- Licensing - There is a chance I would license a database from a company or institution, so I want to have this option in there. Even if this is open source, I want the license referenced.
- Purchase - Also the chance I may buy a database from someone outright, or pay them to put the database together, resulting in one-time fee.
Development (One Time or Recurring)
- Normalization - What does it take me to cleanup, and normalize a dataset, or across content. This is usually he busy janitorial work necessary.
- Design - What does it take me to generate a Swagger and API Blueprint, something that isn't just auto-generated, but also has a hand polish to it.
- Database - How much work am I putting into setting up the database. A lot of this I can automate, but there is always a setup cost.
- Server - Defining the amount of work I put into setting up, and configuring the server to run a new API, including where it goes in my operations plan.
- Coding - How much work to I put into actually coding an API. I use the Slim PHP framework, and using automation scripts I can generate 75% of it usually.
- DNS - What was the overhead in me defining, and configuring the DNS for any API, setting up endpoint domain, as well as potentially a portal URL.
- Compute - What percentage of my AWS compute is dedicated to an API. Flat percentage of the server its one until usage history exists.
- Storage - How much on disk storage am I using to operate an API? Could fluctuate from month to month, and exponentially increase for some.
- Bandwidth - How much bandwidth in / out is an API using to get the job done.
- Management - What percentage of API management resources is dedicated to the API. A flat percentage of API management overhead until usage history exists.
- Evangelism - How much energy do I put into evangelizing any single API? Did I write a blog post, or am I'm buying Twitter or Google Ads? How is word getting out?
- Monitoring - What percentage of the API monitoring, testing, and performance service budget is dedicated to this API. How large is surface area for monitoring?
- Tier(s) - Which of the 7 service tiers is an API available in, and which endpoint paths + verbs are accessible in the tier (api-pricing definition).
- Credit(s) - How many credits does an API use when any single endpoint is engaged, specified as entire endpoint or individual paths + verbs (api-credit definition).
- Monthly - How much revenue is being brought in on a monthly basis for an API and all of its endpoints.
- Users - How much revenue is being brought in on a monthly basis for a specific user, for an API and all of its endpoints.
- Applications - How much revenue is being brought in on a monthly basis for a specific application, for an API and all of its endpoints.
I am looking for this framework to help me set pricing, and rate limits for any API I publish. My goal is to rapidly make available some valuable databases, and more functional APIs using common open source software, available for free, but also generate enough revenue from high volume users, to run the whole thing. To do this, I need to understand exactly what an API is costing, allowing me to set a price, with the intent of breaking even, and then generating some revenue where it makes sense.
As part of this work I will be generate an APIs.json type I am calling api-pricing, which I am looking to help be balance out consumption across my API stack. Using my 3Scale API infrastructure I am able to easily add and subtract credits for API usage across users, and apps, then handle the billing based upon what pricing I have set for all my API usage.
My pricing will not just be about retail usage. Some of my APIs I will be deployed in other people's infrastructure, and letting them control the pricing, credits, and service composition. This is the wholesale layer to my strategy, allowing me to go beyond my own internal usage, by B2D usage, and open up new opportunities for API deployment and consumption.
As with most of my work, I'm going to be very transparent about my pricing, making sure it is indexed within each APIs.json file, and available alongside each APIs Swagger, API Blueprint, Postman Collection, and API Science monitor--encouraging wider consumption, and processing.