03 Sep 2015
It has been almost 10 years since Google launched the Maps API. With as many APIs as Google have, you'd think they'd have a better handle on a standard approach to pricing across all of them. As we've seen with Google Translate, they have struggled with the right way of pricing APIs, as well as communicating this to their developer ecosystem.
This week Google took further steps on the road to standardizing how you pay for Google Maps API, opening up pay as you go purchasing via the Google Developer Console. Here is how they put it:
In this new purchasing structure, the Google Maps Geocoding, Directions, Distance Matrix, Roads, Geolocation, Elevation, and Time Zone APIs remain free of charge for the first 2,500 requests per day, and developers may now simply pay $0.50 USD per 1,000 additional requests up to 100,000 requests per API per day. Developers requiring over 100,000 requests per day should contact us to purchase a premium licence.
It is good to see Google run with some pretty standard pricing, and running with something in line with what other API providers are putting to work. Just like Google worked to standardize OAuth 2.0 across the APIs, and get their approach to API management organized via the developer console, it looks like they are now working hard to get their API pricing in order.
I still see many API providers struggle with API pricing, both large and small. Many of the larger API providers do what Google did, and offer a free access level, and then once they need to generate revenue they shut things down, or tighten things up, rather than just offering a clear path to paying for resources. This approach has fueled the debate around whether you can depend on APIs, and also whether or not a freemium approach to APIs will work at all.
Another common stumbling block I see API providers trip over when it comes to pricing, is a disconnect between each level of access, meaning there is a freemium layer, and there are paid tiers of access, but there is no even steps between them, making it impossible for many developers to traverse. This generally makes a clear statement that it is ok to play with our API, but you really need to be the enterprise to actually do business with us.
This is all something that can be mitigated with a clear, pay as you go, pay for what you use API model with a free tier of access, and sensibly tiered paid levels of access, as well as a robust, transparent partner program for higher levels of needs. There are plenty of models out there to follow when crafting your own API monetization strategy, so make sure you something in place from day one, even if you are tweaking along the way like Google has had to do.
03 Sep 2015
If you have heard any of my talks at API Days Sydney, in Barcelona, or at Gluecon in Colorado this year, you've heard me talk about wholesale APIs, and how I am using Docker to help make some of my APIs available for use as private or white label APIs. The concept of a white label, private label, or wholesale API is something I've been talking about for a while, and is something I am starting to see more of in the wild.
The most recent sighting is from payment API provider Dwolla, with four new API endpoints that deliver a white label ACH payout API solution. According to the announcement from Dwolla, the white label API solution will:
"..leverage Dwolla’s existing infrastructure—our fraud analysis, bank partnerships, real-time fraud analysis, communication protocols and more—while maintaining your brand’s look, feel, and name on end-user interactions. This functionality will provide platforms a powerful, flexible, and custom ACH infrastructure at a fraction of the cost and time."
When you use the Dwolla white label ACH payout API, "there are no individual transaction fees or percentages—regardless of how many times you, your business, or your application transfers money". Providing a pretty interesting alternative to the common, pay for what you use business model used by man API providers today.
I predict this approach to API deployment will become more common place in coming years. Many developers, and API consumers will be quite content just calling your regular retail API, but more specialty, and larger partners are going to want the functionality you offer, but are not going to want to be metered for each API transaction.
When I deploy wholesale, or white label versions of my APIs I will not be charging for each API call, however I will still be managing API access, and usage with my 3Scale API infrastructure, because I still want to be able to get analytics, and other awareness generated via my API management layer. The cost of each wholesale API installation will depend on the scope of the instance, whether it is in our infrastructure, or yours, and how much ownership me and my APIware team will have in API operations.
02 Sep 2015
I'm always on the hunt for API driven solutions that will help the average business user be more exposed to the power of APIs. It is common for us technologists to kick up dust, and get all excited about what we build, when in reality, it doesn't mean anything to the average person in the real world--after 25 years in business, I still do this regularly.
This is why, having a constant reminder to look for solutions that bridge the technical world of APIs, to the average business users world, helps me break free from my own biased approach. When I lean back in my chair, and think about what solutions are currently available in the space, that have this potential, these are the two companies.
Access web services from spreadsheets. Blockspring is the world's library of functions, accessible from everywhere you do work. Your tables automatically stay up-to-date. Save hours of manual, error-prone exports and copy/paste. Get fresh data from any source whenever your spreadsheet recalculates. Build and share powerful spreadsheet tools.
Form.io is a platform that enables developers to rapidly build modern web and mobile applications. Our intuitive drag & drop interface allows you to create both the forms and the RESTful API's all in one easy step!
Both of these platform speak API, but more importantly they speak spreadsheet and forms, two of the most common dialects spoken by the average business user today. I'm closely working with both groups to better understand how they can communicate what they do to the business world, and repeatedly the examples in play are not focused on API, they are focused on solutions that make sense to the end user, but just happen to be API driven.
While much of the API design, deployment, management, and integration services and tooling that has evolved over the last five years will play a significant role in providing the API base we will need for the API economy, services like Blockspring and Form.io will bridge the critical last mile to the average user. These two companies are making APIs accessible via the most ubiquitous client out there, the spreadsheet, and helping average users understand what an API is, and what they do, through another ubiquitous interface, that almost everyone gets--the form.
I just wanted to take a moment to highlight the importance of these types of services and tools, among the wealth of other companies I am highlighting.
01 Sep 2015
One of the reasons I setup individual research projects, is to provide me with a structure for better defining each aspect of the API world, something I am working hard to jump-start within my API security research. You will notice the project does not have any building blocks defined, which when you compare with one of my oldest research areas, you start to see what I mean.
The blog posts, and other links I curate as part of my API security will help me find companies and tools that are providing value to the space. As I break down each company, and what they offer, I often have to read between the lines, in trying to understand how an API, service, or tool can be used by API providers, as well as potentially API consumers. I am looking for APIs that offer security, but also APIs that offer security to APIs--make sense?
As part of this research, I am playing with Metacert, which bills itself as a security API for mobile application developers, helping them block malicious ads, phishing links & unwanted pornography inside apps, but I think it is so much more. I could see Metacert being pretty valuable to API providers, as well as API consumers building web and mobile apps. Security isn't always about brute force attacks, and could easily just be in a simple link, added with some content, via your API.
I am adding Metacert to my API security research, with a focus on its potential to API providers. I could see API providers seamlessly integrating the Metacert API into their own stack, processing all links that are submitted through regular operations. I will also be adding link screening like this as a building block to my API security research.
If you are looking for a wise investment in the API security space, you should be talking with Metacert. APIs like Metacert provide us with a model for thinking about how we deliver API driven security services for web, mobile, and IoT applications, but it also provides a potential wholesale API layer that other APIs can use to better secure their own APIs. I consider it a strong blueprint, because its API driven, they have all the essential building blocks, which includes a monetization strategy, and they do one thing, and they do it well.
01 Sep 2015
As I make my way across my research projects, I'm learning more about how companies like Metacert can deliver valuable security services to API providers. I'm also getting a better idea of the nuance that goes into monitoring APIs, from API Fortress.
API Fortress has very interesting API monitoring story, derived from the Etsy API. This isn't the API monitoring story you'd think, it isn't about the overall stability of the Etsy API, and whether its up or down, it is about the details of API payloads, and inconsistencies they found scanning the API.
Through a payload toast of the Etsy API, the API Fortress team found an occurrence of NULL values in 3,600 out of 50,100 items scanned--which negatively affected what results that were actually accessible via the API vs the Etsy interface. Something that can result in loss of exposure for products, translating into reduced sales.
While full API outages are definitely front and center when you think about monitoring, things like this could potentially drain revenue over a long period of time, in a way that makes it very difficult to discover. I can only imagine, when you are doing things at the scale Etsy is, this kind of monitoring could plug some pretty big holes in your sales bucket.
API Fortress is already part of my API monitoring research, but just like with API security, I am going to add content analysis, and pattern matching like this as building blocks for API monitoring. After I make my way through all of the current companies I have listed in my API monitoring, I will publish the master list of common building blocks I've collected.
31 Aug 2015
I am always pleased to see API providers publishing Swagger definitions, and using them to drive their interactive documentation. Projects like the Global Change Information System API, are getting on the API definition bandwagon, and this is a good thing. I have been pushing API definition formats like Swagger and API Blueprint since 2012, but in 2015, while I want to keep on-boarding folks to the concept of using API definitions for interactive documentation, but I also want them to also understand that their APIs definition will be used in other areas of API operations as well.
Most people think Swagger is the documentation, and have not been able able to separate the specification from the interactive documentation. I think Apiary has done a better job of this separation with the separation of API Blueprint from Apiary. As an API provider, you may not have evolved to a full API-first level of operation, which is ok, but I encourage you to make your Swagger or API Blueprint definition as accessible as possible, so your API consumers can put to use in other ways--even if you don't have the time.
As soon as I saw that the Global Change Information System API employed Swagger for their documentation, the next thing I wanted to do was also use in my Postman client. While the Swagger UI provides me with a hands-on way of getting to know the GCIS API, I have to come back to the site to play with more, and it doesn't give me as much detail about how the API works, as my Postman client does. All GCIS API team has to do, is publish a text or image link to their Swagger definition in a prominent location, so it is obvious to us consumers.
I get easily reverse engineer the Swagger UI, using my Google Developer tools, but it is a couple of extra steps for me, that stands in between me, and making calls to the GCIS API in my local Postman Client. Interactive API documentation via Swagger and Apiary has significantly moved the API definition conversation forward, but just like we should be thinking beyond just language specific clients, we also need to be enabling our API consumers to speak the formats popular HTTP clients like Postman speak.
31 Aug 2015
When it comes to API security, there are a number of things to consider, something I will be be working to define, and share as part of my ongoing research. However there are three building blocks that are front and center in most security conversations--SSL, API keys, and OAuth. SSL is a must-have, and OAuth is fast becoming a must-have when there is personal data involved, but I still encounter numerous misconceptions around the role API keys actually play in security.
API key, and its accompanying secret are a common way to secure API access. You require developers to register for an account, create a new application, and they are then given an application key, plus a secret that is passed along with each API call. You cannot call the API, without passing in your API key and secret. This act alone, is what people view as the security role API keys are playing, and I get a number of questions if this is truly secure.
No it is not. Looking for two values being passed with each API call, really doesn't do that much. The actual security of your platform requires a much higher level, IT view of security (which I won't go into here), with API keys being just one tool in your security toolbox. Where keys do play a huge role, is around the awarness that is introduced, of who is accessing what, and managing how much they can access (aka mitigating how much damage is done, when there is a breach).
This short sighted views of API keys is why many companies feel they can just roll their own API management solution, and why many API developers and architects think the keys themselves possess some magical security pixie dust. The actual power comes from your awarness of the resources you are exposing via APIs, organizing into coherent service tiers, applying meaningful rate limits on these resources, and evolving your detailed awareness of who is accessing your resources, how much they are using, and when.
An API key and its companion secret offer very litle security, but the awarness they bring when you have a modern API management layer in place, can bring huge security benefits to the table. Something that will not prevent every security breach, but with the right mechanisms in place you can be alerted of breaches in real-time, and dramatically limit the extent of their damage, when they do occur.
31 Aug 2015
While there are numerous examples of APIs that successfully offer more than one option when it comes to the content types their API returns, the concept is missing from the large portion of the APIs I review. When I see good examples of this in the wild, I try to make time to showcase, and share, to help with the HTTP literacy of my readership.
One API which is rocking several fronts when it comes to API design for me, is the Global Change Information System, which interestingly is an alliance between government agencies, not something born out of Silicon Valley. When requesting resources from the GCIS API, the following HTTP accept headers are honored:
When integrating with the GCIS API, it all begins with some basic JSON, but then look at all the other linked data goodness in there! JSON-LD, RDF, Turtle!! Discovering APIs that support robust content type negotiation like this can be hard to find, most APIs don't talk about it, so even if they do, there is no way to know about it. Thankfully the GCIS API shares it via email, as well as describes in the GCIS Swagger definition, which is how I learned about it.
Content-Type negotiation is one of those HTTP literacy areas that need a lot of discussion, helping on-board API newbies to the power and possibilities that are present here, and that you can go beyond just plain ol JSON, and receive much richer API responses.
29 Aug 2015
I'm immersing myself in the Data Sharing and Open Data for Banks, published by the HM Treasury and Cabinet Office in the United Kingdom, "to explore how competition and consumer outcomes in UK banking could be affected by banks, giving customers the ability to share their transaction data with third parties using external Application Programming Interfaces (APIs)."
While I do not think an open API standard will ride into town like a white knight, saving England from all its banking problems, I do think there is a lot of good that will come from this effort. The banks have a lot of technical debt, corruption, and many other issues to deal with before APIs will even work, but I think there is enough pressure on them right now from consumers, and startups, that they will have to change at some point--or die.
As I look for evidence of movement around an open API standard online, I came across the response to the government proposal back in March. I thought there was a lot to learn from in these responses, not just for banks, but for any company that is struggling with a coherent API strategy. You can read the document yourself--here are the specific responses.
Benefits and risks of an open API standard
2.1 The first set of questions in the call for evidence invited views on the benefits and risks of an open API standard.
2.2 The vast majority of the respondents supported the development of an open API standard, and respondents agreed that there are many potential benefits of an open API standard in UK banking. The most commonly cited benefits were increased competition and innovation in banking, a greater degree of consumer choice in financial services, and the development of new and better services tailored to customers that can enhance the overall efficiency of banking. The government also repeatedly heard that an open API standard would result in customers feeling more empowered to engage with their banking or financial services.
2.3 However, many respondents also recognised that there are risks that could arise from an open API standard. One concern that was raised was the risk that an individual (to whom the data belongs) may not have meaningful control over the exchange of the data between third parties and financial institutions. Respondents from the technology industry and the banking industry cited risks around privacy of customers’ data and the potential for fraudulent use of data. In line with this, the need for appropriate security and vetting systems for third parties was regularly mentioned in the submissions received. In addition, responses received from the payments and banking industries explained that an open API standard should avoid ‘locking in’ a set of standards which cannot be enhanced or restrict banks from their own innovations, and that it should be compatible with European legislation.
Development of an open API standard
2.4 The next set of questions in the call for evidence considered the process for the development of an open API standard.
2.5 The government asked what it could do to facilitate the development and adoption of an open API standard. The majority of respondents said that that the government has a key role to play in coordinating between the banking industry and the fintech community, and driving forward the design of the open API standard, particularly in relation to the development of the standards around data security and confidentiality. Respondents predominantly from the fintech community also explained that government has a role in helping to educate customers to understand the privacy and security issues surrounding an open API standard, and the steps that can be taken to mitigate those risks.
2.6 The government questioned who should play a role in the development of an open API standard, who should be able to make use of it and how it should be used. The government received responses suggesting various different institutions could be involved in the development of an open API standard. These include banks and other financial institutions; the British Standards Institute and International Standards Organisation; the Financial Conduct Authority; the Bank of England; the Information Commissioner’s Office; the British Bankers’ Association; the Payments Council; the European Banking Authority; data security experts; software developers; fintech businesses and the government. Submissions from the fintech and technology business said that the open API standard should be available to be used by everyone, subject to the appropriate vetting procedures, and not restricted to those who banks have existing relationships with or can afford to pay for access. The banks agreed that appropriate security vetting procedures should be developed.
2.7 In response to the question on the cost of developing an open API standard, the government received a variety of estimates ranging from negligible costs to tens of millions of pounds. Several respondents from all industries explained that they were unsure of the cost of delivering an open API standard, or that it was difficult to predict how much it would cost, because much of the detail of the design of the open API standard remains to be agreed. Many respondents also highlighted that the costs were likely to vary greatly between institutions. Software and app developers, however, explained that costs may be lower than anticipated because some banks have already developed their own external APIs and so are not starting from a blank sheet of paper.
2.8 The government asked questions on how long it would take to deliver an open API standard. While there was a degree of variation in the responses, broadly respondents were in agreement that 1 to 2 years was a reasonable timescale to develop and deliver an open API standard in UK banking, although more in depth responses explained that the timescale for delivery would be clearer once the design specification of the open API standard is more defined.
2.9 The government also asked what issues would need to be considered in relation to data protection and security. Respondents highlighted that customers must at all times be in control of which third parties they grant access to their bank data and for how long, how that data can be used and the level of granularity of data that can be assessed. The banking industry and many technology firms said that third parties should be appropriately vetted before being able to access customer bank data, and a permissions list of authorisations and authentications should set out all approved third parties. One technology company added that the degree of access to customers’ bank data could be tiered according to the level of security standards met by the third party.
2.10 The government invited views on the technical requirements an open API standard should meet and adhere to. Respondents put forward a number of recommendations, the most popular options involved such requirements as HTTP, JSON, XML, OAuth, REST5 and Cloud based architecture.
I also found the government responses to be equally valuable:
3.1 It is clear from the vast majority of responses received during the call for evidence that the benefits of an open API standard are numerous and widely recognised. The key benefits that were identified by respondents were an increase in consumer choice, more competition in banking and an enhanced process, experience and outcome for consumers. Respondents strongly emphasised their support for the delivery of an open API standard in UK banking.
3.2 While much of the detail of an open API standard is still to be agreed, it is evident that an open API standard can be designed in a way which meets requirements around data protection and security, and at reasonable cost. The majority of respondents also said that developing an open API standard to a timescale of 1 to 2 years would not be unreasonable.
3.3 Respondents from the financial industry and the technology industry were aware of the risks that an open API standard could bring to consumers such as data privacy and the possibility of fraudulent use. It was made clear, however, that these risks are largely addressed through existing data protection laws, and can be mitigated through detailed planning and meticulous scoping of the open API standard design.
3.4 Submissions to the call for evidence also made clear that the publication of more open data in banking can have benefits to customers and financial institutions, and help to boost competition. The government, however, notes that steps will need to be taken to ensure the appropriate degree of data security and protection to customers is maintained.
3.5 The government is therefore committing to deliver an open API standard in UK banking, and will set out a detailed framework for the design of the open API standard by the end of 2015. The government will work closely with banks and financial technology firms to take the design work forward and, as part of those discussions, will also take forward ideas to introduce more open data in banking for the benefit of customers.
3.6 Delivering an open API standard in UK banking will help to drive more competition in banking for the benefit of customers, and enable fintech firms to make use of bank data on behalf of customers in a variety of effective and creative ways. It will ensure that the UK remains a global hub and a world-leader for financial technology and innovation.
I think 3.5 is the most important: The government is therefore committing to deliver an open API standard in UK banking, and will set out a detailed framework for the design of the open API standard by the end of 2015.
Let's get to work. I'm setting up my monitoring to track on things as they roll out, and hopefully provide a blueprint that can be employed here in the U.S. I'd love to see similar work come out of our Department of Comerce, but I guess, as with other aspects of the open data and API movement here in the U.S., we will be following a UK lead.
29 Aug 2015
Late in 2014, the HM Treasury and Cabinet Office in the United Kingdom, published Data Sharing and Open Data for Banks, "to explore how competition and consumer outcomes in UK banking could be affected by banks, giving customers the ability to share their transaction data with third parties using external Application Programming Interfaces (APIs)."
Very similar to the Obama open data mandate in May of 2012, I saw a lot of discussion shortly after the release of the document from the HM Treasury, but not many details on how things are going so far. There was a response to the document published back in March, which holds some interesting perspective, but not much else from the government, or banks, that I can find.
It is only August, I'm sure we'll see more activity this fall and into 2016, when it comes to standardizing banking APIs in the UK. I've tracked on efforts like the Open Banking Project for a while now, but I think what is potentially happening in the UK is worth me taking things up a notch or two. To help me, I will setup a banking research project, which will push up banking and related APIs in my daily and weekly monitoring, something I can review regularly, and maybe push back on some key players to get more information on where things are. Establishing an official research project really helps me find the key players, both individual and companies, which in turns increases the amount of information I gather as part of my research.
If you work for a bank that does business in the UK, or possibly for the government, I'd love to hear your thoughts, even if it is off the record. I'm hopeful for what can occur in the UK, but I'm also eyeballing the effort as a potential blueprint that the rest of the world could follow. I know the banking industry suffers from some crippling legacy technical debt, and are slowed by government regulations, but I'm pumped about the potential after reading through the Data Sharing and Open Data for Banks again--we will see a significant shift in the next five years, when it comes to how we manage our money.