My Oxford Dictionaries Talk About The World Of APIs
10 Apr 2017
This is from a conversation I had with the Oxford Dictionaries API team last week while in Oxford. I led a conversation with 30-40 folks across several teams at the Oxford University Press offices. I tried to paint a relevant and realistic picture of the world of APIs, as it would pertain to their organization. I talked for about an hour, with another hour of discussion with the group, where we discussed some of these areas in more detail.
History of APIs
To help connect the dots of where the world of APIs is going I wanted to take a brief walk through the history of APIs and make sure everyone is up to speed. The current wave of web APIs began in early 2000's with SalesForce, Amazon, and eBay leveraging web technologies to deliver commerce related IT services that leverage web technology over traditional enterprise approaches. These early API efforts focused on the integration of CRM, sales, and auction or affiliate related commerce activity, following the first dot-com bubble. Then around 2003, a new type of companies emerged that was employing APIs to connect people online and help them share links, images, and other social content, over the buying and selling of products. Flickr, then Facebook and Twitter emerged as API pioneers, opening up simple web APIs that allowed their communities of developers to integrate, and develop new websites and applications that helped drive the social evolution of the web.
By 2006, API pioneer Amazon had internalized APIs at the commerce giant, requiring internal groups to use them to conduct business internally at the company, something that would result in a new web service for storing information on the web they called Amazon S3. Shortly after they released Amazon S3 they also launched a second API for deploying and managing servers on the Internet, changing how companies can deploy infrastructure for the web--something we now collectively call cloud computing. This is when the web API thing became real for many across the tech sector, as APIs weren't just about commerce or social activities, you could actually deploy global infrastructure using APIs. Cloud computing is something that would transform how we do business on the web, changing how we delivered web applications, but would also bring IT out of the basement of the enterprise, and onto the web to operate in "the cloud"--setting the stage for the next transformation of computing occurring via mobile phones.
Web, Mobile, and Beyond
Early APIs focused on the syndication of data and content via the web. By 2009, this was changing how we think about computing, something that was greatly aided by the cloud which was put in motion by Amazon's early API efforts. In 2007, Apple introduced their new iPhone to the world, and by 2010 application developers were working hard to develop APIs that could deliver valuable resources to mobile applications, providing what was needed for applications being sold through the Itunes marketplace, and consumed via each wave of smartphones. With the introduction of the Android operating system from Google, the world of computing was clearly no longer just on our desktops, or even via our laptops--our digital bit would now be needed anywhere, in our hands, and via a growing number of Internet-connected devices.
APIs had enabled a whole new approach to delivering resources to mobile applications, as well as allowing these applications to interact with the camera, gyroscope, and other physical features of popular smartphones. This way of thinking about computing would quickly spread to other devices that we would eventually strap to our bodies, install in our homes, cars, and across the landscape of our personal and professional lives. APIs helped deliver content and data to websites, but they were also now driving mobile applications, and now the companies were realizing the potential of connecting sensors, and other devices across a variety of industries, setting the stage for what has become known as the Internet of Things. In 2017, this approach to connect everyday objects to the Internet is helping the Internet penetrate our personal and professional lives, changing how we think from healthcare to transportation. While not all of this connectivity is proving to benefit everyone involved, posing some significant security risks, it is continuing to change how we think about and engage with Internet technology in our lives.
Website + Dev Portal
Along this evolution in computing, companies were realizing that APIs weren't just the latest technological trend, and that they were the next iteration of the web, making the web something we just didn't consume, that it was also something that could be programmed, and tailored for each user or device connected to it. Similar to how companies were struggling with the need to have a website during the years between 1995 and 2000, between 2010 and 2015 many companies were realizing that they needed to have their digital resources available via APIs, accessible via a developers portal, where consumers could find what they needed to put valuable data, content, and algorithms to work, adding a wholesale layer to the web. These central portals often live alongside a companies website, changing how a company does business, opening them up to a new way of thinking if the right building blocks were present.
Some companies and startups quickly realized that there was more to all of this than just having APIs. If they also included documentation, code, and other resources, as well as a variety of communication channels and support features, some new and exciting possibilities often emerged. Providing APIs on the web via a rich, informative, and helpful portal would eliminate many of the bottlenecks and polarizing elements that traditional IT operations are often known for. When you opened up your digital assets, made them easily accessible to all stakeholders, and potential consumers, new things were possible, shifting how companies do business, and opening up new opportunities, that might not have occurred behind the corporate firewalls, and the often toxic environments that had developed within the IT environments of previous generations.
Innovation & Serendipity
API pioneers like SalesForce, Amazon, Twitter, Facebook, and others demonstrated that opening up your digital resources via APIs could enable new and exciting things to occur. In 2006, Twitter launched as a simple website where you could signup, follow people, and "tweet" out messages, six months later they launched the Twitter API and developer portal, and everything else we know as Twitter today was developed by the community. A growing number of technology startups quickly saw that APIs were enabling a new type of innovation, one that allowed 3rd parties to safely and securely use valuable digital resources and develop new web, mobile, and a variety of other integrations and applications. The API approach was fueling innovation that might never have occurred within a company and has the potential to service long tail ideas that teams might not have the time or resources to tackle.
The API approach to doing business was allowing for a form of digital serendipity to enter the picture, allowing for new and beneficial ways of putting digital resources to work. When information has only IT gatekeepers the opportunity for power struggles, and other damaging events to occur, can significantly increase. When there are self-service, yet secure access to resources, complete with identification, logging, and reporting, a much healthier balance to accessing digital resources can often be established. Those of us who are too close to digital resource management are often blinded by the immediate needs of day to day operations, and cannot see the possibilities, or even have the room to breathe and dream about what is possible. More open approaches to web APIs allow for 3rd parties who are free from the operational burdens to ideate, experiment and iterate until they find new applications that can bring some positive change to a platform and it's users.
Internal & Partners
While web APIs employ the same technology that the web operates on, and often have publicly available developer portals alongside websites, the APIs aren't always publicly available to everyone. Making corporate resources for use by partners the number one reason companies publish APIs in 2017, providing self-service, often plug and play access to data, content, and algorithms, as well as access to applications developed by 3rd part developers, and even to much-needed developer talent that is available within API ecosystems. An API-first approach allows business groups to quickly get access to the resources they need for partner engagements often without actually having to get the approval of, or wait for IT groups to weigh in, making business development, and the development of projects much more efficient, without the friction, often experienced in the past.
Amazon's experience with APIs gave us the cloud, but the way they internalized APIs within the company has famously given the rest of us a blueprint to use in thinking how API resources are put to work internally. Jeff Bezos, the Amazon CEO, mythically directed all of his employees to use APIs for internal interactions, standardizing how departments shared resources--again, bypassing classic bottlenecks introduced IT organizations. Many of the tools and applications developed on top of an API allow for even non-developers to put resources to work, allowing business groups to do more, with less, bypassing IT, and the need for highly technical talent in many scenarios. APIs are shifting how companies are doing business on the web, engaging with partners, and internally between disparate groups, setting the stage for thinking about different approaches to doing business.
Modular & Portable
The process of API design involves taking often large systems and breaking them up into smaller, more reusable chunks. The process of doing this helps us think through how we define, store, share and put our digital resources to work. Breaking up digital assets into smaller pieces is one of the things that contributes to the innovation, and reuse described previously, allowing individual resources to be put to work, without the dependencies and legacy constraints that are often introduced by their creators and operators. This modularity allows for innovation to occur at the hand of 3rd parties, but also allows for the evolution of individual resources, without having to do major system-wide upgrades, allowing for change to occur in smaller, more management chunks. This modular way of thinking is what has allowed API pioneers like Amazon and SalesForce to dominate and take the lead, as they are able to move much faster than their competition.
The modular way of thinking that APIs can bring to the table has also quickly spread to all parts of the software stack, changing how we deploy databases, our APIs, as well as the applications that are built on top of them. This evolution of API-driven cloud computing has brought us more containerized approaches to deploying and managing resources, as well as helping us develop more modular and portable versions of our applications. This modular approach to delivering architecture has helped concepts like microservices, DevOps, and continuous integration flourishes, allowing for change to occur at a more individual component level, something that is enabling us to deploy infrastructure and applications anywhere, making operations more portable. Through the standardization of containers, APIs, schema, and the other building blocks of the space, we are now able to deploy infrastructure wherever we need, in any cloud, and on any device.
Automation & iPaaS
It is commonplace to think about 3rd party developers building the web and mobile applications on top of APIs, but it takes more work, imagination, and exploration to understand the potential for automation and integration using a new breed of integration platform as a service (iPaaS) platforms. This is another area that APIs are empowering non-developers to put APIs to work, bypassing IT and the need for more specialized technical talent. This is another area that is proving difficult for many API providers to think about when it comes to what iPaaS providers like Zapier bring to the table, and how to target complementary APIs, process, and workflows that would benefit business operations. Integration is often perceived as a resource intensive, and costly IT project, but this new breed of platforms make integration something anyone can achieve, with projects that can range from a permanent implementation to a solution that is more ephemeral, going away after just a couple of hours, or after a single event occurs.
Most companies encourage developers to make a request for data, content, and algorithmic behavior using their APIs, but the providers who are further along in their API journey don't rely on APIs just being a one-way street, and have begun employing approaches like using webhooks to make API engagement a two-way street. Webhooks allow for other systems to be called via APIs when specific events and circumstances have occurred. The primary need for doing this is to alleviate the number of API calls that developers will have to make on any single API, but the secondary motivation is often about the orchestration and automation using API resources we depend on. Webhooks allow API providers and consumers to automate common business tasks, scheduling them to occur on a regular basis, or being triggered when a specific situation occurs, making the web much more automated and working for whoever is in defining orchestration.
Messaging & Bots
APIs are driving development across a variety of business verticals, and one of those sectors that are being changed in some interesting ways is messaging. Enabling users to communicate via a variety of API driven messaging protocols is at the heart of successful platforms like Facebook Twitter and Slack. These API providers are leading discussions when it comes to enabling users to collaborate and communicate while also encouraging developers to get involved using their APIs. An example of the automation described above can be seen in the latest wave of bots to emerge via Facebook messenger, as well as the Slack messaging platform. Bots are small, automated, API-driven scripts that engage with users via messaging platforms, exposing data, content, and algorithms in a conversational format, and hopefully augmenting human interactions in a meaningful way.
Bos are emerging to help users answer common questions, automate tasks, make payments, receive analysis, and many other digital aspects of our professional lives. Slack is leading the charge when it comes to business use cases, but Twitter and Facebook are also fueling the potential of bots when it comes to influencing the news, stock markets, elections, or just providing entertainment by delivering poetry, jokes, or funny pictures and memes. Messaging between humans is not new, and messaging between systems within code is also nothing new, but we are now seeing a convergence of these two worlds fueling human to computer interactions at a scale we haven't' experienced before, with (some) interesting outcomes. Like the web and mobile, bots need APIs to deliver the data and content they need to engage with users and emulate human behavior via this new wave of conversational interfaces.
Voice & Conversational
Similar to the messaging bot evolution discussed previously, there is another layer of API driven conversations going on, but this time is leveraging actual voice technologies, allowing humans to talk with the web and other online, API-driven systems. Voice enablement isn't anything new, but with the increased availability of API resources, and the advancement of voice technology like Siri and Alexa, having a conversation with a machine is becoming more common. The latest breed of voice solutions translate voice recordings into text, send that text to a variety of API driven systems, and then translate the response back into a familiar voice that responds to humans via their Internet-connected devices. The availability of cloud resources has significantly contributed to this latest explosion in voice-enablement, making it no surprise that Amazon is leading the charge with their Alexa enabled API ecosystem.
Voice enablement is another area that API providers, who are further along in their API journey are making their APIs available for, crafting endpoints specifically for use in voice, and studying how end users are putting voice technologies to work at home, in automobiles, and in the workplace. Like iterating API operations for mobile usage, conversational interfaces will require the iteration of API resources, something a modular approach to design will aid considerably. Web APIs have gone a long way to simplifying some often very complex back-end systems, making them more intuitive and usable by end-users. This is something that will help determine which APIs are usable in a conversational environment, requiring APIs to be able to speak more fluently to humans, but it will be a fine balance because in order to scale properly they also need to be machine readable as well.
Artificial Intelligence & Machine Learning
Historically, most APIs have been about sending and receiving data or content, with a handful of APIs providing access to more algorithmically focused resources. While these APIs do have data inputs, they are more about applying some sort of computational solution, with data just being the byproduct. This approach to delivering API resources is seeing the significant resurgence with the evolution a growing number of what services that are being labeled Artificial intelligence (AI) or Machine Learning (ML). While the hype around these solutions often exceeds the reality, some of these API resources are providing some very valuable computational resources that can be used in everyday tasks. There are numerous startups emerging to cater to this new wave of Ai and ML solutions, with technology giants like Amazon, Google, Microsoft, and even IBM investing heavily so that they can compete.
The artificial intelligence and machine learning APIs often seem magical and mysterious as they operate in a black box, but the solutions they provide range from practical solutions like sentiment analysis and categorization of text, to image transformation, and object or facial recognition in photos. This is just a small subset of what is possible with artificial intelligence and machine learning, with many specialized solutions emerging for using in specific industries like agriculture, policing, or healthcare. APIs are being used to deliver artificial intelligence and machine learning solutions, by wrapping these algorithms in web technology, making them accessible for use in web, mobile, and device applications. In the world of artificial intelligence and machine learning, there is an often missed opportunity by API providers when it comes to augmenting and complementing artificial intelligence and machine learning APIs with other data and content APIs, allowing for much richer, meaningful, and contextually relevant responses.
Analysis & Awareness
Using APIs can deliver a heightened level of awareness of a company or organization's digital assets, opening them up in a variety of ways. By employing modern approaches to API management, where all API requests and responses are logged, and all applications and their users are identified, an entirely new level of analysis and surveillance is possible regarding how digital resources are being put to use, or possibly not put to use. Companies who are further along in their API journey have come to realize that in addition to have APIs valuable for business operations, it is valuable to also craft APIs on top of the API operations as well, adding an entirely new dimension to how APIs can be put to use across aa platform. This approach to API deployment has opened up entirely new business models, taking the innovation explored earlier to new levels, driving home the opportunity for serendipity to occur across operations.
This newfound awareness (sometimes turning to surveillance) across operations is the number one benefit of doing APIs. Going API-first helps map out digital resources while making them more accessible, and reusable, and when reporting is properly applied across the logging that is common across API operations, the analysis opportunities can be just as compelling as the core APIs themselves. Understanding how resources are used, what is of most interest to users, and what types of applications are making the biggest impact rise to the top. Because this layer is API driven as well, the same approach to integration, iteration and innovation can occur as other areas of the ecosystem. This layer API access should be carefully controlled, and may or may not be available to the public, and often only available internally, or to a select group of partners, making the analysis and awareness much more guarded, so that it protects the privacy and security of end-users, as well as platform operators and developers.
Visualizations & Embeddable
Whether it's directly from core API resources, or the operational level of APIs, building off the modular and portable approach available with modern API architecture, one of the most valuable applications within an ecosystem is centered around the ability to visualize, and deliver embedding solutions on the web, mobile, in emails, and other applications. A dashboard is a common way to look at this, or possibly API-riven infographics, but the savviest API providers are developing a wealth of API embeddable that leverage open source solutions such as D3.js to make everything visual and embeddable. When this approach accompanies a branding strategy, it can extend the value of the platform beyond the domain, extending the reach of the ecosystem. While often centered around more visual elements of the platform, embeddable tools can also be more functional and interactive, bringing another dimension of benefit to operations.
Embeddable tooling comes in all shapes and sizes, with the best solutions adhering to web standards, and leaving proprietary approaches behind the firewall. The most well-known examples of this in action can be found with Twitter and Facebook share buttons, or with the Youtube video player which extends the video platform on sites across the web. Embeddables do not have to be developed by the platform alone and can become a community effort, showcasing simple, copy and pastable solutions crafted by an APIs developer within the ecosystem. The first layer of API-driven embeddable should speak to the value delivered by core APIs, with the second layer of embeddable taking advantage of the programmable web, and making the experience more interactive for consumers, generating content, and data--bringing value to the platform that can contribute to the analysis and awareness layer of API operations discussed previously.
Evangelism & Storytelling
APIs begin with a technological seed but they need to make an impact at the human level if they are to find success, making the single most important tool in the toolbox storytelling, advocacy, and evangelism around what an API does. If people do not know an API exists, or what is possible with, none of this matters. API operations often suffer from the same search engine optimization (SEO) challenges that regular website efforts do. For an API to stand out and be found on the web it takes a great deal of work telling stories about API operations, and the applications that are being build on top of them. This storytelling needs to be included as part of existing marketing strategies, but will also require a special touch for it to pass the inspection of an often very picky developer community, who tend to be a slightly different beast when it comes to getting the attention of, and keep them engaged.
Storytelling and evangelism have to be genuine. It has to speak to the value an API brings to the table, and reflect the solutions that developers and integrators will be looking to provide. This outreach isn't just a public effort, it should include public consumers, as well as partners, and internal users. Ideally, API outreach has a dedicated team that can craft a message that speaks to the platform, but also respond to support operations, and be intimately aware of what is happening within the ecosystem, establishing and maintaining a deep relationship with the community. If outreach is going to be effective, it has to be relevant to the operations, match the technical and other support challenges that developers are facing--possessing strong context for the community. Finding a voice, establishing, and maintaining a true presence for an API is the number challenge I see API platform's struggle with, and is the number one contributor to their demise.
Establishing a Presence
There are thousands of public APIs available on the web, a number that is only growing, so standing out takes a significant amount of work. 75% of the API providers I encounter suffer from the "build it and they will come" disease, and think developers will just find them, and understand the value an API delivers. This is why you see a handful of blog posts on many APIs, a couple months of Tweets, and then radio silence. The successful APIs out there have sustained operations, with robust storytelling, evangelism and outreach both on and offline. This goes for APIs that aren't 100% public too. If you want to reach partners, or even make internal groups aware across large, and often global organizations, evangelism is essential. Establishing a presence is not a one time thing, it take months, and years to do properly, and will require constant investment and maintenance.
The most efficient way to establish a presence is by being transparent and observable in everything you do. Adding new features to the road map? Publish them online, blog and tweet about them. Get the community involved. Ask for feedback, follow and retweet your users. Get to know them on the platforms and in the channels they already exist on. Be a regular presence, but not one that is always preaching and selling. Make sure you listen, study, and understand your target audience, and speak to what they'll need. Even when a developer does learn about an API there is no guarantees they will have an immediately pressing need and put it to use right away. A fine tuned API presence will ensure that potential consumers are aware that an API exists, and is constantly being reminded that it exists by blogging, Tweeting, storytelling, outreach, and engagement--when done properly developers will know where to go when they have a problem to solve.
Repeat, Repeat, Repeat
I have been doing API Evangelist for seven years this summer. Success has been difficult. It has taken a regular drumbeat to stand out in the crowd. It has required some difficult decisions regarding who I partner with, and give access to my resources and brand so that I maintain my desired level of trust amongst my consumers. While I'm not evangelizing a single API, I am helping cultivate the awareness of a variety of APIs, as well as the best practices across these API operations--the process is the same for individual API providers. Often times I'm repeating what I have written before, and giving similar talks at conferences, meetups, and presentations like this. However, I am always iterating and evolving based on feedback I receive, and how my work is applied (or not), something that can be very painful to move forward on some days. The creativity doesn't always come right away, and sometimes I have to just to maintenance work just to keep busy, and come back to my storytelling when it feels right.
The most important part of what I'm doing is that I'm consistently out there, telling stories online, and in person. Early on in my presence, it was critical for me to be at public events shaking hands and making the in-person connections. After three years of doing a majority of in-person events I'm able to do more of it online, than I do offline, but I still have to be blogging, tweeting, and committing regularly. This is how I've established the presence that I have, and made the connection with people that I have. When someone is ready to do APIs or ready for the next step in their API operations, they know where to find me, and the resources that I provide. This is the core of any API operation, my world is no different than yours. I guarantee I have way less budget, and time than you, but somehow I've found a way to keep going. I have also had several times where I faced burnout, but I find if I just keep pushing forward, creating, writing, developing, and iterating I get through it--the solution is always repeat, repeat, and repeat until I find the right path forward.
While this talk was designed just for my conversation with the Oxford University Press team, it is generic enough for any other company to consider, and use to learn about the API space. I worked pretty hard to keep it a pretty positive view of the API landscape, and left most of the realities of this world for the in-person conversation, but is also something I'll cover in future posts this week. Honestly, when working with organizations like the Oxford University Press, I am left pretty optimistic about this whole API thing--something that is increasingly difficult with some of the ways that I am seeing APIs be wielded in 2017.