This is one of my talks from APIDays Paris 2018. Here is the abstract: Understanding the PSD2 regulations unfolding for the banking industry is a daunting challenge for banks, aggregators, and regulators, let alone when you are an average user, journalist, or analyst trying to make sense of things. As the API Evangelist I have used a common approach to mapping out transit systems using the universally recognized subway map introduced by Harry Beck in London in the early 20th century. I’m taking this mapping technique and applying it to the PSD2 specification, helping provide a visual, and interactive way to navigating this new regulatory world unfolding in Europe. Join me for an exploration of API regulations in the banking industry, and how we can use a subway and transit map to help us quickly understand the complexities of PSD2, and learn what takes to get up and running with this new definition for the API economy.
Using Transit As API Analogy
I’ve been fascinated with transit systems and subway maps for most of my adult life. Recently I’ve begin thinking about the parallels between complex municipal transit systems, and complex API infrastructure. Beginning in 2015, I started working to see if I could apply the concept of a subway map to the world of APIs, not just as a visualization of complex API systems, but as a working interface to discover, learn about, and even govern hundreds, or even thousands of APIs working in concert. While very much a physical world thing, transit systems still share many common elements with API infrastructure, spanning multiple disparate systems, operated by disparate authorities, possessing legacy as well as modern infrastructure, and are used by many people for work and in their personal lives.
While the transit system isn’t a perfect analogy for modern API infrastructure, there is enough there to keep me working to find a way to apply the concept, and specifically the transit map to helping us make sense of our API-driven systems. I’ve called my work API Transit, leaning on both the noun definition of transit, “the carrying of people, goods, or materials from one place to another”, as well as the verb, “pass across or through (an area)”. The noun portion reflects the moving of our digital bits around the web using APIs, while the verb dimension helps us understand what is often called the API lifecycle, but more importantly the governance that each API should pass through as it matures, and continues to move our important digital bits around.
The History of Subway Maps
The modern approach to mapping transit systems can be traced back to Henry Beck who created the now iconic map of the London Underground in 1933. The methodology was decoupled from earlier ways of mapping and communicating around transit resources, in a way that was focused on how the resources would be experienced by end-users, evolving beyond a focus on the transit resources themselves, and their location in our physical world. At the beginning of the 20th century, subway maps were still being plotted along roads and rivers, leaving them coupled to legacy elements, and ways of thought. By 1915, public transit engineers like Henry Beck were beginning to rethink how they mapped the transit infrastructure, in a way that helped them better communicate their increasingly complex infrastructure internally, but most importantly, in a way that helped them communicate externally to the public.
This is what I’m looking to do with applying 20th century transit infrastructure mapping while applying it to 21st century digital API transit infrastructure. We are needing to get people(s) bits, and digital goods and materials around the web, while also understanding how to develop, discover, operate and manage this infrastructure in a consistent, and intuitive way. However, API infrastructure is rapidly growing, and with the introduction of microservices becoming increasingly complex, and difficult to logically map out the increasingly evolving, shifting, and moving API landscape. I’m hoping to suspend reality a little bit as Henry Beck did, and take utilize the same visual cues to begin to visualize API infrastructure in a way that is familiar to not just developers, but hopefully regulators, and average consumers. Much like the transit system of any major city, initially an API transit system will seem overwhelming, and confusing, but with time and experience it will come into focus, and become a natural part of our daily lives.
API Transit Applied To Governance
Over the last two years I have been actively working to apply the API transit model to the concept of API governance. Beginning with API design, I have been looking for a way to help us understand how to consistently craft APIs that meet a certain level of quality set by a team, company, or even an entire industry. I’ve pushed this definition to include many of the almost 100 stops along the API lifecycle I track on as part of my work as the API Evangelist. Allowing API governance transit maps to be crafted, which can be used to help understand API governance efforts, but also help be applied to actually executing against this governance framework. Delivering the verb dimension of API transit, to “pass across or through (an area)”, allowing each API or microservice to regularly pass through each area, line, or stop of each API Transit governance map.
API Transit began as a way to define and visualize the API lifecycle for me, acknowledging that the lifecycle is rarely ever a straight line. Often times it begins as that, but then over time it can become a mashup of different areas, lines, and stops that will need to be realized on a regular basis, on a one time basis, or any other erratic combination our operations can conceive of. With API Transit applied as an API governance model, I wanted to push the analogy even further, and see if I can push it to accommodate an individual API definition, or even a complex set of microservices working in concert. I began playing with a variety of existing API implementations, looking for one that could be used to tell a bigger story of how the transit model could be use beyond mapping how buses, trains, and other transit vehicles operate.
API Transit For PSD2 Landscape
To push the API transit concept further, I took the Payment Services Directive 2 (PSD2) governing banking APIs in Europe, to see what might be possible when it comes to mapping out API infrastructure in a impactful way. I was able to easily map out lines for some of the most common aspects of PSD2, visualizing accounts, transactions, customers, and then eventually every other stop I wanted to include. The trick now, is how to I articulate shared stops, transfer stations, and other common patterns that exist as part of the PSD2 specification. I didn’t want to just visualize the banking regulations, I want to create an interactive visualization that can be experienced by developers, regulators, and even end-users.
The HTML Canvas transit map solution I’m using allows me to plot each line, and stop, and give each stop a label as well as a link. This allows me to provide much more detail about each stop, allowing someone exploring the specification to obtain much more information about each aspect of how the PSD2 specification is put to use. The resulting API transit map for the PSD2 landscape is pretty crude, and not very attractive. It will take much more work to bring each line in alignment, and allowing for overlap and redundancy to exist across lines. The reasons behind each line and stop will be different than a physical transit map, but there are still existing constraints I need to consider as a craft each transit map. The process is very rewarding, and helps me better understand each intimate detail of the PSD2 specification, something I’m hoping will eventually be passed on visually, and experimentally to each map user.
API Transit Is OpenAPI Defined
As part of my rendering of the API Transit map for the PSD2 landscape, I was able to generate the HTML5 Canvas using the OpenAPI for the PSD2 API. Each line was created by the tags applied across APIs, and each stop was created to reflect each individual API method available. Providing a machine readable set of instructions to quantify what the structure of the landscape should be. I feel that the tagging of each API method can be improved, and used as the key driver of the how each API transit map is rendered. Something that will be used to make sense of complex renderings, as well as deliver on much more simplified, distilled versions to help make the landscape more accessible to non-developers.
The details present as part of each PSD2 API path, method, parameters, responses, and schema can be rendered as part of the information provided with each stop. Providing a clear, but detailed representation of each stop within the API Transit system, from a machine readable core which will be used to make the API Transit map more interactive, as well as part of a larger continuous deployment and integration experience. While each stop along each PSD2 line will not always be linear in nature, this approach allows for stops to be organized in a meaningful order, driven by a set of common machine readable rules that can be versioned, evolved over time, and used throughout the API Transit system, and tooling that is deployed to keep the API Transit system operational.
API Transit Has Hypermedia Engine
With the ability to map out an overall API governance system, as well as a set of microservices that meet a specific industry objective, I needed a way to put in all into motion, so it could work together as a complete API transit system. I chose hypermedia to be the engine for the interaction layer, and OpenAPI to be the definition for each individual API. Hypermedia is used as a scaffolding for each area, line, and stop along the transit system, allowing each API to be navigated, but in a way that each API could also navigate any overall API governance model. Introducing the concept that an API could be experienced by any human, and any API governance model could be experienced by an API–again, allowing it to “pass across or through (an area)”, thus API Transit.
Each API has a machine readable OpenAPI definition allowing requests and responses to be made against the surface area of the API as it is explored via the API Transit experience, much like interactive documentation provides already. However, the same OpenAPI definition allows the API to also be validated against overall API governance practices, ensuring certain design standards are met. I’m also playing with the use a machine readable APIs.json index for each API, in addition to the OpenAPI definition, allowing governance to be easily expanded to other API governance lines like management, documentation, pricing, support, terms of service, and other aspects of API operations not defined in an OpenAPI, but could be indexed within an APIs.json document.
API Transit Runs As Github Repository
To continue evolving on the API Transit experience, I’ve begun deploying the API Transit map, with OpenAPI core, and hypermedia engine to Github for continuous deployment, and hosting of the API Transit map using Github Pages. Github, combined with Github Pages, and the static CMS solution Jekyll introduces an interesting opportunity for bringing each API Transit map to life. Jekyll transforms the hypermedia engine, and each API definition into a collection of objects which can be referenced with the API Transit map using the Liquid syntax. This allows each area, line, and stop to be easily traveled, moving from stop to stop, transferring between lines, and consuming rich information all along the way.
Running each API Transit map makes exploring an interactive experience, but it also makes the experience easily part of any continuous integration or deployment pipeline, using Git or the Github API to engage with, or evolve the API Transit definition. This makes each API definition seamlessly integrated with actual deployment workflows, while also making API governance also part of these workflows, helping realize API testing, monitoring, validation, and other elements of the build process. With seamless integration as part of existing continuous deployment and integration workflows, I can easily envision a future where API Transit maps aren’t just interactive, but they are continually changing, streaming, and visually updated in real time.
Each Stop Filled With Information
Each stop along an API Transit line is defined by its OpenAPI path, and navigated to by the hypermedia engine, but that is just the beginning. The hypermedia object for each stop contains the basics like the name and description of the stop, as well as the reference to the path, method, parameters, responses, and schema in the OpenAPI, but it also can possess other fields, images, video, audio, and links that deliver rich information about each individual stop. The hypermedia engine allows us to define and evolve these over time without breaking the client, which is the API Transit map. Providing an huge opportunity to educate as well as implement governance at each stop along the API Transit line(s).
Since a stop can be along an API governance line, or along an individual API line, the opportunity for education is enormous. We can educate developers about healthy API design practices, or we can educate a developer about a specific API method, where the healthy API design practices can be fully realized. With this approach to using Github Pages, Jekyll, and a hypermedia engine as an interactive CMS, there is a pretty significant opportunity to make the API Transit experience educational and informative for anyone exploring. Each developer, aggregator, regulator, and even end-users can take control over which aspects of the PSD2 specification they wish to engage with and learn about, allowing for the PSDS API Transit to serve as wide of an audience possible.
Map Progress Of Individual Banks
While there is an OpenAPI, and now API Transit map for the overall PSD2 specification. I will be using the approach to track on the evolution of each individual bank in Europe being impacted by the regulation. Ideally each bank will actively maintain and publish their own OpenAPI definition, something I will be actively advocating for, but ultimately I know the way things end up working in the real world, and I’ll have to maintain an OpenAPI definition for many of the banks, scraping them from the documentation each bank provides, and SDKs available on Github. I will use this as a literal map of the progress for each individual bank, and how compliant they are with EU regulation.
Each individual bank PSD2 API Transit map will allow me to navigate the APIs for each bank, including URLs, and other information that is unique to each provider. I will be using APIs.json to track index API operations, which I will work to bring out as visual elements through the API Transit map experience. In theory, each banks APIs will be an exact copy of the master PSD2 specification, but I’m guessing more often than not, when you overlay the master PSD2 API Transit with each individual bank’s API Transit map, you are going to see some differences. Some differences will be positive, but most will demonstrate deficiencies and deviation from the overall guidance provided by EU regulatory agencies.
Compare Over PSD2 To Individual Banks
I am using the hypermedia engine for the API Transit map to crawl each individual banks implementation to deliver the overlap map I discussed. I’m working on a way to show the differences between the master PSD2 specification, and one or many individual PSD2 API implementations. I’m not just looking to create an overlay map for visual reference, I will actually crawl the OpenAPI for each individual bank and output a checklist of which paths, parameters, and schema they do not support, ensuring their definition matches the master PSD2 specification. The first approach to doing this will be about comparing OpenAPI definitions, with a second dimension actually running assertions against the API for those that I have access to.
There will be too much information to show on a single API Transit map, especially if we are comparing multiple banks at once. There will need to be a report format, accompanying a PSD2 API Transit map overlay showing the differences at the overall line and stop level. The programmatic comparison between the master PSD2 API and each of the banks will be easy to accomplish using the hypermedia engine, as well as the detailed OpenAPI definition. What will prove to be more challenging, is to create a meaningful representation of that map that allows the differences to be visualized, as well as the detail that is missing or present to be navigated via an aggregate Github repository.
Apply Governance To Individual Banks
In addition to comparing the OpenAPI definitions of individual banks against the master PSD2 OpenAPI definition, I’m using the API Transit hypermedia engine to compare each individual API against the overall API governance. At first the API governance will just be about the API design elements present in the OpenAPI definition for each bank’s API. I will be looking for overall design considerations that may or may not be present in the master PSD2 specification, as well as beginning to add in additional API governance areas like testing, monitoring, performance, documentation, and other areas I mentioned earlier, which will be tracked via an APIs.json index.
There are a number of details I’d like to see banks provide that isn’t covered as part of the PSD2 guidance. Having an API is only the beginning of usable banking solution for 3rd party applications. There needs to be documentation, support, and other elements present, or a well designed, and PSD2 compliant doesn’t mean much. I’m using API Transit to take the governance of PSD2 implementations beyond what the EU has put into motion. Taking what I know from the wider API sector and getting to work to map out, validate, and rank banking APIs regarding how easy to use, responsive, and usable their PSD2 APIs are. Making sure 3rd party developers, aggregators, and regulators can get at the APIs that are supposed to be defining their compliance.
Use to Map Out Countries & Regions
Using API Transit, I will be aso zooming out and mapping out the PSD2 landscape for individual countries, and regions. Creating separate transit maps that allow each country to be navigated, using the API Transit hypermedia engine to crawl each banks API Transit system map, and eventually provide aggregate comparison tools to see how different banks compare. With a machine readable index of each country, as well as individual indexes of each bank, the entire PSD2 regulatory landscape will become explorable via a series of API transit maps, as well as programmatically using the hypermedia engine present for each individual bank API implementation.
API Transit is meant to help map out the complexity of individual API systems, as well as the overall API governance in place. It will also enable the mapping of the complexity of many implementations across cities, regions, and countries. Allowing for regulatory tooling to eventually be developed on top of the thousands of Github repositories that will be created, housing each individual banks API Transit maps. Leveraging the Github API, Git, and the hypermedia and OpenAPI core of each individual API Transit implementation. Allowing for searching, reporting, auditing, and other essential aspects of making sure PSD2 is working.
Making Sense Of Complex API Landscapes
The goal of API Transit is to make sense of complex API landscape. Tracking the details of thousands of API operations across the EU countries who are required to be compliant in 2018. Providing machine readable blueprints of the regulation, as well as each individual bank that is required to be compliant, and each individual detail of what that compliance means. Then tying it all together with a hypermedia engine that allows it to be explored by any system or tooling, with a familiar transit map on top of it all–providing a way for humans to explore and navigate this very complex banking API landscape.
As I am evolving the process of creating API Transit maps, the API Transit hypermedia engine allows for the development of a gradual awareness of a very complex and technical landscape. It all feels the same as when I land in any new city and begin understand the local transit system. At first, I am overwhelmed, and confused, but the more time I spend exploring the transit maps, riding the subway and buses, developing a familiarity with each system, the easier it all becomes. There is a reason transit maps are used to provide access to complex municipal transit systems, as they help provide a comprehensive view of the landscape, that allows you to find your way around, and use as a reference as you develop an awareness–this is what I’m looking to build with API Transit for the banking sector.
Tours For Individual Roles
One of the benefits of an API Transit map client that is built on a hypermedia engine is that you can take advantage of the ability to change the map, and experience depending on who you are targeting. The concept of the transit maps allows us to suspend realities of the physical world, and the hypermedia client for the API Transit map can suspend realities of the virtual world, allowing for the creation of unique tours and experiences depending on who the end user is. The complete API Transit map for each bank’s implementation will still exist, but the ability to suspend some details to distill down complexity into more simpler experiences will be possible.
The objective of individual API Transit tours and experiences will allow for onboarding new developers, business users, or possibly regulator oversight looking at just a specific dimension of the PSD2 guidance. Allowing for the ability to zoom in just at the customer experience layer, or possible just the transactional capabilities across many banking API providers or even a city, region, or country. Think of how a city might have separate transit maps for the subway, buses, and other systems, but still have the overall system map. We can do the same with API Transit, further reducing complexity, and helping users make sense of the complex banking API systems.
Interactive Industry API Governance
For API governance to be effective it needs to be understood by everyone being impacted, and are required to participate. It has to be able to measured, quantified, tested, audited, and visualized in real time, to understand how it is working, and not working. Modern API operations excel at doing this at the company, organization, institutional, and government level, and we should be using API solutions and tooling to help us understand how governance is being applied. There should be machine readable definitions, and common media types, as well as API driven tooling for educating, implementing, and measuring API governance–completing the API loop.
This is another reason why the transit analogy is working so well for me when it comes to making API governance an interactive experience. I’m able to develop not just linear experiences where participants can click next, next, next and walk through everything. They can choose their own paths, experience the lines they feel need attention, while allowing managers, auditors, and regulators to understand how well each implementation is responding, evolving, and interacting with regulations. Making API governance at the industry level interactive not just for the banks and the EU regulatory body, but also for every other participant in between trying to make sense of what is happening with PSD2 in 2018.
Continuously Deployed And Integrated
As I mentioned before, API Transit runs 100% on Github, within a Github repository. The API Transit map client is unaware of each individual banks API implementation. The uniqueness for each implementation resides in a series of Siren hypermedia, OpenAPI, and APIs.json files, that provide a machine readable snapshot of the implementation. The combination of Github, and the machine readable YAML core of each API Transit instance makes it all able to be integrated into continuous integration and deployment pipelines. Evolving each API Transit instance, as each actual banks API implementation is built and released–hopefully ensuring they are kept in sync with the reality on the ground at each bank.
At this point, API Transit has become another application being developed upon the banking APIs that PSD2 are designed to expose. It ultimately will be an aggregated application that uses not just one banking API, but all the banking APIs. The difference from other aggregators is API Transit is not interested in the data each bank possesses. It is interested in continuously understanding how well each banks API is when it comes to complying with PSD2 regulations, while also continuously helping developers, aggregators, regulators, and anyone else make sense of the complexities of banking APIs and the PSD2 regulations.
Mapping Technology, Business, & Politics
While API Transit seems very technical at first look, the solution is meant to help make sense of the business and politics around the PSD2 rollout. The first part of the conversation might seem like it is about ensuring each bank has a compliant API, but it is more about ensuring they have common operational level components of API operations like a developer portal, documentation, support, and other business aspects of doing business with APIs. Next, it will be about ensuring all the technical aspects of PSD2 compliant APIs in place, validating each API path, as well as it’s responses, schema, and other components.
While we need to make sure 100% of the PSD2 OpenAPI definition is represented with each bank, we also need to make sure that API is accessible, secure, and usable, otherwise none of this matters. If a bank plays politics by making their API unusable to aggregators and 3rd party developers, yet appear on the surface to have a functioning API that is compliant with the PSD2 specification, we need to be able to identify that this is the case, make sure we are testing for these scenarios on a recurring basis, and be able to visualize and report upon it to regulators. Properly addressing the technical, as well as the business, and politics of API operations.
Making API Regulation More Familiar
The main reason I’m using the transit map approach is because it is familiar. Secondarily, because the concept continues to work after two years of pushing forward the concept. Everyone I have shown it to immediately responds with, “well the trick will be to not make things too overwhelming and complicated”. Which I response with, “do you remember the first time you used the transit system in your city? How did you feel?” Overwhelmed, and thinking this was complicated–because it is. However, because there is a familiar, universal map that helps you navigate this complexity, eventually you begin to learn more about the complexity of the city you live in, or are visiting. Helping bridge the complexity, making it all a little more familiar over time.
The transit map concept is universal and familiar. It is in use across hundreds of transit systems around the globe. The transit map is familiar, but it also has the ability to help make a complex system more familiar with a little exploration, and over the course of time. This is why I’m extending the transit approach to the world of APIs, and using it to make complex API systems like the PSD2 banking API regulation more familiar. Banking APIs aren’t easy. Banking regulations aren’t easy. We need a map to help us navigate this world, and be able to make sense of the complexity, even as it continues to evolve over time–API Transit is that.
Helping API Regulation Be More Consistent
API Transit is machine readable, driven by existing specifications, including Siren a hypermedia media type, OpenAPI, and APIs.json. The first machine readable definition I started with was the OpenAPI definition for the PSD2 regulation. This was the seed, then I generated the hypermedia engine from that, and took the API governance engine I had been working on for the last couple of years and built it into the existing API Transit core. The objective of all of this work is to use the framework to introduce more consistency into how I am mapping out the rollout of PSD2 across the European banking landscape. This is something that is too big to do manually, and something that requires a machine readable, API driven approach to get the job done properly.
The PSD2 OpenAPI definition provides a way to consistently measure, and report on whether each banks APIs are in compliance with PSD2 regulations. The APIs.json is meant to make sure each bank’s APIs are in compliance with API Evangelists guidelines for API operations. The Siren hypermedia engine is meant to enable the consistent crawling and auditing of everything as it evolves over time by other systems and applications, and when combined with the HTML5 Canvas API Transit map, it allows humans to explore, navigate, and visual banking APIs, and the PSD2 regulations in a more consistent, and organized fashion.
Seeing PSD2 In Motion In A Visual Way
Ultimately, API Transit is about being able to see API operations across an industry in a visual way, where you can see everything in motion. Applying it to the PSD2 landscape is meant to help visualize and understand everything happening. It has taken me many hours of work to get intimate with the core OpenAPI definition for PSD2. Learning about all the paths and schema that make up the regulation. As I work to evaluate the compliance of hundreds of banks across France and UK, I’m quickly realizing that it will be beyond my capacity to be able to see everything that is going on. API Transit helps me see the PSD2 landscape in a way that I can be regularly updated, revisited, and experienced as the landscape evolves.
The PSD2 API Transit application is just getting started. The methodology has been proven, and I’ve begun profiling banks, and collecting OpenAPI definitions for any APIs that I find, and profiling the wider presence of their API operations using APIs.json. Then I will begin improving on the mapping capabilities of the API Transit map client, making for a more visually pleasing experience. Along the way I am working with partners to better monitor the availability and performance of each banks APIs. Within a couple of months I expect a clearer picture to begin to come into focus regarding how the PSD2 landscape is unfolding in 2018, and by the end of the year, it will become clear which banks have emerged as competitive leaders within the API economy.
