I am reworking the API Evangelist developer area, and shifting most of my content to be available as YAML and JSON data on the Github repositories that drive my network of sites. I'm doing this partly because I am not in the business of managing and growing an API community, and because there are some really badly behaved people out there that I'm just not really interested in having keys to my internal network. I am happy to open up read only access to my work publicly and write capabilities to my trusted partners, but having self-service access to my server(s) just isn't fun in the current online climate.
I get it when folks want to keep their valuable data, content, and algorithm under lock and key, and require developers to build a relationship before they get access or increased levels of consumption. However, this hasn't always been the tune I'm whistling. There are plenty of examples of me telling API providers to provide self-service access to their resources in the past--well, we've fucked that off, with our bad behavior as API consumers. It's not that the concept won't work, it is just that it won't work with the number of assholes on the web these days it just isn't a good idea.
Even with keeping the lock and key on our API resources, we can still be public with our API operations--there are many positive reasons for doing so. SEO is probably the first, I mean how are people going to find your APIs if you can't Google for them. Providing information to the press, and making the resources that support your APIs self-service can reduce the workload when you do give people access. Some examples of this can be found when looking at the Target vs. Pandora developer programs, both required approval to get access, but Target is much more open than Pandora with their overall story.
Target really doesn't have that much more than Pandora does, but they have a blog and link to their Github, and of course their jobs page. I'm still going to publish the documentation for my own APIs, and go further than Target has, but I have to give them credit for being at least a little more creative, and public than Pandora. Like I said I won't be pushing back on providers when they do not have self-service public access to their API resources anymore, but I will be encouraging y'all to be creative when thinking about the public presence of your operations and engaging in some meaningful storytelling via a publicly available portal.
Talking to people, and telling stories on a regular basis always pushes me to evolve my understanding of how people see (or don't see) APIs, and pushes me to keep shifting the way I tell stories. I've always felt that that education about APIs should be relevant to the users, but usually this centers around making it familiar, and speak to whoever I am helping onboard. After talking to some folks recently at @APIStrat, I'm adding to my thinking on this, and focusing on making my stories more precise for folks I talk with.
One of the reasons I really like APIs is that they are so versatile. You can take and a piece of data, content, or an algorithm (code), and wrap with an API, and provide read and write access via the Internet. However, I think the average person does not thrive in this environment and need an explanation that is more precise. It doesn't help them to know that APIs can do anything, they need it to be relevant, but also providing a single solution that they can wrap their heads around, and apply in their world.
I am just sharing this thinking as I'm working to add it to my storytelling toolbox. I'm really committed to helping people understand what APIs are, so they can help push back on platforms to have APIs, as well as be more open with existing APIs. It doesn't do any good to confuse people with an unlimited number of API scenarios. We should be dialing in our storytelling so that we can help onboard them with the concept, and increase the number of folks who understand that they can take control over their own information on the platforms they depend on each day.
We haven't' had a lot of time to move forward the APIs.json index lately, but honestly it doesn't need much pushing forward at this point in time. Our primary objective is to continue getting adoption like this, and not radically shift the specification until we have more feedback from the community, across a large number of API operators.
I tried to get back to normal last week on API Evangelist -- I failed. The previous week was @APIStrat in Boston, which was a success. It was the Presidential election that caused me to swerve and put things into the ditch. I was devastated and saddened by the results. Not because my party lost, but because we chose someone who ran on such a racially, and religiously charged platform, that was so threatening to women.
It is easy to mistake what I do as the API Evangelist, as being a voice for the startup community -- cheerleading APIs in the service of the seemingly endless wave of tech companies coming out of Silicon Valley. While I do pay attention to the technology, and business of how these companies are using APIs, making sure the content, data, and algorithms they employ are transparent and observable by partners, 3rd party developers, end-users, and industry regulators is my primary objective.
My mission is not about open APIs because open always makes things better, or simply open for business. I believe corporate, government and institutional data, content, and algorithmic resources should be as transparent and observable as possible, by those who are impacted by their existence. Meaning if you are collecting data about me, I should know what you are collecting, and have access to it. I should be able to move it around or delete it. Trusted regulators and auditors should also be able to peek behind the curtains of the algorithms that are increasingly impacting our world, and make sure they size up with the claims being made.
I have spent six years pushing on startups and the enterprise to be more transparent and inclusive with their resources by employing APIs. During this time I did the same for the city, state, and the federal government. I've also extended this to higher educational institutions. With Donald Trump in office, this does not change, it just ups the stakes for me. In this climate, being transparent about the data we collect and share, and how the algorithms operate will only be more critical.
I am nervous about the future. Much of the existing rhetoric around algorithms and the surveillance economy has worried me, but all of this in the hands of a sexist, racist, and Islamophobically charged administration and climate scare the hell out of me. In 2017 I will continue to cover the technology and business of APIs, but as I have already been doing, I will be spending a larger portion of my time talking about the politics of APIs, including terms of service, privacy, and the observability of the systems and algorithms that are fueling our increasingly Internet-connected world. As always, I will need your support -- thank you!
There aren't too many startups doing interesting things in the API space right now. One of the exceptions is Stoplight.io. I am working really hard to find some of the good things in the API space to focus on, in hopes of not being too dark with my writing, after the election. Stoplight's new Scenarios is something new, something progressive, and I think could have a larger impact down the road--making it worth covering here on API Evangelist.
Stoplight Scenarios is billed as "Test, automate, and debug web APIs + AWS Lambda". You can make API and AWS Lambda calls, and test, automate, and debug the responses -- pretty standard stuff we are seeing across several API service providers. Where it gets interesting for me is that you can daisy chain these requests together into a variety of scenarios. You can do this with a single API, or you can do it across a variety of disparate APIs--making for some pretty valuable "juju" in my opinion.
As I am learning about a new API, I am often frustrated by having to connect the dots between many different paths, adding, then pulling, then updating, and other common "scenarios" I will want to accomplish. I would love for API providers to do this heavy lifting for me, and provide me with a variety of the common scenarios I need as an API consumer to get up and going. This is just the local possibilities around using scenarios, I'll explore the possibilities between many distributed APIs in future posts.
If you haven't played with Stoplight before, I recommend heading over there. Their API design, definition, mocking, and documentation tooling is leading edge stuff in the world of APIs right now. The Stoplight Scenarios just ups the value their platform brings to the table, continuing to make them on of the few bright spots in the API world for me these days. Let me know what you think after you play with more.
The 7th edition of API Strategy & Practice wrapped up last week. It has been difficult to gather my thoughts with the election going on, but I wanted to shift my attention back to the API community for a bit. Steve Willmott and I started APIStrat back in 2012 to help establish an open, vendor-neutral community to discuss the technology, business, and politics of APIs -- we more than succeeded!
It always warms my heart when people come up to me and share that they didn't see anything different about APIStrat, from other events before they attended and that they do now. Steve and I worked our asses off to make it a rich, inclusive, and meaningful conversation around APIs. It makes me very happy to hear people seeing the event as we intended, and going home with a headful of API knowledge.
One thing I heard from several of the core members who have been there since the first one, and is something that has lingered in my mind, not just because of APIStrat, but also because of where are at in the wider API space -- is that the API world is expanding, and there are concerns regarding how we can maintain community. I consider our hard work in evangelizing APIs to be a success, as space is expanding like never before. Everybody is doing APIs, it isn't just a niche anymore. We are at the point we were with the web in 2001, and folks aren't asking if they should be doing APIs anymore--they are just doing them.
Many of us in the original group are tired. We've been at this for a while. Many of the younger ones are still easily excited and distracted by new technology. There are many new entrants who just need to hear the same old stories we've been telling for years so that they can just get up to speed. Many people expressed their concerns around maintaining community in the face of this evolution, and growth in the space. It leaves a pretty big smile on my face that we are having trouble scaling the API community -- maybe we should try micro-communities or a dev ops approach...no, no a containerized API community! ;-)
I don't know the answer. I'm confident we will find a way forward. I'm super thankful for the community we've built. It literally saved my ass this summer. I just wanted to put these thoughts out there, and let folks know I heard, and Steve and I care.
The 7th edition of API Strategy & Practice Conference happened last week. While I wasn't fully engaged throughout the planning process for this edition, due to my summer being disrupted, I wanted to take the time to share some of what happened to make it more of an inclusive technology event. There is a lot of people who are "interested" in making their events more diverse and inclusive, but APIStrat is "committed" to this (thanks, Charles Ashley III @CAsh_The3rd), and here are some of what we did.
Strong Female Lead - Put a woman in charge. Period. She will set a good tone.
Invite Women To Speak - Work to ONLY invite women when getting started.
Invite People of Color To Speak - Work to ONLY invite people of color when getting started.
Have Code Of Conduct - Make there is a code of conduct present, and communicated well.
Enforce Code of Conduct - Sadly, we had to do this round, but it sets the right tone.
No Manels - Do not have any panels where you only have men.
Numbers - Know your numbers, and work every moment to increase them wherever you can.
Repeat, Rinse - Repeat all of this at all levels, session chairs, keynotes, reg counter, etc.
We are nowhere near where we want to be with making #APIStrat a truly inclusive event, but we are making improvements with each edition. We have our checklist and have been building on it with each event. It takes a leadership team that is committed to this. Steve, Lorinda, Amelia, and the team delivered this round -- I wish I could take credit. I saw more women, diverse faces, and topics at #APIStrat this round -- leaving me very, please.
If you are running a tech conference, please put in the extra work. You can't just give this 5 or 10% effort. You literally have to invite NO WHITE MEN to your event for the first couple of waves of outreach. You need to do some serious homework on the women and people of color who are making an impact across the space. If you do the work, set the right tone, it will go far beyond just being able to pat yourself on the back--the ideas and conversations will be richer for it.
Difficult To Keep My Attention When I was young I was always curious when it came to technology. I set up the entire computer lab for my 7th-grade math teacher back in 1983. I programmed computers all through high school, even having a job programming the software used by schools in the State of Oregon in COBOL. I was really good at school until I wasn't interested. If I got bored, which I did growing up in rural Oregon, I tended not to pay attention in school. Eventually, it got kicked out of high school in my senior year, and I ended up getting into drugs, and a lot of trouble. From 1990 through 1995 I spent my time traveling the country partying and dealing drugs until I finally hit a wall and needed a way out. To get my life out of the ditch I turned to what I already knew, the outdoors (I grew up out in the woods), and computers. I was good at paying attention to the bits and bytes in the emerging world of personal computing, and I would leverage technology to get me out of this mess I found myself in.
Attention To Family & Career After spending a summer in the Oregon wilderness getting clean and healthy, I moved to the nearest city and got to work building a career. By the first dot com bubble, I had found success, married a young lady, and had a beautiful baby girl. I had left my troubled past behind. It was a period in my life where there the harder I worked, the better I felt. I had a good job and bought a house (two), but my attention always seemed to be on finding further business success, a sort of chronic entrepreneurial condition, resulting in having at least one, and often times multiple startup projects going on at any point in time. In total, I had almost 14 separate startups with only two I can even remotely consider a success. Maybe I was too early? Maybe I was just paying attention to the wrong details, and ignoring what mattered to people around me--most critically, to my now ex-wife.
Attention To My New Partner In Crime In 2009, after putting my startup ambitions behind me, and moving on with my life after a divorce, I met an amazing lady. She had a teenage son, and I had a tween daughter. She worked in education technology, and I worked in tech. What we were paying attention to seemed to be compatible. After we connected she increased her focus on the world of educational technology and encouraged me to increase my focus to something that mattered to me. I got to work exploring what that would be--at the time I was finding success using this new thing called the "cloud". I knew it was going to be significant in the future, but I also knew there was more to it, beyond just things being in the "cloud".
Attention To The Business of APIs I saw the commercial potential of making digital resources like compute and storage available in the cloud using APIs and increasingly saw entrepreneurs deliver the resources needed for mobile application development using APIs. What did both cloud and mobile have in common? APIs! This is where I would pay attention. As I dug deeper, I noticed there were a number folks paying attention to the technical merits of web APIs, aka REST, but that very few people were paying attention to the business side of API operations. I was watching companies like Twilio, SendGrid, Stripe, and others demonstrate that there was more to this API thing than just having a RESTful API on the open web, and that through sensible identity and access management (API keys, OAuth), rate limiting, metrics and reporting, aka API service composition, that you could develop some pretty interesting business models. I felt I had hit on something significant, and would focus all my attention on the business of APIs.
Attention To Bankruptcy & Addiction Things were going reasonably well. I was paying attention to the fast growing world of APIs. The only problem was that I hadn't been paying attention to the finances as much as I probably should have, and my previous startups and my divorce were finally catching up with me. I had nowhere else to go, I had to declare bankruptcy. Around the same time, my partner's son began to have problems in his world, involving depression, and pharmaceutical addiction. He needed some assistance. We tried a change of scenery, moving him to another city in another state, hoping to shift his attention to a new environment, and job. At this point in our career, we were living 100% on the road, traveling to events, evangelizing APIs and education technology, but we did what we could to help him find some balance.
Attention To The Politics of APIs My hard work paying attention to the world of APIs was paying off. I was acquiring new partners, sponsors, and tackling more projects that were focused on the business of APIs, telling the stories of how startups were making an impact across a variety business sector. As I paid more attention to the business of APIs I began to see there was more to APIs, and that things like terms of service, copyright, service level agreements, security, and even regulation were beginning to define the space. My work ended up getting me invited to work at the White House, helping out the Obama team with open data and API strategy across the federal government, and helping the Electronic Frontier Foundation with the Oracle v Google API copyright case. I was still paying very much paying attention to the business of APIs, but increasingly I was also paying more attention to what I had come to call the politics of APIs.
Attending To My Health & Well Being It can easy to get caught up in work, especially when you are an API Evangelist. It can also become a distraction from your actual life, and the material world around you. API Evangelist was a character for me, a persona, but I was living on the road, spending every waking moment as this character--resulting in me paying attention to very little else. During this time I was present, front and center in the API space, but I was neglecting to pay attention to my own health, something that finally caught up with me. Because of the Affordable Care Act (thanks, Obama!) I went to the doctor early this year, and he said that I would not live beyond 50 if I didn't stop drinking alcohol. I quit drinking immediately. Then shortly after this happened, my partner's son showed up on our doorstep. He had been kicked out of the treatment center we had placed him in 8 months earlier, after a continued struggle with his depression and addiction. I was sober. He was using opiates again. We had no money. What could we do?
Attention In A Material World I did the only thing I knew how to do. I did what I had done before and headed up to the woods to get clean. We rented a truck and drove to Yosemite, and deep into the Sierra Nevada mountains of California. We found ourselves on Peterson Mountain on the border of California and Nevada digging for quartz crystals (another place from my past), after which we would make our way north through North California, and on into Oregon. We avoided cities, and spend our days hiking and clearing trails. We put signs back up. We often were paying attention to trails that seemed forgotten by humans and were just melting back into nature. We rock hopped in rivers and creeks. Swam in the snow melt. We walked 10-15 miles most days, paying attention to the kid's health, and as I would learn--my own health and sanity along the way. We talked about getting high, the online world, having oxycontin delivered to your door enabled by the Fedex API, being paid for with help from the Bitcoin API. Convenience and online culture. We walked. We walked. We walked. We walked until we slept each day.
Attention To The Digital World While I was marching the kid around the mountains, it was an emotionally and physically exhausting time for both us, and through all of this I didn't want to neglect (our) more digital selves. The kid is your average 20 something who spends much of his time online, and gaming. While out in the wilderness I wanted to still attend to the digital parts of his (our) personality and I settled in on drones as a viable distraction. Drones worked. We could fly them on the trail, capture video, and as I would come to learn, we could also pay attention to much more around us using this new and wonderful Internet and radio connected device. We chose the DJI Phantom 3 Pro drone which has an iPad as the visual experience for the radio controller. Something that when you put in the kid's hands he'd head into a bush, under a rock overhang, and focus 100% of his attention to flying and experiencing the world through the drone's camera. Contrasting his approach, when you put the controller in my hand, I would always keep one eye towards the sky, paying attention to this very expensive, physical object flying through the canyons, over the lakes, and down the river.
Attention At The Intersection Of Physical & Virtual Drones are an amazing piece of tech. They give you an entirely new view of the world. It is a 4K video of the world wrapped in longitude, latitude, altitude, airspeed, and a wealth of other data. The physical drone has its own APIs. The guidance systems have APIs. The camera has its own APIs. The battery has its own APIs. The radio controller has its own APIs. The mobile interface for the controller has its own APIs. It uses APIs to acquire GPS and mapping information. When a wireless network is present it can stream live to Youtube and Facebook using APIs. It receives updates in the controller app regarding nearby airports, military bases, weather, and forest fires using APIs. Data is captured every second when operating and can be uploaded to the cloud synchronously or asynchronously using APIs. It is an amazing Internet of Things (IoT) device to consider when it comes to API usage, as well as just for flying through mountain meadows, and through river canyons.
The Digitial Demanding Our Attention This summer we were able to successfully escape out into the woods, at the tops of mountains, and down into the deep river valleys. However, even though we were often hundreds of miles into the wilderness, the digital world seemed to seek us out. The battery would need a firmware upgrade. The controller was out of sync guidance firmware. Where is the damn cell phone charging cable? The batteries (with firmware)--were they warm enough to operate? When we'd get to the end of trails, and I would grab a cell phone to call for a ride, or communicate that we were safe, the cell service was so bad, we'd wait for hours sometimes for push notifications, alerts, syncs, and other digital noise to complete before we could gather enough bandwidth to send a simple SMS. We eventually learned to remove all but the essential applications, so the demands for our attention were reduced to only what we needed to survive and get from point A to B, and to acquire supplies.
Attending To My Online World As we approached 3 months in the woods, it was time to come back. The kid settled into an apartment in Portland OR, and I went back to Los Angeles, CA. I turned on all the regular channels, my feeds, email, Github, Twitter, Facebook, LinkedIn, Medium, Slack, Disqus, Reddit, DZone, Lanyrd, Meetup, Stack Overflow, SoundCloud, Tumblr, and others. Everything was demanding my attention. Read Me! We want to work with you! Our tech is the next thing! Come speak at our event! This is the next big thing! By 2020 everyone will be doing this! I realized how much I had been paying attention to, how much momentum I had built up. I realized how much of an API driven assault I was under each day. Now I needed to understand what was actually worthy of my attention, bend the concept of real time to my personal definition, and establish a sort of "air gap" between me and the digital world that had an unending appetite for my attention.
Attention On The Drone Even back in the real world, the drone continued to linger in my mind, working to get my attention. I was learning more about the device and it's APIs, and how it consumed APIs in the cloud, and how it used APIs to publish video, images, and other data to the clouds. I learned about how companies like Airmap were providing updates to my DJI drones from federal government agencies like the Department of Interior. I also learned how these agencies had themselves discovered that their fleets of DJI drones were updating video, images, and all this data to Chinese servers. DOH! Drones seemed to be capturing everybody's attention, as they interfered with forest fires, flew in front of commercial airlines, and gave us a new perspective of our both the material and digital worlds. Providing us with an entirely new way to gather data, develop awareness, and pay attention to infrastructure, nature, or even a police protest. Drones are capturing our attention, but it also feels like they were now also paying attention to us, whether we like it or not.
Attending To What Matters I consider myself a success. I do what I enjoy for a living, and make more than enough money to survive. I have an amazing partner in crime. I have a strong and smart teenage daughter. None of this success is due to any single event along the way. My success is because I always realized that all if this is a journey and never a single destination. It is not the money. It is not about a lack of money. Not any single piece technology. Not about any of the startups I have helped build or worked with. Not even API Evangelist. APIs simple have given me lens to look through as I pay attention to the "seemingly fast-paced" world of web, mobile, and now these amazing devices we are connecting to the web. No single piece of technology is what truly matters, not even the API, it all depends on what I do with them, and how we empower (or hurt) humans with them. This is a human story. This is my story. It is not a technology story. It is my personal journey. The technology is just one of the tools in my toolbox. The usefulness of my tools depends on which ones I choose to use (or not), how I care for them, and the knowledge behind how I put these tools to work. Taking a moment with each application, to always consider what really matters.
Definition Of Attention To close I wanted to borrow from my partner in crime Audrey Watters (@audreywatters) on talk a couple of months ago on attention, where she said: According to the OED, “attention” – derived from the Latin “attendere,” to attend – means “The action, fact, or state of attending or giving heed; earnest direction of the mind, consideration, or regard; especially in the phrase to pay or give attention. The mental power or faculty of attending; especially with attract, call, draw, arrest, fix, etc.” “Attention” is a noun, but it refers to an action and/or a state of being. Attention is a mental activity. An earnest activity – which I particularly like. Attention is a military activity. It refers to how we stand and how we see and how we think.
For more information about our journey, you can visit Drone Recovery, and thank you to the API community for your support.
I was reading the latest Yahoo transparency report, as well as the Tumblr. When a company releases their latest version of this data, it tends to prompt me to take a look at some of the other providers who have them, like Google and Twitter. I am interested in what I can learn from these reports, about what the government is up to, as well as the incentives behind each platform publishing their reports. I fascinated by studying the process and approach of each company, tracking on what some of the common building blocks so that I can include in my API transparency research.
The addition of CSV downloads of information requests by Twitter is worth noting, and adding to my list of building blocks. Platform providers are going to have to consider adding machine-readable versions of their transparency reports to help address the growing number of requests they will bet getting in this area. Providing CSV, JSON, or even YAML access to transparency report information is going to become important for us to understand how this legal and very political layer of our worlds is evolving.
The number of government and court requests for information are up. The number of companies publishing transparency reports in response to public demand for transparency is increasing. Providing machine readable representations of this data, as well as evolving a common API and data schema definition is the only way we are going to be able to manage this growth, and make sense of the data across platform providers. I will keep adding company transparency reports to my research, and take the time to aggregate the common elements in play across them, and see if I can't contribute to this common API and schema definition for use by providers
I went down to the police department in Hermosa Beach and filed my application for a drone permit. It's been two weeks and I haven't heard back. When I get done with @APIStrat I will go down there and talk to them again, and probably have to resubmit my application. Hermosa Beach is purported to have some of the strictest laws. I'm submitting mine so I can just play with my drone, and program it on the beach without getting into trouble.
Before I fly my drone I use the B4UFLY mobile app from the Federal Aviation Administration to know whether I should fly my drone or not. It tells me whether I'm near an airport, military base, other location or event. This helps, but it doesn't help me with the legal side of things, what the laws are in my area, and any assistance in acquiring permits and approval. The industry is young, I'm sure eager startups out there are already working their asses off to aggregate the data, and serving it up via APIs for use in mobile and drone applications.
As an operator, I am going to need the laws applying to the drone, bu also laws applying to video, and other data gathered. Can I be filming? What are private / public property data and privacy laws? I am gathering video, weather, temperature, and other crop data over commercial farms, am I allowed to keep it? What are the rules with other types of imaging like infrared? Are there any noise ordinances? There are numerous considerations when flying a drone for both personal and commercial scenarios.
To be an informed pilot I need all this at run(flight)time. I guess I might also need some of it before I head out to my flight location, but also whatever I can also get at flight time. The drone sector seems to begin the process of tackling many of the urgent questions facing the industry. The FAA has stepped up with their guidance, and companies like Airmap are also getting involved with the flight restriction level, as well as weather. We will need the same layer for the legal side of things, helping us receive up to date details on the laws impacting me flying drones, filming, and gathering data--maybe systems to help us manage.
Sounds like another drone API opportunity. I'd say the three top things impacting me when operating my drone(s) right now are 1) flight restrictions 2) weather, and 3) legal. I'm sure if you pulled together the data, provided a simple commercial API, you could do well delivering it to drone application developers and other platforms.
I am still learning about it, but here is a short description from the post by WaaG, that I think describes it well:
The Dowse box gives you back control. Dowse is free and open source software. There is no big company behind it. By installing the Dowse box you will get insight down to the network layer of what is actually going on in your home. Who is talking to who, what, where and when? You can see which device connects to which company and you can turn that communication off. Or allow it.
This reflects the transparency I would like to see at the network level for every home and business. With the recent wave of IoT launched DDOS attacks we are going to need more discoverability, awareness, transparency, and control at the home and business and home network and Universal Plug and Play level.
I'm not sure saying the Dowse box is the solution, but it provides us with one possible blueprint, similar to what I was talking about in my post on transparent data transfer control. In this IoT age, we are going to need device awareness to be a default, and transparency about which devices are talking to the Internet, when and what data they are transferring, and possibly even more security auditing and observability solutions.
It seems like the Dowse box concept should be baked into all home and business networking. I am well aware that a version of this does exist at the network administration level for most modern networking solutions, but I'm talking about the next generation, end-user friendly, mobile edition that makes devices monitoring, awareness, and control dead simple for anyone.
Of course, this layer would have an API to allow for the extending, and customizing of all of this. Now that I am coming across examples of this type of control at the IoT network level, I will keep scratching, and I'm confident I'll find other examples. Maybe enough thtat Ithatadd a section to my Internet of Things and APIs research.
More powerful APIs - Your nanoscale.io hosted microservices can run longer and perform more complex operations, or access slower source systems, without timing out.
Get premium support - Having a problem with nanoscale.io? Think you may have found a bug? Submit your inquiry and get a guaranteed response from our technical team.
Influence product roadmap - We are open to new feature consideration, and give preference to paid accounts to influence which ones get built sooner.
Deploy anywhere - Upgrade your downloadable nanoscale.io server with a production license to deploy your microservices on any infrastructure you choose.
More power and support seems like no-brainers. Being able to influence the roadmap is a compelling reason and something I would pay money for! ;-) The deploy anywhere I think is a sign of the future, not just for how you will buy services for your API, this is how you will deploy your APIs for your consumers. In cloud. On-premise. On-device. You can consume our API anywhere you want--if you pay for it!
There doesn't look to be a lot of activity around the project in the last year, but it provides a good model for what I'd like to eventually see across all the major stops along the API lifecycle. I picture a wealth of aggregate tooling like Denominator that can act as a broker between API service providers and help switch, migrate, and sync between providers whether you are deploying, managing, testing, monitoring, or dialing in your DNS.
As I read the multiple investigations into what happened with the DDOS attack on Dyn last week, it seems relevant to learn more about aggregate DNS API solutions like Denominator. I will spend some time looking for other similar open tooling that is vendor-neutral, as well as vendor-switchable. We are going to need open source circuit breakers like this to help route, switch, migrate, and sync DNS across many service providers in this volatile landscape.
The functionality felt a lot like the social aspects that made Github more attractive than just Git. When it comes to developing messaging apps and bots I can see a social layer doing pretty well. "Once a Collaborator is added, they’ll receive a notification from Slackbot letting them know they now have access to your app." Smells like an opportunity for API management providers to bake into user management solution, and if you wanted to take it to the next level, add a Slack messaging layer as option.
Its a small feature, but I think it is one of the things that made Github work, and could have a similar impact at the API management level, allowing for more engagement between users who are working together on API integration. I am going to add it as a building block for my API management research, even if I haven't seen it elsewhere. It is something I'd like to see more of, and maybe if I plant the bug, more providers will implement.
DJI provides four methods for managing the drone data transfer control:
Start Transfer - Inform guidance to start data transfer.
Stop Transfer - Inform guidance to stop data transfer.
Release Transfer - Release the data transfer thread.
Wait For Board Ready - Set callback function handler for hen data from guidance comes, it will be called by data transfer thread.
The "wait for board ready" method acts as a sort of web hook, that can notify any application build on the API that data is now being transferred, opening up the possibilities for notifying a device owner, and operator that data is being transferred. To me, this can be a critical aspect of building trust that our devices have our best interests in mind, providing some essential transparency in the data layer of the IoT space.
Data transfer control APIs for IoT devices like this will not ensure healthy data practices. This implementation is designed to provide transfer control capabilities to the developer, it is now up to the developer to include the end users in this process. Many current mobile application business models do not incentivize this type of transparency, as you do not want end users, and often times 3rd party developers involved in data gathering, and revenue generation this valuable "exhaust" of Internet-connected devices.
I am hoping this evolves and changes as the Internet matures, and the number of connected devices increases. We need transparency at the device data transfer level, and we need all humans involved / impacted to be a literate, active, and will participate in the IoT data flow. I know many entrepreneurs think the end user isn't sophisticated enough to be involved, but this is more about your desire to keep any value generated for yourself than it is about the end-users capacity to grasp this stuff.
Many folks see me simply as a cheerleader for APIs when in reality I am more of an evangelist for the bad that can happen with APIs. I believe that sharing of data, content, and algorithms using web APIs has the potential for good, but in reality, they are often be used for doing some pretty shady shit.
An example of this is found in my inbox this morning, and I'm sure is something everyone will encounter at some point in their daily lives. It is an email for an undelivered Fedex package, which I know better than to click on, but sadly I think it is one that many folks will fall for.
Why do they fall for this? Because the email potentially has relevance, as I just ordered a handful of packages from Amazon, which were being shipped via Fedex (I do not order much online). Using the FedEx API, anyone can query the status of a package. I'm assuming that there are folks out there who are scanning for the presence of delivering notifications--I'm not up to speed on the details of how you can do this. I'm unsure if they can get my email alongside this information, but I don't think this matters. I think they can correlate data about where I live, and the fact I'm receiving packages--whether the email came from API, or through other forms intelligence, it doesn't matter.
My point is more around the fact that APIs are increasingly opening up signals about our daily lives, providing a wealth of context for phishing campaigns, increasing the chance that people will fall for these attacks. My solution to this problem does not involve a knee-jerk response to providing APIs, I am just looking to just warn API providers that they should be monitoring for this type of behavior on top of an API, and we should help the average email users and Amazon package receiver that these dangers exist. Everyone should pause and think deeply about each link or file attachment we click on--no matter how relevant it might seem in our daily flow.
I am going through the DJi DJI drone developer area which has three distinct SDKs, which allow us to leverage a variety of APIs that make the drone magic happen. I'm still wrapping my head around the intersection of drones and APIs, and this is my attempt to distil down what I'm finding in their developer area, and absorb some of what is going across the industry. This is not meant to be a complete list. It is meant for my learning, and hopefully yours along the way.
There are a variety of devices being connected to the Internet, but other than the automobile I don't think there is another object that is as complex as the drone. I'm fascinated by what is possible with this device, and the variety of APIs it has, the interaction with the RC controller, mobile device, and with other resources the clouds. I personally fly a DJI drone, so I am going through the DJI developer area, learning about their three SDKs, as they seem to be the ecosystem furthest along in their understanding the API potential -- think Twitter for IoT.
The DJI Onboard SDK This SDK allows for communication with the DJI flight controller over a direct serial connection, to monitor and control aircraft flight behavior with the Onboard API while utilizing the built-in Intelligent Navigation Modes to create autonomous flight paths and maneuvers. Some of the actions for the onboard SDK are:
Activation - Before you start exploring DJI Onboard SDK functionality via our ROS examples, you will need to go through the "Activation" process.
Obtain/Release Flight Control - Managing the process to get flight control.
Take Off - Initiate a take-off for the drone.
Landing - Tell the device to land.
Go Home - Tell the device to go home.
Gimbal Control - Manage gimbal for camera.
Altitude Control - Manage the altitude for the drone.
Photo Taking - Allow for taking photos
Start/Stop Video Recording - Start and stop the video for camera.
Virtual RC Control - Control the drone through serial port by simulated channel values.
Broadcast Frequency Control - Manage which frequency drone is broadcasting on.
Arm/Disarm Control - Arm or disram the controls.
Timestamp Synchronization - Synchronize the timestamp -
Native Waypoint - Manage the waypoints for mission.
Hotpoint - Manage the hotpoint for circling.
Local Navigation - Go to specified local position
Global Navigation - Go to specified global position
Waypoint Navigation - Manage flying through a series of GPS coordinates)
WebSocket With Baidu Map (for navigation) - Real time mapping.
MAVLink And QGroundStation - Managing the vehicle to air communication.
These are the actions that onboard SDK open up, but the SDK has other layers, that go beyond the drone itself, and is more about the space and environment around a drone, and its interaction with this world.
Velodyne LiDAR Light Detection and Ranging (LiDAR) sensors are for commercial UAS applications, such as 3D aerial mapping, surveying, inspection, collision avoidance, and autonomous navigation in potentially either indoor and outdoor environments. There are three distinct elements of the drones LiDAR system:
Lidar API - Supporting the computation of point cloud and logging LiDAR data in real-time into a standard LAS (LiDAR Aerial Survey) or PCAP (packet capture) file
Lidar Simulator - A LiDAR simulator for playing back PCAP files in real-time via the same Ethernet output as a real LiDAR for facilitating development and debugging for the system integration
Lidar Logging - An API based example to demonstrate how to use the API of real-time point cloud computation and LAS/PCAP logging. It can also be used for the use case of the integration of Velodyne LiDAR with M100 without the onboard SDK
LiDAR opens up another universe within the DJI onboard SDK, allowing for access the system through API, manage the logging around activity, also simulating common navigation elements in the environment.
uAvionix ADS-B Receiver The pingRX ADS-B receiver allows the drone to receive real-time traffic information broadcasted by other manned or unmanned aircraft, as well as temporary flight restriction (TFR) information broadcasted by the government. With this type of situation awareness, the onboard embedded system (OES) will be able to make some safety-critical decisions like collision avoidance or self-separation.
Precision Trajectory Mission Planning With the Onboard SDK Precision Trajectory Mission Planning suite, DJI developers can now plan complex missions without having to use GPS waypoints. The new DJI Precision Trajectory Mission Planning library has the flexibility to deal with complicated trajectories, issues with GPS accuracy and cases when GPS is simply unavailable.
Trajectory following library that can autonomously execute preplanned smooth spiral trajectories
SketchUp plugin to visualize trajectories, import 3D CAD models and geolocate the scene
Configurable speed, start/end radii and pitch for the spiral
Start your drone from anywhere - real-time path planning to get to the trajectory's GPS location
Integration with DJI Assistant 2 to visualize simulations of the drone following the trajectory in the SketchUp scene
That is a pretty robust SDK. I'm taking the time to learn about each action, as well as the more communication and mission planning components separately. I can tell that they are having trouble to keep the large amounts of functionality and features coherent, and organized in the documentation, one of the reasons I'm breaking out separately here on my blog.
DJI Guidance SDK Guidance SDK enables allows you to develop vision-based applications, by granting you full control over drone guidance. You can access all output data for any device using the DJI Guidance SDK--they break things down into five separate groups.
Reset Config - Clear subscribed configure, if you want to subscribe data different from last time.
Initialize Transfer - Initialize Guidance and create data transfer thread.
IMU - Subscribe Inertia Measurement Unit (IMU) data.
Ultrasonic - Subscribe ultrasonic data.
Velocity - Subscribe velocity data, i.e. velocity of Guidance in body coordinate system.
Motion -Subscribe global motion data, i.e. velocity and position of Guidance in global coordinate system.
Set Callback and Exposure
Event Handler - Set callback function handler. When data from Guidance comes, it will be called by data transfer thread.
Exposure Parameters - Get stereo calibration parameters.
Get Online Status -Get the online status of Guidance sensors.
Get Sterio Calibration - Get stereo calibration parameters.
Get Device Type - Get the type of devices. Currently only support one type of device: Guidance
Get Image Size -Get the size of image data.
Start Transfer - Inform guidance to start data transfer.
Stop Transfer - Inform guidance to stop data transfer.
Release Transfer -Release the data transfer thread.
Wait For Board Ready -Set callback function handler. When data from guidance comes, it will be called by data transfer thread.
This SDK seems to be real time senses of the drone, allowing you to develop the experience you need to be in control, and guiding the device.
Data Types One of the thing that captivates me about the whole drone thing is its data collection capacity. I'm still learning about what is possible with the drone itself, but I know that much of the value generated by these flights will be based on the data that is gathered, as well as the images and video recorded. Here are the data points I have found so far in the DJI documentation for the DJI Guidance SDK.
Error Code - Enumerates possible error codes. When error occurs, usually an error code will be given, and the developer can reference this enum to find the error type.
Velocity Data - Velocity in body frame. The unit is millimeter/second and the frequency is 10 Hz.
Obstacle Distance Data - Obstacle distance from five Guidance Sensors. The unit is centimeter and the frequency is 20 Hz.
IMU Data - IMU data including accelerometer (in unit of acceleration of gravity g) and gyroscope (in quaternion format) data. The frequency is 20 Hz.
Motion Data - Pose and velocity data including quaternion orientation, position in the global frame, velocity in the global frame.
Ultrasonic Data - Outputs ultrasonic data from five Guidance Sensors, including obstacle distance (in unit of meter) and reliability of the data. The frequency is 20 Hz.
Greyscale Image - Outputs Greyscale images for five directions. The image size is 320*240 bytes for individual sensor. The default frequency is 20 Hz and can be scaled down using API functions.
Depth Image - Outputs depth images for five directions. The image size is 320*240*2 bytes for each direction. The default frequency is 20 Hz and can be scaled down using API functions.
Disparity Image - Outputs disparity images for five directions. This data is useful when developers want to further refine the disaprity images using functions like speckle filter.
This data is generated constantly by a drone, and you have control over this transfer process through the DJI Guidance SDK. I'm thinking I need to aggregate some JSON schemas for this data, to better help me understand the depth and relationships in this data. There is a lot going on here, and a wealth of data to consider in a wide range of scenarios.
DJI Drone Mobile SDK I use the DJI drone application to operate my two drones. The DJI Drone Mobile SDK is where you can get to work crafting your own custom application, to deliver exactly the drone operation experience you want. This is what the iPhone application was for mobile, but this is for the consumer and commercial drone world. There are a wealth of areas you can develop around in this SDK:
Flight Controller - The flight controller is an onboard computer that combines control information from the pilot with sensor information to adjust the thrust at each propellor and fly the aircraft as desired.
Camera - The camera captures photos and videos. Many different modes of operation, resolutions, frame rates, exposure settings, picture settings and file types can be selected. Cameras have local storage to hold the media which will typically be an SD card, and in some cases an SSD (solid state drive).
Gimbal - Cameras fixed to an aircraft will record images that pitch and roll with the aircraft as it moves. Multi rotor aircraft need to pitch and roll simply to move horizontally, and so getting a stable horizontal shot is not possible.
Airlink - AirLink describes the wireless link between aircraft, remote controllers, handheld cameras and mobile devices.
Remote Controller - The remote controller allows manual flight, gimbal and camera control, and provides a robust wireless control link for aircraft. The mobile device can connect to the remote controller to communicate to the aircraft and receive the live video stream from the camera.
Smart Battery - Smart batteries provide the energy required to run the products. Together with the flight controller, the smart battery can estimate remaining flight time and provide warnings when low battery thresholds are crossed. Batteries are easily swapped between flights, extending product use considerably.
Missions - Missions can be used to easily automate flight. There are many different mission types that offer different product behavior. Some missions can be uploaded to and managed by the aircraft, while other missions are managed from the mobile device.
SDK Manager - Application registration to use the DJI Mobile SDK, product connection, debugging and logging services are handled through the SDK manager class DJISDKManager.
I think that this stack of features speaks for itself. Providing a wealth of valuable API-driven resources to think about. This blows my mind as I begin to think about the possibilities for developing drone applications but gets even better when you think about how this also applies to the rest of the IoT world. Flight controller might not apply universally, but cameras, batteries, remote control, and the network are all ubiquitous with other IoT devices, and in my world should be considered beyond just drones.
I just needed to wrap my head around what programmatic resources are available to me as a DJI drone operator and developer. Next, I will be diving in and learning about some of the more interesting layers of this drone ecosystem, but first I am more interested in spending time looking through the platform API and SDK resources for other drone platforms, as well as some of the data solution providers like Airmap, and other physical components providers like FLIR for imaging, and LumeCube for lighting. I am always having to pick my battles on how deep on want to go down each rabbit hole or stay at the high level for a wider perspective.
There is a lot going on here. I find drones fascinating from a technical perspective, and terrifying when it comes to surveillance, privacy, policing, and some of the other bad behavior we've seen recently. Like other areas of the tech space, I think APIs are important for not just managing devices and the experience, but also provide transparency, logging, auditing, and other observability considerations when it comes to the Internet of Things space.
I get why people are interested in voice-enabled solutions like Alexa and Siri. I'm personally not a fan of speaking to get what I want, but I get the attraction for others. Similarly, I get why people are interested in bot enabled solutions like Facebook and Slack are bringing to the table, but I'm personally not a fan of the human-led noise in both of these channels, let alone automating this mayhem with bots.
In short, I'm not 100% on board that voice and bots will be as revolutionary as promised. I do think they will have a significant impact and are worthy of paying attention to, but when it comes to API driven conversational interfaces, I'm putting my money on push driven approaches to making API magic happen. Approaches like Push by Zapier, and Webtask.io, where you can initiate a single, or chain of API driven events from the click on a button in the browser, in a web page, on my mobile phone, or hell, using the Amazon Dash button approach.
These web tasks operate in an asynchronous way, making them more conversational-esque. Allowing those of us who are anti-social, and have adequate air gapped our social and messaging channels, and haven't fully subscribed to the surveillance economy, alternate solutions. These mediums could even facilitate a back and forth, passing machine readable values, until the desired result has been achieved. Some conversations could be predefined or saved, allowing me to trigger using a button at any point (ie. reorder that product from Amazon, retweet that article from last week). I'm not saying I don't want to have an API-enabled conversation, I'm just not sure I want a speaker or bot always present to get what I need to get done in my day.
I understand that I am not the norm. There are plenty of folks who have no problem with devices listening around their home or business, and are super excited when it comes to engaging with half-assed artificial intelligence wich accomplish basic tasks in our world(s). But I can't be that far out in left field. I'm thinking the landscape for conversational interfaces will be diverse, with some voice, some chat, and hopefully also some asynchronous approaches to having conversations that can be embedded anywhere across our virtual and physical worlds.
One of the things I love about my world as the API Evangelist is the time I get diving into rabbit holes and learning about different areas where technology is being applied. I do not always agree with the business motivations behind what is going on, which can result in some often pretty shady situations, but I enjoy stepping back and understanding the data, API and approaches behind what is going on.
An alpha generation platform is a technology solution used in algorithmic trading to develop quantitative financial models, or trading strategies, that generate consistent alpha, or absolute returns. The process of alpha generation refers to generating excess returns.
The article that triggered this is about drones generating alpha was focused on the data generation, and communication capabilities of drones, and how they can be used in trading algorithms. Companies are looking to fly over crops, and predict yields, aggregate data, and add to the resources that available in the "alpha generation platform".
I'm not convinced of the reality of this approach, but it does provide for some interesting scenarios as I am learning about the data drones can gather, how this transfer occurs to the cloud, and be put to work using existing approaches to video, image, analysis, and visualization APIs. Also, it provides fuel for my alternate design fiction writing, where I explore the possibilities of technology, both good and bad.
From what I understand, this type of data would be considered "dirty" in the alpha workflow, in the same category of other data, you might gather from Amazon sales, Twitter sentiment, and other common API implementations. It's experimental, and not proven (yet) to produce the consistent or absolute returns I guess. What the article is discussing, are just early discussions, speculations, and desires of people looking to create a market--interesting fairy tales to tune into. Well, this is how markets operate right? On stories?
I'm adding high-frequency trading and alpha generation to my list of sectors potentially being impacted by drones. Another area I'm diving into that intersects with this work are satellite APIs. Satellite and drones intersect as drones are being used to fill in where satellite imagery can fall short (weather, low level buildings, etc.), but satellites are also being used as an alpha generation tool for high-frequency trading. I'm guessing drones and satellites are going to overlap with many other sectors like agriculture, mining, and the environment.
Lots of API and drone goodness to keep my mind busy.
Internet-connected devices generate data. The most recent wave of mobile devices has opened up an unprecedented world of data generation and harvesting from the network, device, and application layers. The location data, photos, videos, and other valuable exhaust from these devices is why there has been so much investment in technology, and why we are seeing continued investment in the Internet of Things (IoT).
When it came to mobile phones this opportunity was new, and it isn't always clear that we are the product when it comes to making money off connecting these devices to the Internet. People aren't always aware of how much data they are generating, and how much this data is generating revenue for the latest generation of entrepreneurs--because it's new. Things have moved along, and it's not a secret anymore that devices connected to the Internet generating data has the potential to be very valuable in the right situation.
I have been historically frustrated with people's lack of awareness of this, but I'm hoping that with each wave of technology that comes in the future, we will get smarter about this, and stop being the product when we can, and begin demanding a piece of this action (if we do it at all). If our wearable fitness device is used in any healthcare study, if that weather, water, or pollution sensor in our yard is generating revenue, we should get a piece of the action. If a device in our homes or businesses is generating data, we should be a more aware of, and willing participant in this new supply chain.
The average person may not always care about their privacy in the surveillance economy, but maybe they'll care about lost opportunities for making money. At the consumer level this isn't always a coherent argument, but as you approach the work at home world, and into the professional territory, it can begin making more sense. Not all weather and pollution monitors might make sense for this, but when you start thinking about participating in clinical trials and operating your own drone fleet in farm country, revenue share models for Internet-connected devices might seem a little more realistic.
Silicon Valley prefers an unaware, data-illiterate consumer, and even business class of users. I prefer a more aware, and data literate society, which is one reason I'm so big on APIs. Opening up 3rd party access, metering, and potentially revenue share opportunities for business, professionals, and individuals is one of the reasons I think APIs are important for all of this to work. I will keep studying how APIs are being used to generate, transmit, and generate revenue for platform providers, and device operators, so that I can hopefully keep pulling back the curtain on how the sausage is being made so that more of us can play in the game, and get a piece of the action.