I have had Steve Willmotts of 3Scale's response to the announcement that Twitter was cutting off firehose access for over a week now, and bringing the higher level access completely in house via their Gnip acquisition. You can read the response from Gnip via their blog post Working Directly With the Twitter Data Ecosystem, and from DataSift in their The Twitter/Gnip Gap: Data Licensing vs Data Processing. Steve and I are in agreement with the points made via his How Not to be a Platform: Twitter’s Firehose Mistake post. Where I really struggled with over the last week, in producing a official response, was after responding How To Be A Platform: Making Tough Partnership Choices, from my friend Tyler Singletary.
You see, I agree with everything Tyler pointed as well. He's right. Twitter has probably made the right decision for Twitter, and based upon their goals and understanding of the Twitter API ecosystem, did exactly what should be prescribed. I have lots of opinions around the Twitter ecosystem, and how Twitter Continues to Restrict Access to Our Tweets. However, when I step back, and put on my API operations hat, everything Tyler points out, is valid in my opinion. Twitter gets to decide who has access, and in respect to their goals, chooseing the Gnip offering over the DataSift offering makes sense. It's all about the winning solution, and where the ROI is when it comes platform operations? Life is good when you are on the winning team. However #winning != #good, and #winning != best outcome. I also guess, that #open != good, and #open != best outcome either. There are no absolutes in API land--sorry.
Where Tyler stance really begin to lose its shine for me, is when you zoom out. You see Tyler, you are on the winning team. You have access to the firehose, and you operate squarely within in the #winning conversation (I know you don't have direct firehose, and rely on Gnip). Those of who aren't, the conversation looks very different. Let's take a look at the Gnip vs. Datasift product comparison you provide, but from my stance.
After two phone calls with Gnip sales team, they realized I didn't have the requisite $45K to move forward, not worthy of continuing to talk to this one. All I'm left with is Twitter REST and Streaming API, and DataSift. Gnip isn't even on the menu, isn't even an option in my world. So only the players who can afford to be at the table, get to play--kind of sounds like a familiar story to me. Where have I heard this before? Oh in every industry, throughout history. This isn't the API promise I've been hearing! ;-)
I remember in 2007 and 2008 when Twitter was built on these same principles. Only people with the big money could step up and play on the platform. Oh wait no! It was the other way, the people with money didn't give shit about Twitter, and only us small fish were playing on the platform. We were building widgets, Wordpress plugins, developing meaningful analytics, mobile apps, and Twitter encouraged it all. Prove what you are building on the public API, and when you are ready, we'll get rid of any rate limits, or even give you the firehose-just ask us, we'll give you the access you need--the firehose rhetoric was much different 5 years ago:
The rush of innovation did happen, but does anyone have access to the same volume of data regarless of company size in 2015? Does the argument that Tyler lays out today, apply in a 2007 or 2010 Twitter ecosystem? How does this line of thinging impact the new startups who are trying to get API developers to come out to use their ecosystems, now, when the big money players don't care about them? Sure things have changed, things have evolved, but I just don't buy the argument that the better product (Gnip) won out over the other product (DataSift). In the neatly package and positioned arguement sure, it makes sense, but if you zoom out, it just doesn't hold weight, now or then. Things have changed for Twitter, but things haven't changed for new startups who are building platforms--they are still depending on early adopters to build on their platform and turn them into a success story, and attact VCS and users.
This is what leaves many of us so disgruntled, its not Twitter making the hard decisions around platform operations. It is their abrasive stance towards those of us who are not in the club, and their lack of communication about the ecosystem, and their sharecropping, and poaching of the best of ideas along along the way, and changing the tune slowly over time. Honestly, I feel like they've done a good job in correcting much earlier bad behavior recently, with their overall partner program, API operations, and evangelism, but their whole firehose position is lacking the bigger ecosystem vision--it is just self centered. Twitter exploited the community when they could, they hand picked a group of partners (Gnip, DataSift, DataMinr, Mass Relevance, and many more) to serve this need when they didn't have time or resources to do it on their own, and now that they see that there more dumbass enteprise peopleare willing to pay Gnip prices for our data, over people building on the long tail, I respect the whole thing even less--to the point where I almost have stopped giving a shit, and begun moving on.
Steve is right, Tyler is right, and sure Twitter is right. You might even be able to build an argument that Twitter doesn't owe the long tail of ecosystem developer anything for helping make Twitter what is, which would be a tough argument in my opinion, but you can't argue that Twitter doesn't owes a certain amount of responsibility to end-users to keep platform operations real. This means not making firehose access something only the IBM, Intel's, Oracle's, and other big $$ companies of the world can afford to build (at scale) on Twitter. Sure the rest of us can still build on the public REST and streaming API, but what then? Why would we build on the platform if we reach the 100K token limit, we have no guarantees for the future--that is unless we are in the good ol boy club, and don't have anything to worry about.
I guess in the end, this is all about access (pun intended). Like with many other areas of the world, if you are lucky enough to have a priveleged position, which I'm not just pointing out the Klout / Twitter releationship. I'm also talking about the other cozy VC relationships I've witnessed companies enjoy with Twitter, versus the companies who I've seen that do not enjoy such priveleged access. The conversation cannot just be about he merits of Twitters decision, and the performance of Gnip and DataSift, without including the bigger ecosystem picture. Making the hard API ecosystem decision argument is only valid, if they take into consideration all of the macro issues that truly impact a platform, not just those of the platform owners and its most cherished partners and investors have. While I think Twitter and its investors feel like they own Twitter, us developers, and power users who have been on the platform since early days, also feel like we have a certain amount of ownership too--an ownership Twitter has shown they don't respect.
Twitter definitely holds the cards at the moment, but eventually as more of us power user realize we are just fine without a Twitter presence, Twitter will face becoming the NBC of social media--still relevant, but are you still truly relevant? Not forever.
