Monday, May 5, 2008

Killing Twitter Before It Can hurt Us

This weekend there has been a fair amount of talk about the idea that Twitter is too important to be relied on as a centralized, non fault-tolerant platform. Both Dave Winer and Michael Arrington at TechCrunch are talking about the problem and how to fix it. The argument is that Twitter is not distributed. This means that if Twitter goes down no one can Twitter. It also means if Twitter goes out of business, or just screws up, we could lose all of our tweet history.

The idea being posited is to make a parallel open source run "twittersphere" that supports the Twitter API -- lets call it Twitshadow. Twitshadow is built to be a shadow of Twitter that all third party apps can use. The shadow could even be *the* way to talk to Twitter by implementing what would essentially be something like a write-through cache. Third party apps write to Twitshadow, and Twitshadow writes to Twitter. Or it could just be a second call that all third party apps make along with writting directly to Twitter. When Twitter goes down, this open source non-centralized Twitshadow keeps on chugging. And the more time that passes, the less important Twitter actually becomes because most of your data is in Twitshadow. In fact Twitshadow could even support sucking in tweets from Twitter so it could be a full replacement of Twitter, history and all.

This whole conversation is fascinating to me.

It is entirely possible that before Twitter makes its first penny, it will become too important to exist in its current form, and the community will feel it has to be replaced by an open source distributed framework. This should strike fear into the hearts of anyone who decides open their API. While the Open API strategy has clearly worked in terms of adoption, it may have worked too well. In fact it may have worked so well that Twitter may be killed before it has even really made it out of the womb, by people that find it so important that they can't afford to really have it be a company.

Technically, Twitter is ripe for destruction by open sourcers. It is a super simple API. It is loved by lots of really smart people that definitely have the intellect, the means, and the motivation to create Twitshadow. In fact open source will tend to be far more effective a development model for something like Twitter than closed source, in large part because a proper solution to this is a very pure computer science problem. This means the open source community will almost certainly be more clever about the solution that one tiny closed source company because there will be lots of heads focused on an easy to understand problem.

The lesson from all of this may be that communications apps can't live without an open API, But they can't live with them either. Of course I have been skeptical that any communication app can make money, and particularly Twitter, but I could not envision that they would and could be undermined as a platform like this. It is truly astonishing to watch.


  1. When was the last time you looked at your Twitter history past the period of time you last logged in?

  2. This is an interesting take. I agree completely with what you are saying. I think Social Portability will become a big issue in the years to come.

  3. Quite interesting. I've thought of "making my own twitter" several times--every time twitter goes down. I think "I can do this better!" and then I try to price things out and the feature I like most about twitter, SMS, can be really expensive to get a short code that works with all carriers. I think that's going to be the biggest hurdle for this decentralized one is the sms feature.

    Although conversely twitter can take advantage of that and know people will still be going to twitter when they want sms updates/notifications. All the internet api stuff can be handled through the network.

  4. the fact that people "rely" on data stored in Twitter baffles me. It's like relying on ephemeral conversations at a cocktail party. If an important concept comes out of Twitter (or a cocktail party conversation), write it down someplace more reliable.

  5. Scott - it's all about (some) blogger's inflated sense of self-importance, isn't it. It's not so much that they rely on the history of their tweets, it's that they can't bear the thought of _us_ being denied access to their 140-byte pearls of wisdom.

  6. Any twitter client, even the tiny bash script that I use for twittering, can easily save a log of past tweets locally. If you twitter from multiple locations, run a cron job to periodically update the local copy of your twitter history.

  7. I think you hit the nail on the head, this is another example of the

    1) Get eyes
    2) Something
    3) Profit!

    Loop failing. Although I have to wonder, if their uptime was better would anyone need this? They might have shot themselves in the foot with the server crashes.

  8. Honestly, outside of the Bay Area and a few select international regions, no one a) cares about Twitter and/or b) can describe what Twitter is.


    For those that use it (and I'm one), it is a great communication channel. But a centralized/decentralized argument is not a sign of the apocalypse.

  9. I never really caught on to Twitter. Maybe I never used it enough. But I do think that open source would not work for Twitter because:

    1. Server costs and maintenance (what's a huge open source project, besides Wikipedia,
    2. The brand name: twitter. It's a catchy name and is already in the lexicon.

    Also, I would possibly have a heart attack if Gmail disappeared, but I haven't lost any sleep over it yet.

  10. I don't really agree with you, Hank. Your argument could be used for any communication service, but it isn't because it doesn't make sense. Facebook is not reliable, so let's make an open "Facebook network", right?

    I know you could say that Twitter is new and it has a much higher chance of going down or going out of business, but who cares? If Twitter fails, someone will make something new just like it, but better and more reliable. That's the way business works. And it's a good thing, isn't it?

    And I think David is right. "When was the last time you looked at your Twitter history past the period of time you last logged in?"

  11. @Mattsy: Seriously. You could have said such a thing one year ago but today, so many organizations use Twitter to share informations with customers or users.
    And keep in mind I'm saying that from Paris, France. I did not scan the US market but, obviously, Twitter is going mainstream.

  12. Shasha,

    I think you misunderstand. *I* do not believe that Twitter should be disintermediated. I really could care less as I am not a heavy twitter user. The reason I am fascinated by this is that a chorus of influential people on the net has started saying this. My main point is that you really need to think about opening your API if powerful people can get together when you get successful and say that you are to "important" to operate as a single point of failure.

  13. Technically, Twitter is ripe for destruction by open sourcers.

    Twitter is a slick copy of txtmob, a pre-existing open source project.

  14. This comment has been removed by the author.

  15. "So many organizations use Twitter to share informations with customers or users"

    No they aren't, the vast majority of the world doesn't have a clue what Twitter is, Twitter has a long long way to go to go mainstream. Your average MySpace user (the mainstream) hasn't head of it, and doesn't care.

    The people who know what Twitter is are those in the tech industry and those smart enough to be tuned into blogs and RSS feeds, which is a very small minority in comparison to the mainstream.

  16. Dude, I love your blog. You don't suck! - Chase Saunders

  17. What a bunch of FUD, Hank.

    As other readers have already pointed out, this logic could be applied to any service at all.

    And it will be applied to many services in the next few years.

    The difference between reality and the Hank Williams reality is, apparently, whether users will seek first to replace services which work the way they want them to by offering open APIs, or the services which lock up your data, discourage interoperability, and inevitably fail to keep pace with their flexible and accessible counterparts.

    Even if Twitter were the very first implementation of this sort of app, the reason it might one day fail to be the most popular implementation will probably be because -- oh, I don't know -- the server is flaky and it's a ludicrously simple service. You even said yourself that no one could (or ought to, for that matter) make money on internet communication apps.

    They're just too damn simple.

    Did you happen to notice which challenge one of your readers identified as being the biggest hurdle to replacing Twitter? It was buying a good SMS number.

    So in other words, if we imagined that Twitter generated revenue, what would the service really boil down to?

    Electronic real estate!

    Hoo boy, you're right Hank Williams. Time for web technology firms to batten down the hatches and close up the APIs.

    If you don't, at least in the Hank Williams universe, some "powerful people" (ie. your users) might "get together" and, oh I don't know.. compete with you because your service lacks something that your users want?

    What I really find astonishing about all of this is that whenever I think about mirroring an existing service and abstracting my data away from a particular provider of that service, the primary shortcoming I am attempting to combat is that the service provider isn't inter-operable with its competitors.

    The final nail in the coffin for all the FUD in this article: Guess what Hank? This isn't the first time people have been worried about what you were worrying about now. There used to be this company that had a proprietary network for exchanging messages between users, or subscribing to common channels by which more than two users could exchange messages at a time. Whenever "powerful people" got together and decided the service was inadequate (interoperability on the list of complaints) the company thrashed and retaliated for many many years. The new "shadow messaging system" allowed users greater interoperability, and even gave them the option to leave a particular service provider altogether without too much trouble.

    I'd tell you what company and service I was talking about, but the truth is that this tale sounds identical to many many tales of companies trying to make a profit on electronic real estate. The logic must sound something like we built it first, and it's popular and proprietary, therefore we get to make a profit.

    Your problem Hank is that you refuse to respect the fact that these "powerful people" are users and their needs come before the needs of some company to be able to charge them.

    On the other hand, maybe you can be like that guy who brought a civil case against some open source software distributor on the premise that open source software was anticompetitive because proprietary software can't offer the advantages that open source software can.

    Damn this comment has turned into quite a long rant. Still, I think every point here is relevant. Including this last one:

    I really think Hank Williams may be losing his mind.


Note: Only a member of this blog may post a comment.