The Power Of Serendipity

I believe that a big part of the next phase of computing productivity will be around the serendipitous presentation of useful information to the user.

In the old days, we used to talk a lot about artificial intelligence, but the term has fallen out of favor because, artificial intelligence it turns out just wasn’t all that intelligent. Or perhaps the expectations were just too high. And yet today we have techniques both related to machine learning, and more broadly graph data models, that can substantially increase the ability of the user to discover relationships that they would otherwise not see.

Ideally, your software helps you to see latent actionable information. But in such a system, you don’t ever want to be dependent on the insights that the software may glean. When you are dependent, you are not at all forgiving. And so, whether it is a relationship that the software can see and present, or a recommendation that the software can calculate, it is critical that the information be presented in a way that is secondary – being available without being obtrusive.

This is the idea behind current recommendation systems implemented in Amazon, and Netfix, though they are very limited. You are happy when they suggest something that works, but it doesn’t upset you when none of their suggestions are particularly helpful.

The truth is our web systems tend to capture an incredible amount of information about us. And depending on the category of software, there are likely patterns and connections that we don’t see that the system *should* be able to see. The software, not being human, will tend to see things that it thinks are important that you may not. But every time it gets it right and suggests something really useful, you are happy.

Serendipity!

And so, in the next phase of the web’s evolution I believe a key element is going to be how to marry serendipity to user interface. When your systems have many more potentially helpful ideas, suggestions, and connections, how do you expose these artifacts in an unobtrusive way so that the process does not become a burden to users? Certainly at some level, the more insights you can present to the user the better. But that is only true to the extent that the insights both *tend* to be helpful, and when they aren’t helpful are not burdensome.

This tension between being helpful and being annoying is a common one, even among human beings and so it is on some level familiar territory. Solutions may to some extent application specific, but I do believe there is an opportunity to create generally useful metaphors for unobtrusively exploring latent insights and connections.

We have, at Kloudshare, been thinking a lot about this issue, which is why it has been on my mind. I think we have some interesting ideas in this area, which in the next several months we will certainly start to share. But that doesn’t mean we can’t have a broader conversation about the big picture right now.

What other tools do you use that are serendipitously helpful? What do you like and not like about them?