More Dreaming Of Better User Interface

I have lots of ideas, and no time to implement most of them. And so I decided it would be fun to begin airing some of them out. They certainly will not do anybody any good in my head and perhaps some of them might catch on.

Today’s thoughts are again around user interface. A while ago I wrote about UI and the problems I see with computer interfaces versus plain old paper. Since then I have been thinking more about why the computer screen in many respects feels like such a poor user interface, when compared to paper, or more broadly, the real physical world.

I have come to the conclusion that a big part of the issue is the speed with which it is possible to “pan and scan” on the computer vs what you can do in the physical world.

With a newspaper I can quickly scan all the headlines and even quickly flip between pages to scan them. In so doing I am taking in massive amounts of information and making decisions about where I want to go.

In the real physical world, I can, in less than a second, look out of my window and very quickly determine if there is anything I should take note of.

On the computer these things are much harder. Our rate of interaction is *much* slower. To get to the heart of the matter, the best way to break down the problem is to think about two issues: zoom, and as mentioned above, pan and scan.

Zoom is the ability to step back and look at the big picture, or to step forward and focus on one specific thing in greater detail. This is a very difficult thing to do on the computer and particularly the web. We have organized information on the web in terms of pages, which is fine, but the pages are generally rather small, and changing pages is painfully slow. I want to be able to “zoom” as quickly on the computer, and the web as I do in physical world with my eyes. I want to move instantly between seeing an overview of lots of information and focusing on one specific thing.

Pan and scan goes hand in hand with zooming. Because when you are zoomed in you can’t see very much and so you need to be able to very quickly look around in a space. Again in the real world, our eyes are exceedingly good at this. But on the computer, scrolling is very slow and difficult.

Interestingly, the iPhone has highlighted the importance of zoom and pan and scan. They have implemented zooming through their new “pinch” feature which allows you to touch the screen with two fingers and pinch the display to zoom out and to do the reverse action to zoom in. Similarly, and more familiarly, you can drag your finger across the display to move around.

The popularity of these features speaks to the incredible importance of both zoom and pan and scan. And while they are great advancements in user interface, they are still, to me, not sufficiently fluid.

The physical friction of the finger against the glass with both actions makes them feel more rigid. And the speed of their operation while impressive relative to other interfaces does not really compare to our ability to zoom and pan and scan with our eyes.

Last time I wrote about this issue, I discussed the virtual reality glasses, popularized as a concept back in the early 80s, and how they were used to allow one to pan and scan by turning ones head. And while one day that might be an interesting advancement in mainstream computer interface, I don’t see people wearing VR glasses in the near future to browse the web or work on a document.

Nevertheless, the fluidity of the VR glasses experience I believe can and should be implemented on mainstream computing devices – just without the headsets.

The idea is fairly straightforward.

Imagine a computer screen that could sense the distance of your finger from it, and where your finger was in relation to the screen. For example, it could tell if your finger was 2 inches or 3 inches away from the screen and it could tell if you were in the upper left hand corner or the bottom right, or anywhere in the middle. This would be a kind of 3D touch-less touch screen.

Now imagine that this interface allowed you to control zoom and pan and scan. The closer your finger gets to the screen, the more zoomed in it gets to a particular area. The further away it gets, the more zoomed out it gets. Moving left or right would be the equivalent of scrolling left or right. By not physically touching anything, the interaction is frictionless, and therefore much more fluid, allowing it to integrate much more natually with the way the eyes work. Of course the key to this is that all zooming be instantaneous. Of course with the graphics CPUs we have, this has been long ago possible, but given the physical interfaces we have been working with, it hasn’t been particularly important.

Another challenge is that in order for this to work well, software would have to be redesigned to take advantage of the interface. The good news is that I believe an optimized browser could solve most or much of the software problem. In this new paradigm, what you’d really want is to be able to scan around a large set of pages very quickly to see what is interesting. A new type of web browser could pre-fetch and render the surrounding pages for any given page. These pre-fetched pages would be laid out on one giant virtual surface, which would give you the ability to scan around a large number of pages very quickly.

For example, imagine the New York Times rendered in this way. When you go to the home page, it renders the home page, but also all of the pages that home page is linked to, and they are laid out in some yet to be determined but well organized fashion on this large virtual surface.

By positioning your finger, perhaps 6 inches from the screen you can see all of the pages on the virtual surface, but in thumbnail form. As you move in closer, things start to become just barely readable, and you can start to make out pictures. You then start panning around for what headlines and pictures look interesting. You find something interesting, and so you zoom in (move your finger) closer. You then lock the screen there, perhaps with a button on the keyboard accessible with your other hand, and you begin to read the page.

To me, this feels like a much more intuitive way to read the New York Times than the way I currently do it online, and it might even compete with the way the physical paper feels. But the most interesting aspect to me of this is that I think it would make big bold advertising more effective, more acceptable, and perhaps even actually pleasurable, in the same way many full page print ads are today. In other words, I think this not only represents a better web user experience, but a better monetization platform than today’s web based display advertising.

Of course there is a lot to think about here, and I am sure there is already some paper somewhere by some UX guy who has proposed something similar. After all there are no new ideas.

That said, I strongly suspect no one major is actually putting anything like this together, particularly combined with the re-conceptualization of the browser, which I think could be key to fixing the web’s display ad problem.

And so I’d love to get a conversation about this going and hear what people think. I know there would be lots of things to get right here including things like a need for hysteresis to compensate for jittery hands. And I am sure there are many more details. But what I am most curious about it whether people actually agree that this would be a better way to explore information on a computer.