Introduction: Olden Days
In February 1984, seeking a reprieve from the very cold and windy streets of Boston, Massachusetts, I ducked inside of a computer store. I spied the normal array of IBM PCs and peripherals, the Apple ][, probably even an Atari system. Prominently displayed at the front of the store, I spied my first Macintosh. It wasn’t known as a Mac 128K or anything like that. It was simply Macintosh. I walked up to it, intrigued – already, the Reality Distortion Field was capable of luring geeks like me to their doom – and spied the unfamiliar graphical desktop and the cute little mouse. Sitting down at the chair before the machine, I grasped the mouse, and moved the cursor across the screen. But how do I get it to do anything? I wondered. Click. Nothing. Click, drag – oh look some of these things changed color! But now what? Gah. This is too hard.
That’s when I gave up, pushed myself away from that first Macintosh, and pronounced this experiment in ‘intuitive’ computing a failure. Graphical computing isn’t intuitive, that’s a bit of a marketing fib. It’s a metaphor, and you need to grasp the metaphor – need to be taught what it means – to work fluidly within the environment. The metaphor is easy to apprehend if it has become the dominant technique for working with computers – as it has in 2010. Twenty-six years ago, it was a different story. You can’t assume that people will intuit what to do with your abstract representations of data or your arcane interface methods. Intuition isn’t always intuitively obvious.
A few months later I had a job at a firm which designed bar code readers. (That, btw, was the most boring job I’ve ever had, the only one I got fired from for insubordination.) We were designing a bar code reader for Macintosh, so we had one in-house, a unit with a nice carrying case so that I could ‘borrow’ it on weekends. Which I did. Every weekend. The first weekend I got it home, unpacked it, plugged it in, popped in the system disk, booted it, ejected the system disk, popped in the applications disk, and worked my way through MacPaint and MacWrite and on to my favorite application of all – Hendrix.
Hendrix took advantage of the advanced sound synthesis capabilities of Macintosh. Presented with a perfectly white screen, you dragged the mouse along the display. The position, velocity, and acceleration of the pointer determined what kind of heavily altered but unmistakably guitar-like sounds came out of the speaker. For someone who had lived with the bleeps and blurps of the 8-bit world, it was a revelation. It was, in the vernacular of Boston, ‘wicked’. I couldn’t stop playing with Hendrix. I invited friends over, showed them, and they couldn’t stop playing with Hendrix. Hendrix was the first interactive computer program that I gave a damn about, the first one that really showed me what a computer could be used for. Not just pushing paper or pixels around, but an instrument, and an essential tool for human creativity.
Everything that’s followed in all the years since has been interesting to me only when it pushes the boundaries of our creativity. I grew entranced by virtual reality in the early 1990s, because of the possibilities it offered up for an entirely new playing field for creativity. When I first saw the Web, in the middle of 1993, I quickly realized that it, too, would become a cornerstone of creativity. That roughly brings us forward from the ‘olden days’, to today.
This morning I want to explore creativity along the axis of three classes of devices, as represented by the three Apple devices that I own: the desktop (my 17” MacBook Pro Core i7), the mobile (my iPhone 3GS 32Gb), and the tablet (my iPad 16GB 3G). I will draw from my own experience as both a user and developer for these devices, using that experience to illuminate a path before us. So much is in play right now, so much is possible, all we need do is shine a light to see the incredible opportunities all around.
I: The Power of Babel
I love OSX, and have used it more or less exclusively since 2003, when it truly became a useable operating system. I’m running Snow Leopard on my MacBook Pro, and so far have suffered only one Grey Screen Of Death. (And, if I know how to read a stack trace, that was probably caused by Flash. Go figure.) OSX is solid, it’s modestly secure, and it has plenty of eye candy. My favorite bit of that is Spaces, which allows me to segregate my workspace into separate virtual screens.
Upper left hand space has Mail.app, upper right hand has Safari, lower right hand has TweetDeck and Skype, while the lower left hand is reserved for the task at hand – in this case, writing these words. Each of the apps, except Microsoft Word, is inherently Internet-oriented, an application designed to facilitate human communication. This is the logical and inexorable outcome of a process that began back in 1969, when the first nodes began exchanging packets on the ARPANET. Phase one: build the network. Phase two: connect everything to the network. Phase three: PROFIT!
That seems to have worked out pretty much according to plan. Our computers have morphed from document processors – that’s what most computers of any stripe were used for until about 1995 – into communication machines, handling the hard work of managing a world that grows increasingly connected. All of this communication is amazing and wonderful and has provided the fertile ground for innovations like Wikipedia and Twitter and Skype, but it also feels like too much of a good thing. Connection has its own gravitational quality – the more connected we become, the more we feel the demand to remain connected continuously.
We salivate like Pavlov’s dogs every time our email application rewards us with the ‘bing’ of an incoming message, and we keep one eye on Twitter all day long, just in case something interesting – or at least diverting – crosses the transom. Blame our brains. They’re primed to release the pleasure neurotransmitter dopamine at the slightest hint of a reward; connecting with another person is (under most circumstances) a guaranteed hit of pleasure.
That’s turned us into connection junkies. We pile connection upon connection upon connection until we numb ourselves into a zombie-like overconnectivity, then collapse and withdraw, feeling the spiral of depression as we realize we can’t handle the weight of all the connections that we want so desperately to maintain.
Not a pretty picture, is it? Yet the computer is doing an incredible job, acting as a shield between what our brains are prepared to handle and the immensity of information and connectivity out there. Just as consciousness is primarily the filtering of signal from the noise of the universe, our computers are the filters between the roaring insanity of the Internet and the tidy little gardens of our thoughts. They take chaos and organize it. Email clients are excellent illustrations of this; the best of them allow us to sort and order our correspondence based on need, desire, and goals. They prevent us from seeing the deluge of spam which makes up more than 90% of all SMTP traffic, and help us to stay focused on the task at hand.
Electronic mail was just the beginning of the revolution in social messaging; today we have Tweets and instant messages and Foursquare checkins and Flickr photos and YouTube videos and Delicious links and Tumblr blogs and endless, almost countless feeds. All of it recommended by someone, somewhere, and all of it worthy of at least some of our attention. We’re burdened by too many web sites and apps needed to manage all of this opportunity for connectivity. The problem has become most acute on our mobiles, where we need a separate app for every social messaging service.
This is fine in 2010, but what happens in 2012, when there are ten times as many services on offer, all of them delivering interesting and useful things? All these services, all these websites, and all these little apps threaten to drown us with their own popularity.
Does this mean that our computers are destined to become like our television tuners, which may have hundreds of channels on offer, but never see us watch more than a handful of them? Do we have some sort of upper boundary on the amount of connectivity we can handle before we overload? Clay Shirky has rightly pointed out that there is no such thing as information overload, only filter failure. If we find ourselves overwhelmed by our social messaging, we’ve got to build some better filters.
This is the great growth opportunity for the desktop, the place where the action will be happening – when it isn’t happening in the browser. Since the desktop is the nexus of the full power of the Internet and the full set of your own data (even the data stored in the cloud is accessed primarily from your desktop), it is the logical place to create some insanely great next-generation filtering software.
That’s precisely what I’ve been working on. This past May I got hit by a massive brainwave – one so big I couldn’t ignore it, couldn’t put it down, couldn’t do anything but think about it obsessively.
I wanted to create a tool that could aggregate all of my social messaging – email, Twitter, RSS and Atom feeds, Delcious, Flickr, Foursquare, and on and on and on. I also wanted the tool to be able to distribute my own social messages, in whatever format I wanted to transmit, through whatever social message channel I cared to use.
Then I wouldn’t need to go hither and yon, using Foursquare for this, and Flickr for that and Twitter for something else. I also wouldn’t have to worry about which friends used which services; I’d be able to maintain that list digitally, and this tool would adjust my transmissions appropriately, sending messages to each as they want to receive them, allowing me to receive messages from each as they care to send them.
That’s not a complicated idea. Individuals and companies have been nibbling around the edges of it for a while.
I am going the rest of the way, creating a tool that functions as the last 'social message manager' that anyone will need. It’s called Plexus, and it functions as middleware – sitting between the Internet and whatever interface you might want to cook up to view and compose all of your social messaging.
Now were I devious, I’d coyly suggest that a lot of opportunity lies in building front-end tools for Plexus, ways to bring some order to the increasing flow of social messaging. But I’m not coy. I’ll come right out and say it: Plexus is an open-source project, and I need some help here. That’s a reflection of the fact that we all need some help here. We’re being clubbed into submission by our connectivity. I’m trying to develop a tool which will allow us to create better filters, flexible filters, social filters, all sorts of ways of slicing and dicing our digital social selves. That’s got to happen as we invent ever more ways to connect, and as we do all of this inventing, the need for such a tool becomes more and more clear.
We see people throwing their hands up, declaring ‘email bankruptcy’, quitting Twitter, or committing ‘Facebookicide’, because they can’t handle the consequences of connectivity.
We secretly yearn for that moment after the door to the aircraft closes, and we’re forced to turn our devices off for an hour or two or twelve. Finally, some time to think. Some time to be. Science backs this up; the measurable consequence of over-connectivity is that we don’t have the mental room to roam with our thoughts, to ruminate, to explore and play within our own minds. We’re too busy attending to the next message. We need to disconnect periodically, and focus on the real. We desperately need tools which allow us to manage our social connectivity better than we can today.
Once we can do that, we can filter the noise and listen to the music of others. We will be able to move so much more quickly – together – it will be another electronic renaissance: just like 1994, with Web 1.0, and 2004, with Web2.0.
That’s my hope, that’s my vision, and it’s what I’m directing my energies toward. It’s not the only direction for the desktop, but it does represent the natural evolution of what the desktop has become. The desktop has been shaped not just by technology, but by the social forces stirred up by our technology.
It is not an accident that our desktops act as social filters; they are the right tool at the right time for the most important job before us – how we communicate with one another. We need to bring all of our creativity to bear on this task, or we’ll find ourselves speechless, shouted down, lost at another Tower of Babel.
II: The Axis of Me-ville
Three and a half weeks ago, I received a call from my rental agent. My unit was going on the auction block – would I mind moving out? Immediately? I’ve lived in the same flat since I first moved to Sydney, seven years ago, so this news came as quite a shock.
I spent a week going through the five states of mourning: denial, anger, bargaining, depression, and acceptance. The day I reached acceptance, I took matters in hand, the old-fashioned way: I went online, to domain.com.au, and looked for rental units in my neighborhood.
Within two minutes I learned that there were two units for rent within my own building!
When you stop to think about it, that’s a bit weird. There were no signs posted in my building, no indication that either of the units were for rent. I’d heard nothing from the few neighbors I know well enough to chat with. They didn’t know either. Something happening right underneath our noses – something of immediate relevance to me – and none of us knew about it. Why? Because we don’t know our neighbors.
For city dwellers this is not an unusual state of affairs. One of the pleasures of the city is its anonymity. That’s also one of it’s great dangers. The two go hand-in-hand. Yet the world of 2010 does not offer up this kind of anonymity easily. Consider: we can re-establish a connection with someone we went to high school with, thirty years ago – and really never thought about in all the years that followed – but still not know the names of the people in the unit next door, names you might utter with bitter anger after they’ve turned up the music again. How can we claim that there’s any social revolution if we can’t be connected to people whom we’re physically close to? Emotional closeness is important, and financial closeness (your coworkers) is also salient, but both should be trumped by the people who breathe the same air as you.
It is almost impossible to bridge the barriers that separate us from one another, even when we’re living on top of each other.
This is where the mobile becomes important, because the mobile is the singular social device. It is the place where our of the human relationships reside. (Plexus is eventually bound for the mobile, but in a few years’ time, when the devices are nimble enough to support it.) Yet the mobile is more than just the social crossroads. It is the landing point for all of the real-time information you need to manage your life.
On the home page of my iPhone, two apps stand out as the aids to the real-time management of my life: RainRadar AU and TripView. I am a pedestrian in Sydney, so it’s always good to know when it’s about to rain, how hard, and how long. As a pedestrian, I make frequent use of public transport, so I need to know when the next train, bus or ferry is due, wherever I happen to be. The mobile is my networked, location-aware sensor. It gathers up all of the information I need to ease my path through life. This demonstrates one of the unstated truisms of the 21st century: the better my access to data, the more effective I will be, moment to moment. The mobile has become that instantaneous access point, simply because it’s always at hand, or in the pocket or pocketbook or backpack. It’s always with us.
In February I gave a keynote at a small Melbourne science fiction convention. After I finished speaking a young woman approached me and told me she couldn’t wait until she could have some implants, so her mobile would be with her all the time. I asked her, “When is your mobile ever more than a few meters away from you? How much difference would it make? What do you gain by sticking it underneath your skin?” I didn’t even bother to mention the danger from all that subcutaneous microwave radiation. It’s silly, and although our children or grandchildren might have some interesting implants, we need to accept the fact that the mobile is already a part of us.
We’re as Borg-ed up as we need to be. Probably we’re more Borg-ed up than we can handle.
It’s not just that our mobiles have become essential. It’s getting so that we can’t put them down, even in situations when we need to focus on the task at hand – driving, or having dinner with your partner, or trying to push a stroller across an intersection. We’re addicted, and the first step to treating that addiction is to admit we have problem. But here’s the dilemma: we're working hard to invent new ways to make our mobiles even more useful, indispensable and alluring.
We are the crack dealers. And I’m encouraging you to make better crack. Truth be told, I don’t see this ‘addiction’ as a bad thing, though goodness knows the tabloid newspapers and cultural moralists will make whatever they can of it. It’s an accommodation we will need to make, a give-and-take. We gain an instantaneous connection to one another, a kind of cultural ‘telepathy’ that would have made Alexander Graham Bell weep for joy.
But there's more: we also gain a window into the hitherto hidden world of data that is all around us, a shadow and double of the real world.
For example, I can now build an app that allows me to wander the aisles of my local supermarket, bringing all of the intelligence of the network with me as I shop. I hold the mobile out in front of me, its camera capturing everything it sees, which it passes along to the cloud, so that Google Goggles can do some image processing on it, and pick out the identifiable products on the shelves.
This information can then be fed back into a shopping list – created by me, or by my doctor, or by bank account – because I might be trying to optimize for my own palette, my blood pressure, or my budget – and as I come across the items I should purchase, my mobile might give a small vibration. When I look at the screen, I see the shelves, but the items I should purchase are glowing and blinking.
The technology to realize this – augmented reality with a few extra bells and whistles – is already in place. This is the sort of thing that could be done today, by someone enterprising enough to knit all these separate threads into a seamless whole. There’s clearly a need for it, but that’s just the beginning. This is automated, computational decision making. It gets more interesting when you throw people into the mix.
Consider: in December I was on a road trip to Canberra. When I arrived there, at 6 pm, I wondered where to have dinner. Canberra is not known for its scintillating nightlife – I had no idea where to dine. I threw the question out to my 7000 Twitter followers, and in the space of time that it took to shower, I had enough responses that I could pick and choose among them, and ended up having the best bowl of seafood laksa that I’d had since I moved to Australia!
That’s the kind of power that we have in our hands, but don’t yet know how to use.
We are all well connected, instantaneously and pervasively, but how do we connect without confusing ourselves and one another with constant requests? Can we manage that kind of connectivity as a background task, with our mobiles acting as the arbiters? The mobile is the crossroads, between our social lives, our real-time lives, and our data-driven selves. All of it comes together in our hands. The device is nearly full to exploding with the potentials unleashed as we bring these separate streams together. It becomes hypnotizing and formidable, though it rings less and less. Voice traffic is falling nearly everywhere in the developed world, but mobile usage continues to skyrocket. Our mobiles are too important to use for talking.
Let’s tie all of this together: I get evicted, and immediately tell my mobile, which alerts my neighbors and friends, and everyone sets to work finding me a new place to live. When I check out their recommendations, I get an in-depth view of my new potential neighborhoods, delivered through a marriage of augmented reality and the cloud computing power located throughout the network. Finally, when I’m about to make a decision, I throw it open for the people who care enough about me to ring in with their own opinions, experiences, and observations. I make an informed decision, quickly, and am happier as a result, for all the years I live in my new home.
That’s what’s coming. That’s the potential that we hold in the palms of our hands. That’s the world you can bring to life.
III: Through the Looking Glass
Finally, we turn to the newest and most exciting of Apple’s inventions. There seemed to be nothing new to say about the tablet – after all, Bill Gates declared ‘The Year of the Tablet’ way back in 2001. But it never happened. Tablets were too weird, too constrained by battery life and weight and, most significantly, the user experience. It’s not as though you can take a laptop computer, rip away the keyboard and slap on a touchscreen to create a tablet computer, though this is what many people tried for many years. It never really worked out for them.
Instead, Apple leveraged what they learned from the iPhone’s touch interface. Yet that alone was not enough. I was told by sources well-placed in Apple that the hardware for a tablet was ready a few years ago; designing a user experience appropriate to the form factor took a lot longer than anyone had anticipated. But the proof of the pudding is in the eating: iPad is the most successful new product in Apple’s history, with Apple set to manufacture around thirty million of them over the next twelve months. That success is due to the hard work and extensive testing performed upon the iPad’s particular version of iOS.
It feels wonderfully fluid, well adapted to the device, although quite different from the iOS running on iPhone. iPad is not simply a gargantuan iPod Touch. The devices are used very differently, because the form-factor of the device frames our expectations and experience of the device.
Let me illustrate with an example from my own experience: I had a consulting job drop on me at the start of June, one which required that I go through and assess eighty-eight separate project proposals, all of which ran to 15 pages apiece. I had about 48 hours to do the work. I was a thousand kilometers from these proposals, so they had to be sent to me electronically, so that I could then print them before reading through them. Doing all of that took 24 of the 48 hours I had for review, and left me with a ten-kilo box of papers that I’d have to carry, a thousand kilometers, to the assessment meeting. Ugh.
Immediately before I left for the airport with this paper ball-and-chain, I realized I could simply drag the electronic versions of these files into my Dropbox account. Once uploaded, I could access those files from my iPad – all thousand or so pages. Working on iPad made the process much faster than having to fiddle through all of those papers; I finished my work on the flight to my meeting, and was the envy of all attending – they wrestled with multiple fat paper binders, while I simply swiped my way to the next proposal.
This was when I realized that iPad is becoming the indispensable appliance for the information worker.
You can now hold something in your hand that has every document you’ve written; via the cloud, it can hold every document anyone has ever written. This has been true for desktops since the advent of the Internet, but it hasn’t been as immediate. iPad is the page, reinvented, not just because it has roughly the same dimensions as a page, but because you interact with it as if it were a piece of paper. That’s something no desktop has ever been able to provide.
We don’t really have a sense yet for all the things we can do with this ‘magical’ (to steal a word from Steve Jobs) device.
Paper transformed the world two thousand years ago. Moveable type transformed the world five hundred years ago. The tablet, whatever it is becoming – whatever you make of it – will similarly reshape the world. It’s not just printed materials; the tablet is the lightbox for every photograph ever taken anywhere by anyone. The tablet is the screen for every video created, a theatre for every film produced, a tuner to every radio station that offers up a digital stream, and a player for every sound recording that can be downloaded.
All of this is here, all of this is simultaneously present in a device with so much capability that it very nearly pulses with power.
iPad is like an Formula One Ferrari, one we haven’t even gotten out of first gear. So stretch your mind further than the idea of the app. Apps are good and important, but to unlock the potential of iPad it needs lots of interesting data pouring into it and through it. That data might be provided via an application, but it probably doesn’t live within the application – there’s not enough room in there. Any way you look at it, iPad is a creature of the network; it is a surface, a looking glass, which presents you a view from within the network.
What happens when the network looks back at you?
At the moment iPad has no camera, though everyone expects a forward-facing camera to be in next year’s model. That will come so that Apple can enable FaceTime. (With luck, we’ll also see a Retina Display, so that documents can be seen in their natural resolution.) Once the iPad can see you, it can respond to you. It can acknowledge your presence in an authentic manner. We’re starting to see just what this looks like with the recently announced Xbox Kinect.
This is the sort of technology which points all the way back to the infamous ‘Knowledge Navigator’ video that John Sculley used to create his own Reality Distortion Field around the disaster that was the Newton. Decades ahead of its time, the Knowledge Navigator pointed toward Google and Wikipedia and Milo, with just a touch of Facebook thrown in. We’re only just getting there, to the place where this becomes possible.
These are no longer dreams, these are now quantifiable engineering problems.
This sort of thing won’t happen on Xbox, though Microsoft or a partner developer could easily write an app for it. But that’s not where they’re looking, this is not about keeping you entertained. The iPad can entertain you, but that’s not its main design focus. It is designed to engage you, today with your fingers, and soon with your voice and your face and your gestures. At that point it is no longer a mirror; it is an entity on its own. It might not pass the Turing Test, but we’ll anthropomorphize it nonetheless, just as we did with Tamagotchi and Furby. It will become our constant companion, helping us through every situation. And it will move seamlessly between our devices, from iPad to iPhone to desktop. But it will begin on iPad.
Because we are just starting out with tablets, anything is possible. We haven’t established expectations which guide us into a particular way of thinking about the device. We’ve had mobiles for nearly twenty years, and desktops for thirty. We understand both well, and with that understanding comes a narrowing of possibilities. The tablet is the undiscovered country, virgin, green, waiting to be explored. This is the desktop revolution, all over again. This is the mobile revolution, all over again. We’re in the right place at the right time to give birth to the applications that will seem commonplace in ten or fifteen years.
I remember the VisiCalc, the first spreadsheet. I remember how revolutionary it seemed, how it changed everyone’s expectations for the personal computer. I also remember that it was written for an Apple ][.
You have the chance to do it all again, to become the ‘mothers of innovation’, and reinvent computing. So think big. This is the time for it. In another few years it will be difficult to aim for the stars. The platform will be carrying too much baggage. Right now we all get to be rocket scientists. Right now we get to play, and dream, and make it all real.