SSOTC06 Principles of Ubiquitous Computing

In retrospect, there are some notes to be made. Back then when I was reading the available literature and research, there was a kind of concensus that the peer-to-peer model of communication – device-to-device communication without intermediaries – would play a big role, as this would let the device deployments scale without requiring new or existing static network infrastructure. However, the bulk of the ubiquitous computing devices of today (sensors, smart phones, electrical consumption readers, etc.) rely on static communications infrastructure to function.

Also, the “Spam/Big Brother Society” is as relevant a danger as then. As I see it, the danger has merely evolved and is even more extensive today.

Today, more and more information about private individuals are collected with the justification of “with the information, we can show you more relevant advertisements”. The infrastructure of knowing who you are, what you think and who you know is in place to learn what stuff or services we might be currently missing.

At the moment the Spam Society is very benign. However, once this infrastructure and data is in place, it can be hard to remove it or to escape its reach, or to prevent it from transforming into a Big Brother Society. Even if one were to vanish as the target of the data collection today, the previously obtained information would still contain a lot of data that could be misused.

For example, what can happen if a political party with a violent agenda takes power, one way or another? If your profile indicates you have been thinking wrong thoughts, instead of getting advertisements, you would get night-time visitors taking you for a long car ride that culminates in a neck-shot in the woods. Interestingly enough, there is prior art in this kind of horror scenario: the Nazi government used census data which they data mined with IBM’s help to weed out people with Jewish ancestry.

As for the current state of ubiquitous computing devices, the smart phone stands as a lone king. It helps people organize their lives, entertains them, helps them keep connected with others, helps them document their lives with photographs and videos, and so on.

Although not quite as invisible as Weiser envisioned it, for those who have one, the smart phone is always present, ready to serve – and with modern UIs, it tries to not get in the way too much. I’d say at the moment the smart phone is closest to Weiser’s vision of calm technology. Also, over time, the smart phone has gotten only better and I expect this trend to continue.

Generally, a big downside I see with all current smart phones is the level of trust that needs to be placed on the maintainers and owners of the smartphone ecosystem to not abuse the data they collect (the location data, contact data, calendar data, etc.).

For example, Google backs up your WLAN passwords if you enable the Backup My Data option. It’s convenient in case you lose your phone, but do you know who in the end has access to the data and what they do with it? If you disable the option, the data is said to be removed. Fine; now, how will you know this to be true? You can’t know this, there is no way to check, so you just have to have trust. There are technical ways to remove the reliance on trust (e.g. encrypt the backup locally with a user-given key and then upload it), but at the moment such techniques are not used.

That said, I am a happy user of an Android smart phone. Android is open enough and the phone hardware it runs on is documented enough to let a community of enthusiasts make their own aftermarket firmware. Therefore, if I ever become unhappy with the stock Android, I can always install Cyanogenmod.

Here’s the 10/GUI video by C. Miller.

Why do I like it?

Basically, I agree with Mr. Miller’s points and motivations. I don’t like the mouse either so much, anymore. Mouse is fine and good just like a steam engine train is fine and good: it works, but technology has progressed far enough to provide better alternatives.

In 10/GUI, the Xeroxian UI paradigm of overlapping rectangular windows is radically transformed into application windows residing in a non-overlapping smooth continuum.

The clutter of a desktop is dramatically reduced. Reduced clutter means more efficiency, more time in the zone, and more stuff done.

To paraphrase Mr. Weiser when he talked about ubiquitous computing: technology will fade into the background, will not jump at your face, will not demand constant attention, but just lets you do your stuff. And meanwhile, everything is calm – and you’re staying in the zone.

Now, if only the 10/GUI could somehow combine the keyboard with the touch-part, or have an onscreen predictive keyboard and no physical keyboard at all…

My prediction is that in 15-25 years ubiquitous computing will be an everyday reality in the way Mark Weiser envisioned it to be – computing devices will blend into the environment and “disappear”, letting you do whatever it is you need to do without having to consciously use a computer. And then, when you see people doing their slow tai-chi like movements in the park, instead of exercising, they just might be accessing their data with the help of body-motion detecting wearable computing and reality-augmenting contact lenses.

And, perhaps, William Gibson is the next Jules Verne.

Stay tuned!

Just look back a bit think how portable MP3 players created a new kind of cultural phenomenon. Sure, we had Sony Walkmans before, but back in the days there never was such a massive amount of diverse music available for the portable players at such low cost. Today, an OK MP3 music player with tens of hours of playing time costs something like few tens of euros.

At the moment, a similar phenomenon is about to arise in the form of electronic book readers. Such readers utilize e-paper, which is a display technology giving more or less has the graphical “look and feel” of paper. Electricity is spent only to update the display, not to keep the existing picture intact (unlike LCDs, TFTs and other such display technologies) – this translates to very low power consumption.

There are nowadays many such devices in the market. Personally, my greatest interest is towards the Bebook reader (I haven’t yet bought it). Also, the Sony PRS505 seems interesting as it has a touch screen which makes it easier to make notes and annotate things. There is also a bigger list of e-book reader gadgets at Wikipedia.

Currently the screens are black-and-white, and can’t be updated as fast as, say, LCD screens. But I heard it through the grapevine that this will change soon with the introduction of super-thin, transparent, e-paper screens which can do color too. (Carbon nanotubes, is there nothing they can’t do?)

Once those e-book reader devices truly take off, the result will be that much more people will be reading books more often than before. Also, books which have been “forgotten” will have a new life as it will be easier to read them not just sitting at your laptop, but anywhere you go. Even if you’re on a budget after buying a fancy e-paper reader, you still can find quality reading and many, many classics. For example, there are a lot of Project Gutenberg books (i.e. books which have their copyrights expired).

As I consider reading (quality) books, learning and immersing ones mind in new thoughts and new perspectives to be absolutely beneficial for the individual, I can only see huge benefits with this cultural phenomenon which will increase dramatically the “consumption” of books. In fact, I think in the long term this new “library in your pocket”-phenomenon will have the equivalent effect of dropping a black monolith in the middle of a gang of apes.

Here is the Gartner Group’s list in a nutshell:

• Multicore and hybrid processors
• Virtualisation and fabric computing
• Social networks and social software
• Cloud computing and cloud/Web platforms
• Web mashups
• User Interface
• Ubiquitous computing
• Contextual computing
• Augmented reality
• Semantics

The list seems to be mirroring the biggest trends which are going on already today. So I don’t consider the list so much as any kind of real prediction, but more a validation of how, according to Gartner, things will continue to grow (especially in the business setting).

I’m sure Gartner had to prune their original lists to get down to just ten. If I could add to their list, I would add the following.

Ultramobile computers – in a few years people will wonder how we could haul those monstrously-sized laptop computers with us. The more mobile computers are, the more people will be able to take their internet-connected laptops along truly wherever they go. Expect to see some very interesting progress on this front.

The open revolution – more and more hardware vendors will not try to hinder hobbyists from modifying the hardware or the software. Non-hobbyist consumers will be able to use the device as COTS devices for their intended purposes. Hobbyists will be able to expand the devices in new, unforeseen and innovative ways.

This will result in new applications and modifications, as the hobbyists scratch their personal itches to fullfill their bright ideas which would otherwise never see the daylight due to corporate inefficiencies in communication and decision-making, corporate bureocracy and the short-term profit motive thinking.

Smart companies can benefit from this open revolution, since if the hardware vendors allow selling third-party modifications as a service without harassing the modifiers, the hardware vendors will simply sell more devices without additional risk to themselves. The downside for the hardware vendors is the loss of control over how the device is used, and of course a slight dislocation in the value chain.

Brain-Computer Interfaces (BCI) – already now, there are consumer-grade EEG devices out on the market, for example the OCZ Neural Impulse Actuator. The Emotiv EPOC is following later this year. BCI devices are input devices which measure brainwave activity and (facial) muscle activation and interpret these signals to a form understood by computers. Essentially with these devices one can control the computer with “thinking” alone. There is a small learning curve, of course. The possibilities with these kind of interfaces are absolutely huge in many areas: home, office, military, medical, etc.

Sidenote: Currently, in a laboratory setting, it is possible to also alter the brain functionality in a non-permanent way with strong magnetic fields (so-called Transcranial Magnetic Stimulation, or TMS). When BCI devices become more ubiquitous it will naturally cause more research into devices which can be used to also “send” information directly to the brain (by exciting neurons in certain areas), i.e. consumer-grade TMS. This will bring the world closer to the direct neural jacks as popularized by William Gibson’s cyberpunk novels. I estimate this will happen in maybe 20 years, so don’t hold your breath.

There’s probably more I could come up with, but I think now it’s time to wrap this up. I might tell you, if you paid me as much as Gartner gets from each report ;)

Now that you’ve seen what it can do, maybe you want to try it out yourself. Have a look at the API and then study the the examples. There are many libraries too, for example this particle system physics engine.

When I first heard of Processing, I was somewhat surprised how it could provide such varied and highly amazing and very aesthetic-looking things. First of all, the API did not seem very complex – I was kind of expecting method calls like “makeCoolAntialiasedLensFlareBlobs” and “operateTheWholeWorldAsAKindOfTwistedDonut” and “twistTheUniverseInFrozenHyperspeedLikeWow” and so on. My surprise only grew after checking the source code of some nicely graphical gadgets. The programs were not so complex, really – there was a clever algorithm and at least an equally clever idea behind the whole thing, and, well, that was it. Examining the source code was like a fiesta of “a-ha!” and “of course!”.

I shall let the results of Processing speak for itself, as my words can only approximate the visual aesthetics one can create with it. A good place to start is to check out the “Exhibition” at Processing site.

There are actually a lot of people working with Processing, and there is no point to try to list them all here. However, I do want to point out one work in particular, not because I think it would be the best program ever done with Processing (that’s not the case), but because I think it would be fun to have such functionality as a built-in feature to all PIM and e-mail programs.

Check out My Map by Christopher Baker. Mr. Baker did a visualization of the relationships of people in his address book, using information from his e-mail traffic. Now how cool is that! Imagine, if you had it in your e-mail program… What would your data look like?

Or, suppose you’re a painter, soon to paint a house. You grab some blueberries for the colour and a tiny sponge with which you will slowly wipe the house blue. It’s a bit awkward, not to mention terribly slow, and the quality is bad, but that’s the way it’s always been, so why change it? “But that’s absurd! Nobody does things in such a difficult way!”, you say. Well, suppose you’re a software developer…

You sit next to a screen where the computer displays you an interface which is based on a very much unchanged windowing paradigm from the 1970s. It may look a bit neater and cleaner, but it is the same thing. You have a powerful machine, capable of creating virtual worlds in more or less photorealistic 3D representations, with surround and even three-dimensional sound, yet you will only utilize clumsy 2D interfaces.

Instead of your fingers, your voice or your whole desk, you interact with the computer using a pointing device which resembles a rodent. You don’t touch the screen, you touch the rodent-like apparatus.

When you create a program, you write it as if you were writing a novel on a sheet of paper, using a mechanical typewriter. You command the computer by laying out explicit instructions in linear, time-increasing order, in essentially one dimension (like one tape of a Turing machine), even though the presentation is two-dimensional (like a page in a word processor).

You do not utilize even a fraction of the visualization and usability capabilities the computer could offer you. Of course things could be even worse. You could still be entering data into the computer using paper cards with holes punched into them.

All this is partly due to the burden of the past influencing the thinking of today – we’re so used to the things the way they are that we do not stop to think how things could be if they were re-invented, if everything were to become an upgraded version of whatever they are now. This problem is especially visible within the software industry, where one has to move, move and move continuously, like a shark which needs to swim all the time in order to stay alive. There is so much motion and overall hoopla but no occasions to just slow down and sit down and think about things without hurry. These idle moments are the ones which create innovations, but unfortunately there are so few of those moments available.

I say it’s time to stop for a moment, and think how to shed that burden of the times gone by and how to create something new, something which heeds the great pioneering work of the past but which does not just blindly repeat it for tradition’s sake, something which is not a living piece of history in a new, shinier package.

Some researchers, especially in the ubiquitous computing field, seem to feel the same frustration with having to use the “wrong tools for the right job”. As a result, they have approached the aforementioned problems with fresh ideas, especially for human-computer interaction. Here are some neat and stimulating examples of such research, to show you that there is indeed hope for better tools!

• Tangible Functional Programming lecture
• IntuiLab demo of Mitsubish Electric Research Lab’s DiamondTouch
• Microsoft’s Surface Computing