Mobile, ubiquitous and sensor-based computing

In 1991 Mark Weiser looked past the desktop and saw the computer dissolving into the world. His essay opened with a claim about attention rather than hardware. The most profound technologies, he wrote, are those that disappear (Weiser, 1991). Writing did it. Electric motors did it. Computing had not.

Three decades later the prediction half held. Computing did spread into daily life, but it arrived as a glowing rectangle that asks for attention many times an hour. That mismatch, calm promised and interruption delivered, is the contested question this field is built around.

Weiser's calm vision was a claim about attention, not about hardware

Weiser's target was not portability. He argued that carrying a powerful laptop is like owning one very important book, because it keeps attention on a single box (Weiser, 1991). His alternative was embodied virtuality: computing drawn out of its case and woven into the physical world.

He named three scales of device. Tabs were inch-scale, pads were foot-scale, and boards were yard-scale shared surfaces (Weiser, 1991). Two variables mattered more than the hardware. Location let a system act on where it was, with less input from the person. Scale let a system be present at many sizes without competing for one screen.

Abowd and Mynatt mapped the decade that followed and named three themes: natural interfaces, context-aware applications, and automated capture and access (Abowd and Mynatt, 2000). They added a fourth idea, everyday computing, which scales interaction with respect to time. Continuous interaction turns a tool into a companion, and a companion must handle interruption and resumption.

The critique arrived from Yvonne Rogers. The three dominant research programmes, context awareness, ambient intelligence and monitoring and tracking, had been fruitful but had not matched Weiser's world (Rogers, 2006). Her diagnosis is about initiative. The field tried to make computers act for people, and that is the goal it fell short on.

Rogers proposed the reverse: proactive people rather than proactive computing, and engaged living rather than calm living (Rogers, 2006). Technology should provoke learning and reflection, not quietly remove decisions. The smartphone tests both readings. It is ubiquitous computing at scale, and it is neither calm nor engaged in the way either account intended, because its business rewards capturing attention.

So the vision partly held and partly inverted. The devices spread. The calm did not follow automatically, and the reasons are commercial and social as much as technical.

What counts as context is a decision, not a fact

Context-aware computing needs a definition, and the field's default is Anind Dey's. He offered an operational one. Context is any information that can be used to characterise the situation of an entity, where an entity is a person, place or object relevant to the interaction (Dey, 2001). The definition is deliberately broad, and the breadth is the problem.

Dey unpacked the word into questions of who, what, where, when and why. That list looks tidy until a designer must choose what to sense. Is room temperature context for a messaging app? Is heart rate? The definition cannot answer, because any usable information qualifies.

His contribution was as much engineering as definition. The Context Toolkit separated sensing from application logic, so an application asks for a type of context rather than for a particular device (Dey, 2001). He also proposed a situation, a bundle of context that supports a higher-level reading.

The consequence for practice is that context is selected, not discovered. An engineer decides which signals count, and that decision is where a product's assumptions enter. The open question is whether context can be modelled in advance or only interpreted in the moment. The toolkit assumes the first, and much field evidence favours the second.

Every sensor has an error characteristic you must design around

Each sensor reports a physical quantity with an error characteristic, and the errors differ in kind. An accelerometer measures proper acceleration, the sum of motion and gravity. It detects a phone being lifted, and it misreads tilt while the phone moves. A gyroscope measures rotation rate and is accurate over short windows, but integrating its output accumulates drift. Location is accurate outdoors and poor indoors, and it fails differently in each. A light sensor is cheap and low power and reveals little alone. A microphone can infer activity and also conversation. A biometric sensor such as a heart-rate monitor is noisy and specific to the person wearing it.

The design rule follows from the physics. Choose a sensor whose failure you can see, fuse two that fail differently, and treat each reading as an estimate with an interval rather than as a fact.

Activity recognition turns estimates into a label and inherits their uncertainty. Ling Bao and Stephen Intille collected accelerometer data from twenty people wearing five sensors, with no researcher supervising, and trained classifiers to label twenty everyday activities (Bao and Intille, 2004). Their best decision trees reached about 84 per cent accuracy overall.

Two findings matter more than the headline. Some activities were recognised well using training data from other people, and others needed training data from the person being recognised. Reducing to two sensors, on thigh and wrist, cost only a little accuracy (Bao and Intille, 2004). Accuracy is therefore a property of the activity, the sensor placement and the training population together, and one number hides which of them failed.

Mobile use is interrupted, one-handed and expensive to keep awake

A desktop session assumes two hands, a large target and mains power. A phone cannot. Much mobile use happens while a person stands, walks or carries something, often with one hand and one thumb on a target a few millimetres across. These constraints are handled by enlarging the primary action and placing it where a thumb can reach.

The expensive constraint is the radio. Niranjan Balasubramanian and colleagues measured the energy of 3G, GSM and WiFi on real phones (Balasubramanian, Balasubramanian and Venkataramani, 2009). They found that energy depends on the traffic pattern, not only on the data volume. On 3G nearly sixty per cent of the energy of a typical transfer was tail energy, spent lingering in a high-power state after the bytes had already moved.

The radio kept drawing power for a fixed time whether or not data was sent. That yields a counter-intuitive rule. A few hundred bytes sent intermittently can cost more energy than a megabyte sent at once, because each burst pays the tail again (Balasubramanian, Balasubramanian and Venkataramani, 2009).

Batching updates and deferring them toward a deadline lowered energy in those experiments. An energy budget therefore constrains when a product may talk, not only how much it says. Battery and connectivity become design variables that a screen-level prototype hides.

Interruption research has the strongest evidence and the strangest findings

A notification is an interruption with a delivery mechanism, and the cost has been measured. The counter-intuitive result is that timing matters more than the content or the person's stated preference. Shamsi Iqbal and Brian Bailey built a system that defers notifications to breakpoints, the moments of transition between units of a task (Iqbal and Bailey, 2008).

Their models detected breakpoints reasonably well, around 55 per cent on unfamiliar tasks, but could not reliably classify their type (Iqbal and Bailey, 2008). In a second study, scheduling notifications to breakpoints reduced frustration and reaction time compared with delivering them at once, at the cost of a deferral of about a minute and a half. Relevance decided which breakpoint suited which message, and participants preferred the managed behaviour.

The design lesson is negative. A notification judged important is not therefore best delivered immediately. A short delay to a natural pause often serves the person better than an instant alert.

Attention also has a cheaper channel than a full screen. A glanceable display lets a person read a state in peripheral vision without switching task. UbiFit Garden used one: a small on-phone garden whose flowers and butterflies encoded recent physical activity at a glance (Consolvo et al., 2008). The display traded precision for a low cost of attention, which is the right trade when an approximate reading is enough.

Location is a service and a surveillance surface at the same time

Location is the most useful sensor and the most revealing. The same coordinate that powers a map also builds a movement history, and a movement history is a portrait. Alastair Beresford and Frank Stajano defined location privacy as the ability to prevent other parties from learning one's current or past location (Beresford and Stajano, 2003).

Their point is about scale. Knowing where someone was yesterday at half past four may be harmless. A record of every position at sub-metre accuracy, second by second, is a different kind of object, and the change of scale is a change of kind.

Their remedy was the mix zone, an area where identities are unlinked and pseudonyms are exchanged, with metrics built from anonymity sets and entropy (Beresford and Stajano, 2003). They tested the scheme against three million location samples from an indoor system. Privacy is therefore a property of the whole flow. Letting a user control who queries a coordinate does not stop that coordinate being joined to others later, so purpose limitation and retention belong in the design rather than in the paperwork.

Smart homes fail on installation, not on sensing

The internet of things carries the same questions into the home, where the setting supplies its own failure mode. A. J. Bernheim Brush and colleagues visited fourteen households that already had home automation and asked why the technology had not spread (Brush et al., 2011). The sensing worked. The adoption did not.

They named four barriers: high cost of ownership, inflexibility, poor manageability, and difficulty achieving security (Brush et al., 2011). The household case shows why the laboratory misleads. A motion sensor works in a laboratory because someone installed it, powered it, named it and maintained it. In a real home the same sensor may need structural changes, a compatible hub and a decision about who may control it, and each demand loses households.

Brush and colleagues recommended options that avoid the installation burden, offer security that people can configure with confidence, and let devices compose instead of each needing its own app (Brush et al., 2011). The domestic problem is therefore a work problem. It concerns plumbing, naming and maintenance rather than better sensing.

A field result is not a product that works in a kitchen

The gap between a laboratory result and a deployed product is this area's recurring warning, and UbiFit Garden shows both halves. The system ran for three weeks with twelve participants in their own lives, using on-body sensing, activity inference and the glanceable display (Consolvo et al., 2008). A controlled study would have produced cleaner sensing data. The field study produced something a laboratory cannot: evidence about how people react to being inferred, including when the inference is wrong and when they quietly stop using the system.

Field work carries costs a laboratory does not. Recruitment is slow because participants are not a convenience sample. Consent must be obtained in situ, where the setting changes and the recording is already running. Logging raises ethical questions when the data includes people who never agreed to be measured, such as a household member who crosses the room. Attrition is normal and should be reported rather than smoothed away. The strongest field reports treat non-use as a finding, not a failure.

The transferable rule is that a laboratory result shows an effect under chosen conditions, and a field deployment tests whether it survives the conditions a product will meet. Predicting a real outcome needs both, read as different kinds of evidence rather than as two strengths of the same kind.

What follows from this material

Battery budgets, sensor error, interruption cost and location privacy all reach their hardest test where the setting is safety critical, in alarm design, clinical systems and medical devices. There the cost of a missed signal or a false one is measured in harm rather than inconvenience, and the evidence about attention that guides a design choice becomes an obligation rather than a preference.