What a deployment can claim: the field study as an argument about a setting

A laboratory result holds while its conditions hold: rested volunteers, a task set by the researcher, a device that behaves. Move the system into a household, a ward or a shopping centre and every condition moves with it.

What a deployment can then claim is this unit's question. This first of ten units fixes the term the other nine work in: what a result from a setting can be asked to support. The answer is that the result is a claim about a system, a setting and a method at once. Its reach is fixed by the design rather than settled afterwards.

A deployment result is a claim about three things at once

The sharpest evidence is a trial run as a study of trials. Brown, Reeves and Sherwood deployed a photo-sharing system and watched the method rather than the product (Brown, Reeves and Sherwood, 2011). Demand characteristics are the adjustments people make to what they take to be the investigators' expectations. Participants here said they used the system to give the researchers the results they wanted. Lead participants are the few users who carry a deployment, here one who organised and encouraged use by others. Interdependence of methods and results is the way the questions asked narrow the behaviour that can appear. A trial's findings are a product of method, investigators, participants and system design together.

Rogers and Marshall give the style a name and a position. Research in the wild creates and evaluates a technology in situ, often to augment a place rather than answer a stated user need. It carries a practical and ethical cost that a laboratory study does not (Rogers and Marshall, 2017). That is a position rather than a method.

The field left the laboratory, and the first check on that move was sceptical

The move outward was motivated by what the laboratory could not represent. Abowd and Mynatt surveyed a decade of ubiquitous computing around three themes. Those themes are natural interfaces, context-aware applications, and automated capture and access. They proposed everyday computing as attention to scaling interaction over time (Abowd and Mynatt, 2000). In their account, evaluation strategies form a spectrum from technology feasibility work to long-term use studies. The field study is one point on that scale.

The strongest early check on the assumption that the field is simply better compares laboratory and field evaluation of one context-aware mobile system. Six participants used it in each, none of them described as older or as needing a screen reader. Kjeldskov and colleagues report that the field evaluation added very little. Recreating central aspects of the use context in the laboratory produced the same list of usability problems (Kjeldskov et al., 2004). The variable that matters is not field against laboratory but which conditions of use a study preserves.

The instrument of capture is inside the finding, and a standard measure can answer a different question

If the method is part of the result, the medium of capture is too. Carter and Mankoff ran three diary studies in which participants recorded photographs, audio and location information. They asked how the medium shaped the material captured and how it jogged memory (Carter and Mankoff, 2005). Participants recorded different things in different media, so a diary study's contents are partly an artefact of its stationery.

Rogers makes the point from the other side. In the wild the locus of control shifts to the participant. Isolating specific effects becomes much more difficult, if not impossible, because many factors and interdependencies act at once (Rogers, 2011). That is the price of observing behaviour the laboratory would have prevented.

Some systems are designed for occasional attention, and standard instruments fit them badly. Hazlewood, Stolterman and Connelly report two in-situ evaluations of ambient displays. They name the issues that blocked assessment of the overall impact. They argue against a purely usability-focused laboratory evaluation and propose alternatives (Hazlewood, Stolterman and Connelly, 2011). When the intended effect is peripheral and slow, a measure of attention answers a question the system was never designed to pose.

The setting organises the behaviour, and time makes the setting a finding

Marshall and colleagues placed a shared planning application on a walk-up-and-use tabletop in a tourist information centre for five weeks. They recorded how groups approached it (Marshall et al., 2011). Groups did not behave as the laboratory literature predicted. Visitors often arrived separately and dispersed around the room. They used the tabletop in a staggered way, not as a party planning together. Group formation was being supplied, silently, by the laboratory arrangements.

Hinrichs and Carpendale studied visitor use of a large interactive multi-touch table exhibit at the Vancouver Aquarium. Gesture choice followed the interaction and social context, not a general preference. Gestures came in sequences, and earlier gestures influenced later ones. Gestures also organised social encounters around the exhibit (Hinrichs and Carpendale, 2011).

A six-year deployment shows how much further the point runs. PainPad ran on post-surgery orthopaedic wards in a large UK hospital, and more than 500 patients used it. They submitted more pain scores, at higher ratings, than the nurse-collected records it replaced held. Staff interviews showed how far integration depended on the ward's routines (Gooch et al., 2026). A study of a week would have measured compliance and stopped; six years made maintenance and clinical routine into findings about the setting.

Use in one setting is not portable to another

Henfridsson and Lindgren supply the structural reason generalisation is bounded rather than merely difficult. Their action research studied a system used in and around the car. It argues that ubiquity requires attention to multi-contextuality across the spatial, temporal and social dimensions of mobility. Information systems research has typically studied use in a single context (Henfridsson and Lindgren, 2005). If use is assembled differently in each context, a result from one context is a claim about that assembly.

O'Hara, Glancy and Robertshaw give that argument a worked instance. They studied collaborative play on large urban displays in three United Kingdom city locations. The physical character of each space initiated and coordinated the play, alongside the people passing through it and the game's own interactive properties. Audience and spectatorship were part of the scene (O'Hara et al., 2008). The same game and the same screen produced a different event in each location, and a hosted version differed from an unhosted one. What the case does not show is transfer: three sites establish that the arrangement decides the play, not what a fourth site would produce.

Contested question: what does a deployment claim, and what would settle it?

The literature disagrees about what a deployment may conclude, and the disagreement has named parties. Rogers treats the wild as the place where theory should be rebuilt. Theories developed in the laboratory have not fitted there. The programme is to import theories about real-world behaviour, to reconceptualise their use, and to build new wild theories from in-the-wild findings (Rogers, 2011). Dourish refuses the demand that a field study pays its way in design implications. He holds that implications for design may not be the best metric for such work (Dourish, 2006). Crabtree and colleagues hold the newer ethnographic approaches up to scrutiny. They argue that cultural interpretation and critiques of the design process displace the detailed empirical study of situated action. Design had been able to use that detailed study (Crabtree et al., 2009). Kjeldskov gives the most sceptical reading. A field evaluation added very little over a laboratory evaluation that restored central aspects of the context (Kjeldskov et al., 2004). Brown and colleagues record the exchange in their review. They cite both the claim that field trials add little user-study value and the contrary claim that they surface usability problems. Those problems never arise in the laboratory (Brown, Reeves and Sherwood, 2011).

The programme for wild theory is an argument from theory rather than data (Rogers, 2011), while Dourish (2006) and Crabtree and colleagues (2009) argue from a reading of the literature. The sceptical side rests on one field-versus-laboratory comparison, a single case with six participants in each condition (Kjeldskov et al., 2004). What would settle it is a series of such comparisons across systems and settings, with outcome measures fixed in advance. While the dispute is open, state the setting the result came from, name which of its conditions the design preserved, and leave the transfer to others. The strongest objection is that this turns every deployment into a case report, so nothing accumulates.

Where the area is heading

The direction of travel is towards deployments that report their logistics as method. Martinez-Maldonado and colleagues present lessons from a two-year in-the-wild multimodal learning analytics deployment. It ran with 399 students and 17 educators in nursing education. The lessons cover its technological and physical aspects, its data and interfaces, the design process, participation, ethics and privacy, and its sustainability (Martinez-Maldonado et al., 2024). That is a reporting agenda assembled from what decided the study. It converges with Rogers's proposal to test a claim beyond its setting (Rogers, 2011) and with six years of ward life in Gooch and colleagues (Gooch et al., 2026). Whether the field takes the setting and the method as the price of a claim, or keeps buying generalisation afterwards, is the argument this unit leaves open.

The sources this unit rests on

Each note states what the cited document did and found. Titles are in British spelling, and identifiers were checked against registered metadata on 2 November 2026.

• Brown, B., Reeves, S. and Sherwood, S. (2011). Into the wild: challenges and opportunities for field trial methods. doi:10.1145/1978942.1979185 A photo-sharing field trial examined as a trial of trials. It documents demand characteristics, lead participants and the interdependence of methods and results.
• Rogers, Y. and Marshall, P. (2017). Research in the Wild. doi:10.1007/978-3-031-02220-3 Defines research in the wild as creating and evaluating a technology in situ, often to augment a setting rather than meet a stated need. It covers approaches, methods, case studies and outcomes, and treats the practical and ethical costs as part of the position.
• Abowd, G. D. and Mynatt, E. D. (2000). Charting past, present, and future research in ubiquitous computing. doi:10.1145/344949.344988 A survey of a decade of ubiquitous computing around natural interfaces, context-aware applications and automated capture and access. It proposes everyday computing as attention to scaling interaction with respect to time, and places evaluation on a spectrum from feasibility work to long-term studies.
• Kjeldskov, J., Skov, M. B., Als, B. S. and Hoegh, R. T. (2004). Is it worth the hassle? Evaluating context-aware mobile systems in the field. doi:10.1007/978-3-540-28637-0_6 Compares laboratory and field evaluation of one context-aware mobile system, six participants in each condition. The field evaluation added very little: restoring central aspects of the use context in the laboratory gave the same usability problem list.
• Carter, S. and Mankoff, J. (2005). When participants do the capturing: the role of media in diary studies. doi:10.1145/1054972.1055098 Reports three diary studies in which participants captured photographs, audio and location information. The medium changed what was recorded and how memory was jogged, so the capture instrument belongs in the result.
• Rogers, Y. (2011). Interaction design gone wild: striving for wild theory. doi:10.1145/1978822.1978834 Argues the wild is where theory should be rebuilt: import theories about real-world behaviour, reconceptualise their use, and build new wild theories from in-the-wild findings. It notes that the locus of control shifts to participants, making specific effects hard to isolate.
• Marshall, P., Morris, R., Rogers, Y., Kreitmayer, S. and Davies, M. (2011). Rethinking 'multi-user': an in-the-wild study of a walk-up-and-use tabletop. doi:10.1145/1978942.1979392 A five-week in-the-wild study of a walk-up-and-use tabletop planning application in a tourist information centre. Groups did not approach or interact with it as laboratory findings on tabletop group work predicted.
• Hinrichs, U. and Carpendale, S. (2011). Gestures in the wild: multi-touch gesture sequences on interactive tabletop exhibits. doi:10.1145/1978942.1979391 A field study at the Vancouver Aquarium of visitor use of a large multi-touch table exhibit. Gesture choice followed the interaction and social context, gestures formed sequences, and gestures organised social encounters.
• Henfridsson, O. and Lindgren, R. (2005). Multi-contextuality in ubiquitous computing: investigating the car case through action research. doi:10.1016/j.infoandorg.2005.02.009 Action research on a ubiquitous computing system used across contexts including the car. It argues ubiquity requires attention to multi-contextuality across spatial, temporal and social dimensions of mobility, where information systems research studies a single context.
• Hazlewood, W. R., Stolterman, E. and Connelly, K. (2011). Issues in evaluating ambient displays in the wild: two case studies. doi:10.1145/1978942.1979071 Two in-situ evaluations of ambient displays, arguing for in-situ evaluation over laboratory usability work. It derives the issues that prevented assessment of their overall impact and proposes alternative measures.
• Dourish, P. (2006). Implications for design. doi:10.1145/1124772.1124855 Examines the criteria by which ethnographic work in HCI is judged. It argues that implications for design may not be the best metric for evaluating such work, which is the position debated in this unit's contested question.
• Crabtree, A., Rodden, T., Tolmie, P. and Button, G. (2009). Ethnography considered harmful. doi:10.1145/1518701.1518835 Reviews the status of ethnography in systems design and examines newer approaches that would replace detailed empirical study of situated action with cultural interpretation and critiques of the design process.
• Martinez-Maldonado, R. et al. (2024). Lessons learnt from a multimodal learning analytics deployment in-the-wild. doi:10.1145/3622784 Reports lessons from a two-year in-the-wild multimodal learning analytics deployment with 399 students and 17 educators in nursing education, grouped around its technical and physical aspects, data and interfaces, design process, participation, ethics and privacy.
• Gooch, D. et al. (2026). Deploying PainPad for patient pain logging in the hospital: a 6-year deployment. doi:10.1145/3815419 Reports a six-year deployment of a minimalist pain-logging device on post-surgery orthopaedic wards in a large UK hospital. More than 500 patients submitted more pain scores, at higher ratings, than nurse-collected records held, and staff interviews showed how far integration depended on ward routine.
• O'Hara, K., Glancy, M. and Robertshaw, S. (2008). Understanding collective play in an urban screen game. doi:10.1145/1460563.1460576 A study of collaborative game play on large urban displays in three United Kingdom city locations. Play was initiated and coordinated by the architecture of each space, the people in it and the game's interactive properties, and a hosted version differed from an unhosted one.