Arguments between roles are settled by evidence agreed in advance

A product manager, an engineer and a designer can leave the same meeting with three accounts of what was decided. None of them is unreasonable. Each answers to a different measure. The product manager answers to something that moves a number, the engineer to a system that stays correct, and the designer to the person using it. Carlile (2002) found this is how knowledge is arranged rather than how people are arranged.

Two responses are common, and neither survives contact with the evidence. What the literature supports is narrower: record the reasoning, and agree beforehand what result would change the decision. This seventh of ten units, after the sixth made a case to the client, settles the argument between colleagues.

The disagreement is a design feature of the organisation

Carlile (2002) studied, inside a new product development organisation, how knowledge is structured across the four primary functions of the work. He found knowledge localised, embedded and invested within a function. What an engineer knows is bound to that practice and its consequences. Crossing a function boundary is therefore not a matter of passing information across it. Carlile (2004) sharpened the account into three progressively complex boundaries, syntactic, semantic and pragmatic. These are matched to three processes of transfer, translation and transformation. At a syntactic boundary a shared vocabulary is enough. At a semantic one the parties must translate between meanings. At a pragmatic one they have to transform what they know, because each is answerable for the consequences inside a function of their own. The argument between design, product and engineering is the pragmatic case. It recurs because the organisation is built that way.

Wicked problems have no stopping rule

Rittel and Webber (1973) set out why problems of social policy resist the science built for tame ones. They cannot be definitively described, and their answers cannot be meaningfully correct or false. Buchanan (1992) carried the argument into design and listed the ten properties Rittel had identified, of which three carry the argument here.

(1) Wicked problems have no definitive formulation, but every formulation of a wicked problem corresponds to the formulation of a solution. (2) Wicked problems have no stopping rules. (3) Solutions to wicked problems cannot be true or false, only good or bad.

The consequence is uncomfortable for anyone expecting evidence to end the debate. No study is decisive here in the way a failing test is decisive in engineering, because no criterion of correctness stands outside the argument. Evidence still earns its place: it can show that one framing costs more than another, or that a step defeats most people who attempt it.

The artefact travels, but the arrangement decides whether it matters

Star and Griesemer (1989) studied how amateurs, professionals and administrators cooperated across divergent viewpoints at the Museum of Vertebrate Zoology. They named the two activities that made cooperation possible: the standardisation of methods, and boundary objects. Boundary objects are adaptable to different viewpoints and robust enough to keep their identity across them. The shared artefact is what a designer reaches for: a prototype, a journey map, a specification. Star (2010) returned to the concept to warn that much later work kept only the interpretive flexibility and dropped the structure of work arrangements. Boundary objects therefore came to be treated as useful at any scale. A journey map displayed in a review changes nothing unless the work around it obliges the engineer and the product manager to use it. The artefact travels. The arrangement decides whether the travelling matters.

The durable move is to write the reasoning down

MacLean, Young, Bellotti and Moran (1991) set out design space analysis and its notation of Questions, Options and Criteria. The notation represents the space around an artefact as the issues to be settled, the options considered and the criteria for comparing them. It is a coproduct built alongside the artefact rather than a record of the process. The authors state their work emphasises the representation over the process of creating it, so the payback is a goal rather than a result. Conklin and Begeman (1988) built gIBIS, a hypertext tool capturing early design deliberation as typed networks of issues, positions and arguments. They reported that early experiments left the method incomplete, while tool and method suited each other. The record can be built. In the founding papers, the case for building it was a claim about value rather than a demonstrated return.

Agree the criterion before the result arrives

Rousseau (2006) describes the research-practice gap, in which managers fail to base practice on the best available evidence, and uses evidence-based medicine as the exemplar for closing it. Briner, Denyer and Rousseau (2009) then clean up the concept, arguing that evidence-based management is a family of approaches that support decisions rather than one rigid method, that practitioners do it rather than scholars, and that systematic reviews are its cornerstone. Weber, Wyverkens and Leuridan (2024) tighten the definition from methodology: what counts as evidence becomes checkable only when the method which produced the information is part of it. Tranfield, Denyer and Smart (2003) supply the review method, arguing that the narrative review frequently lacks thoroughness and rigour. The design-relevant form is procedural and cheap. Before the study runs, the team writes the criterion down: this result changes the decision, that result leaves it alone.

Why good evidence does not always move the room

Pfeffer and Sutton (2006) ask why management practices spread slowly when the evidence behind some of them is strong. They answer that practices depend on tacit knowledge and implementation skill, on knowing how as well as what, and resist copying through precedent and ideology. A finding carries a practice with it. That practice must be learned before the finding can be used. Petty and Cacioppo (1986) supply the other half. They open on whether a message persuades through its substance or through the appearance and demeanour of the speaker. They develop the elaboration likelihood model, whose central route turns on engagement with the argument itself. A technical review will scrutinise the method. A steering group may decide on who presented it. Writing the criterion down moves the second room towards the first.

Klein's (2017) fireground study shows what recognition looks like. His team interviewed 26 officers from seven urban fire departments and coded 156 decision points from 32 incidents. In 127 of them the commander considered one option only. The situation matched a prototype built from experience, and the prototype supplied the goals, the cues and the action. One commander took a crew into what looked like a kitchen fire, found it unusually hot and quiet, ordered everyone out, and watched the floor collapse a minute later. The fire was in a basement. The accounts are retrospective, so the study records what commanders report doing.

Contested question: is the disagreement a knowledge gap or an interest difference?

The literature divides on what this disagreement actually is, and the division decides the remedy. The first position treats it as a knowledge gap. Rousseau (2006) frames it as a research-practice gap that better translation can narrow. Briner, Denyer and Rousseau (2009) place systematic review at the centre, and Tranfield, Denyer and Smart (2003) provide the method.

The second treats the disagreement as an interest difference that evidence cannot dissolve. Carlile (2004) argues that at a pragmatic boundary each party's knowledge is invested in its own function, so transformation is needed rather than translation. Pfeffer and Sutton (2006) add that precedent and ideology resist evidence even where it is good. Rittel and Webber (1973) support the second reading from first principles. A problem with no definitive formulation and no stopping rule cannot be resolved by showing one answer true. Buchanan (1992) draws the design version. Solutions to wicked problems cannot be true or false, only good or bad.

Star (2010) cuts across both. The shared artefact the first position relies on was one component of a larger account of work arrangements.

The knowledge-gap case draws on the systematic-review literature, which is methodological rather than empirical about disputes. The interest-difference case draws on Carlile's ethnography and on Rittel and Webber's argument from first principles; neither measures a resolution. What would settle the disagreement is a record of decisions. A study that fixed each criterion before the evidence arrived and tracked whether the decision changed would show how much of a real dispute a criterion carries, and the criterion should name whose case the evidence covers, since the studies run on students leave an older user untested.

While the dispute is open, a designer should fix the criterion before the evidence arrives and keep the decision log. The strongest objection is that this assumes the parties agree to be bound: where a boundary is pragmatic, the party who loses can refuse the criterion afterwards.

Where the argument is heading

The direction of travel is from argument to instrument. Briner, Denyer and Rousseau (2009) place systematic review at the centre of evidence-based management, while insisting the practice belongs to practitioners, which moves the machinery of evidence into the organisation rather than the literature. Carlile (2004) marks the boundary at which a better review stops being enough. Star (2010) is the most cited caution here and the most misread. Her complaint is that the work arrangement was dropped. Weber, Wyverkens and Leuridan (2024) push the same way on the definition itself. The practitioner the next few years will reward sets the criterion before the evidence arrives and keeps a decision record that survives the people who wrote it.

The sources this unit rests on

• Briner, R. B., Denyer, D. and Rousseau, D. M. (2009). Evidence-Based Management: Concept Cleanup Time? Academy of Management Perspectives. doi:10.5465/amp.2009.45590138 Clarifies common misconceptions, arguing evidence-based management is a family of approaches supporting decisions rather than one rigid method, and is done by practitioners rather than scholars. Places systematic review at its centre.
• Buchanan, R. (1992). Wicked Problems in Design Thinking. Design Issues. doi:10.2307/1511637 Carries Rittel's ten properties of wicked problems into design, and argues that design thinking has become a new liberal art of technological culture.
• Carlile, P. R. (2002). A Pragmatic View of Knowledge and Boundaries: Boundary Objects in New Product Development. doi:10.1287/orsc.13.4.442.2953 An ethnographic study of how knowledge is structured across the four primary functions of a product development organisation. Finds knowledge localised, embedded and invested within a function.
• Carlile, P. R. (2004). Transferring, Translating, and Transforming: An Integrative Framework for Managing Knowledge Across Boundaries. doi:10.1287/orsc.1040.0094 Develops a framework of three progressively complex boundaries, syntactic, semantic and pragmatic, and three matching processes of transfer, translation and transformation. Argues the pragmatic case needs transformation rather than transmission.
• Conklin, J. and Begeman, M. L. (1988). gIBIS: a hypertext tool for exploratory policy discussion. CSCW. doi:10.1145/62266.62278 Describes gIBIS, a hypertext system implementing the Issue-Based Information System method, capturing early design deliberation as typed networks of issues, positions and arguments. Reports the method remained incomplete.
• Klein, G. A. (2017). Sources of Power. The MIT Press. doi:10.7551/mitpress/11307.001.0001 Reports the fireground study of 26 officers, 32 incidents and 156 decision points, 127 of which matched a recognised prototype. Argues expert decisions rest on situation assessment and on evaluating one course of action.
• MacLean, A., Young, R., Bellotti, V. and Moran, T. (1991). Questions, Options, and Criteria: Elements of Design Space Analysis. Human-Computer Interaction. doi:10.1207/s15327051hci0603&4_2 (reprint: https://doi.org/10.1201/9781003064053-4) Presents the QOC notation for the design space around an artefact, and argues the analysis is a coproduct built alongside it. States the payback as a goal rather than an achieved result.
• Petty, R. E., Cacioppo, J. T. and Goldman, R. (1981). Personal involvement as a determinant of argument-based persuasion. Journal of Personality and Social Psychology. doi:10.1037/0022-3514.41.5.847 Exposed undergraduates to a counterattitudinal message varying in argument quality, source expertise and personal relevance. Under high relevance attitudes followed argument quality, and under low relevance they followed the source.
• Petty, R. E. and Cacioppo, J. T. (1986). The Elaboration Likelihood Model of Persuasion. In Communication and Persuasion. Springer. doi:10.1007/978-1-4612-4964-1_1 Opens on whether a message persuades through its substance or through the appearance and demeanour of the speaker, and sets out the elaboration likelihood model, whose central route turns on engagement with the argument.
• Pfeffer, J. and Sutton, R. I. (2006). Management Half-Truths and Nonsense: How to Practice Evidence-Based Management. California Management Review. doi:10.2307/41166351 Asks why management practices spread slowly when evidence for some of them is strong. Finds they depend on tacit knowledge and implementation skill and resist copying through precedent and ideology.
• Rittel, H. W. J. and Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences. doi:10.1007/bf01405730 Argues that problems of social policy are wicked rather than tame, cannot be definitively described, and have no definitive or objective answers.
• Rousseau, D. M. (2006). Is there Such a thing as Evidence-Based Management? Academy of Management Review. doi:10.5465/amr.2006.20208679 Examines the promise organisation research holds for management practice and how it falls short, using evidence-based medicine as an exemplar for closing the research-practice gap.
• Star, S. L. and Griesemer, J. R. (1989). Institutional Ecology, Translations and Boundary Objects: Amateurs and Professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39. Social Studies of Science. doi:10.1177/030631289019003001 Studies how amateurs, professionals and administrators cooperated across divergent viewpoints, naming standardisation of methods and boundary objects as the two activities that translated between them.
• Star, S. L. (2010). This is Not a Boundary Object: Reflections on the Origin of a Concept. Science, Technology and Human Values. doi:10.1177/0162243910377624 Revisits the concept and argues that it has three components, of which later work kept only interpretive flexibility. Warns that the structure of work arrangements was the part that made them work.
• Tranfield, D., Denyer, D. and Smart, P. (2003). Towards a Methodology for Developing Evidence-Informed Management Knowledge by Means of Systematic Review. British Journal of Management. doi:10.1111/1467-8551.00375 Evaluates whether the medical systematic review process applies to management knowledge. Finds narrative reviews frequently lack thoroughness and rigour.
• Weber, E., Wyverkens, A. and Leuridan, B. (2024). Rethinking Evidence-Based Management. Philosophy of Management. doi:10.1007/s40926-023-00236-5 Argues that it defines evidence too loosely, and that what counts becomes checkable only when the method which produced it is part of the definition.