Year 3 / Module 24 Industry group project / Unit 1 / A brief is an opening, not a specification: scope, change control and the honest estimate

A brief is an opening, not a specification: scope, change control and the honest estimate

HCI3108 · Unit 1 of 10

A written scope agreement earns its keep through its exclusions, assumptions and change route rather than through its list of deliverables, and the live contest is whether a frozen baseline or an evolving scope produces the more usable product.

A brief is an opening, not a specification: scope, change control and the honest estimate

A brief arrives short and confident. A client writes that they need an application so their volunteers can log their hours, and expects that sentence to describe a system. It does not, because the sentence records a decision about what the client wants now. This is the first of ten units, and it sets the terms the other nine work inside. The unit argues two things. A written scope agreement earns its keep through three parts that look like housekeeping and are not, and the deliverables list, negotiated hardest, decays fastest. The second claim is a live contest between a frozen baseline and an evolving scope. The literature does not settle it, because the two sides measure different outcomes.

A brief records a decision, and reading it as a specification starts disputes that argument cannot settle

Rittel and Webber (1973) give the reason the brief cannot be completed in advance. Social policy problems are wicked, they argue. They cannot be definitively described, and they have no stopping rule and no objectively correct solution. Every attempt to solve one changes it. What counts as a shift, who approves it, and whether the record outlives the project are not facts awaiting discovery. They are decisions the team makes, and each one changes what the client says they wanted.

Simon (1973) supplies the counterweight. The boundary between well-structured and ill-structured problems is vague and fluid, and a problem looks ill structured to a solver with limited access to relevant information. A scope agreement is a chosen representation to build against, not a transcription of the truth, and representation is what turns an ill-structured brief into something buildable. Nuseibeh and Easterbrook (2000) wrote the field's roadmap. They named better modelling of the problem domain, rather than of software behaviour, as an open problem, and treated the gap to formal specification as structural.

Pohl (1994) explains why scope arguments fail to resolve. Requirements work, he argues, has three dimensions: the specification, the representation and the agreement. Many difficulties come from treating a problem in one dimension as though it belonged in another. A client's list of features is an agreement artefact, recording what two parties accepted; a specification is a statement about the system. Read the deliverables list as a specification and every later scope conversation becomes a dispute about the truth of a sentence that was only a record of agreement.

The agreement earns its keep through the parts that look like housekeeping

The deliverables list is written first and used least. The other three parts are used in week twenty, and a team omits them because they read as weakness. Access is the clearest test: a requirement to run on an old handset, over a slow connection and in plain language is easy to defer, because the users it serves are seldom in the room when scope is cut.

  • Exclusions. What the product will not do. Each exclusion closes a question that would otherwise be reopened late. The system above will not schedule volunteers, calculate pay or send reminders, and it will not require a current handset or a fast connection.
  • Assumptions. The conditions the team does not control. Volunteers reach the service on whatever phone they already own, some of them older people, and some read little English; the client supplies shift categories by week four; someone can approve an account. An assumption is a dependency the team can point to if it fails.
  • The change route. Who prices a new request, by what method, and against what reserve. It decides whether a change arrives as a decision or as a surprise.

The change route is also where the supporting evidence is thinnest. Achimugu and colleagues (2014) reviewed 73 primary studies of requirements prioritisation, the practice that decides what is dropped when something is added. They found the techniques poorly scalable. No settled method updates ranks as requirements evolve, and their applicability in real settings goes unreported. A change route is therefore priced with methods of undocumented behaviour, which argues for keeping it explicit and auditable.

Mendez Fernandez and colleagues (2016) report what a team should expect. Their survey of 228 companies in 10 countries analysed 21 problems drawn from the literature for relevance and criticality in context, and traced their causes and effects. The recurring requirements problems are the field's normal weather, and the team's job is to record which of them it is carrying.

The estimate is a judgement, and the literature expects it to be optimistic

An estimate is where the agreement meets arithmetic. Jorgensen (2004) reviewed studies of expert estimation of software effort. He found expert estimation the most frequently applied strategy, with no substantial evidence in favour of formal estimation models. His review validates twelve practices, among them asking estimators to justify and criticise their estimates. A search of four leading journals returned exactly 100 papers on effort or size estimation, of which only 17 discussed expert estimation.

Flyvbjerg (2006) adds that the judgement carries a known bias. He documents systematic inaccuracy in forecasts of cost and demand. He explains it through optimism bias and strategic misrepresentation, and proposes reference class forecasting, which measures a forecast against the performance of comparable projects. Boehm (2002) supplies the counterweight to complacency about change. Overresponding to change, he argues, caused many disasters, among them the $3 billion overrun of the US Federal Aviation Administration's Advanced Automation System.

The method has a worked case. Flyvbjerg (2006) reports its first practical use, after the Treasury told appraisers in 2003 that their estimates were systematically optimistic. The team assembled a reference class of 172 completed road projects. It found that 80 per cent had overrun by no more than 32 per cent. It turned that distribution into an uplift. Its own example is a motorway business case of 100 million pounds that becomes 132 million at the risk level the department accepts. The mechanism is the point: the forecast stops being a defence of one team's plan and becomes a position in a distribution of outcomes other teams produced. What the case does not show is that the uplift was right for any single project.

Contested question: a frozen baseline or an evolving scope?

The evolving-scope position is stated most directly in the Agile Manifesto (Beck et al., 2001).

Working software over comprehensive documentation. Responding to change over following a plan (Beck et al., 2001).

Larman and Basili (2003) trace iterative and incremental development to the mid-1950s. Each succeeding decade, they show, produced large projects run that way, always avoiding a single-pass, document-driven approach. On this account the frozen baseline is the recent exception, not the norm.

The baseline position is the classical plan-driven one, and its evidence comes from forecasting rather than from delivery. Flyvbjerg's (2006) remedy requires a defined object to forecast, so an estimate assembled while scope moves has no fixed subject and cannot be corrected. Boehm (2002) and Boehm and Turner (2003) occupy the middle. Agile and plan-driven methods each have a home ground, they argue, where each performs much better than the other. Boehm and Turner propose a risk-based mix of the two, in proportion to a project's risk profile. This unit's scope discipline follows that position.

What would settle it is a comparison nobody has run. The evolving-scope case rests on practitioner reports and successful projects. It cares about the fit of what was delivered. The baseline case rests on studies of forecast accuracy. It cares about whether the promise made at the start was kept. Neither measures the other's outcome, so the two literatures pass each other. Settling it would need a study that holds project and client constant and varies only the scope regime. It would measure the usability of what is delivered and the accuracy of what was promised. No source here runs it.

The middle position is a risk heuristic with no trial behind it. While the dispute is open, write the agreement so it survives either regime: the change route prices a request before acceptance, and each checkpoint restates the baseline in force. The strongest objection is that this buys a record, not an outcome, since the same problems recur in companies that already write agreements (Mendez Fernandez et al., 2016).

Where the argument is heading

Recent work moves the agreement's centre of gravity from the signature to the maintenance of the record. Vogelsang and Borg (2019) interviewed four data scientists about requirements engineering for machine learning. They found that the shift from coding to training demands changes in the discipline. A team must understand machine learning performance measures to state good functional requirements, and it must handle new quality requirements such as explainability. The study is exploratory, with four participants, so it gives the shape of the problem rather than its size. When the deliverable is a model, acceptance criteria are negotiated against measured performance during development rather than fixed in advance. That strengthens the evolving-scope case and makes the exclusions and assumptions more important.

Frattini and colleagues (2023) name what an unresolved brief leaves behind. From two expert interviews and a survey of 69 practitioners they built an initial theory of requirements engineering debt with 23 falsifiable propositions. Measuring and tracking such debt, they found, is immature in practice. An exclusion left unwritten, an assumption never named and a change accepted without a price are debts of this kind. The team pays them later, in rework or in a product answering a question the client has stopped asking. The direction is that the scope agreement becomes a maintained record of what is known and assumed, revisited at every change.

The sources this unit rests on

  • Rittel, H. W. J. and Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences. doi:10.1007/BF01405730 Social policy problems are wicked: indefinable, without a stopping rule or an objectively correct solution, and changed by every attempt to solve them. It is the originating case for treating a client brief as underdetermined.
  • Simon, H. A. (1973). The structure of ill structured problems. Artificial Intelligence. doi:10.1016/0004-3702(73)90011-8 The boundary between well-structured and ill-structured problems is vague and fluid, and a problem looks ill structured to a solver with limited information. It makes representation the route by which such a brief becomes buildable.
  • Pohl, K. (1994). The three dimensions of requirements engineering: A framework and its applications. Information Systems. doi:10.1016/0306-4379(94)90044-2 Requirements work has three dimensions, the specification, the representation and the agreement, and difficulties arise from confusing them. It explains why a deliverable list read as a specification produces disputes that argument cannot settle.
  • Nuseibeh, B. and Easterbrook, S. (2000). Requirements engineering: a roadmap. Proceedings of the Conference on the Future of Software Engineering (ICSE). doi:10.1145/336512.336523 Names the field's open problems, including better modelling of the problem domain rather than of software behaviour, and treats the brief-to-specification gap as structural. It is the discipline's own admission that better tools do not close it.
  • Achimugu, P., Selamat, A., Ibrahim, R. and Mahrin, M. N. (2014). A systematic literature review of software requirements prioritization research. Information and Software Technology. doi:10.1016/j.infsof.2014.02.001 Reviews 73 primary studies of requirements prioritisation and finds the techniques poorly scalable, without a settled method for updating ranks as requirements evolve, and with their real-setting applicability unreported. It shows the change route is priced with methods of undocumented behaviour.
  • Jorgensen, M. (2004). A review of studies on expert estimation of software development effort. Journal of Systems and Software. doi:10.1016/s0164-1212(02)00156-5 Finds expert estimation the dominant strategy, with no substantial evidence favouring estimation models, and validates twelve practices for justifying and combining estimates; a search of four leading journals returned exactly 100 papers on effort or size estimation, of which only 17 discussed expert estimation. It grounds the claim that an estimate is a judgement, not a measurement.
  • Flyvbjerg, B. (2006). From Nobel Prize to Project Management: Getting Risks Right. Project Management Journal. doi:10.1177/875697280603700302 Documents systematic inaccuracy in forecasts of cost and demand, attributes it to optimism bias and strategic misrepresentation, and proposes reference class forecasting. It is the evidence that early estimates are optimistic as a rule.
  • Beck, K. et al. (2001). Manifesto for Agile Software Development. agilemanifesto.org. https://agilemanifesto.org/ States four value trade-offs, placing working software above documentation and response to change above following a plan. It is the primary statement that a frozen early specification is the wrong default.
  • Larman, C. and Basili, V. R. (2003). Iterative and incremental developments: a brief history. Computer. doi:10.1109/MC.2003.1204375 Traces iterative and incremental development to the mid-1950s and shows that each succeeding decade produced large projects run that way, all avoiding a single-pass, document-driven process. It undercuts the claim that iterative delivery is new.
  • Boehm, B. (2002). Get ready for agile methods, with care. Computer. doi:10.1109/2.976920 Argues that agile and plan-driven methods each have a home ground, and that combining them is feasible, citing overresponding to change as the source of disasters including a $3 billion overrun. It restrains the evolving-scope position.
  • Boehm, B. and Turner, R. (2003). Observations on balancing discipline and agility. Proceedings of the Agile Development Conference, 2003. ADC 2003. doi:10.1109/ADC.2003.1231450 Offers six observations and a risk-based approach mixing agile and disciplined practice in proportion to a project's risk profile. It is the disciplined statement of the middle position the unit follows.
  • Mendez Fernandez, D. et al. (2016). Naming the pain in requirements engineering: contemporary problems, causes, and effects in practice. Empirical Software Engineering. doi:10.1007/s10664-016-9451-7 Surveys 228 companies in 10 countries and analyses 21 literature-derived problems for relevance and criticality in context. It states empirically what goes wrong, where the literature had been dominated by conventional wisdom.
  • Vogelsang, A. and Borg, M. (2019). Requirements Engineering for Machine Learning: Perspectives from Data Scientists. 2019 IEEE 27th International Requirements Engineering Conference Workshops (REW). doi:10.1109/REW.2019.00050 Interviews four data scientists and argues that moving from coding to training demands changes in requirements engineering, including understanding performance measures. It is the recent shift in what a brief becomes when the deliverable is a model.
  • Frattini, J. et al. (2023). An initial theory to understand and manage requirements engineering debt in practice. Information and Software Technology. doi:10.1016/j.infsof.2023.107201 Builds a theory of requirements engineering debt from two expert interviews and a survey of 69 practitioners, giving 23 falsifiable propositions, and finds its measurement immature. It recasts an unresolved brief as a debt with interest.

Check your understanding & apply it · 4 questions, 1 essay

1A team's scope agreement lists twelve deliverables and a date and nothing else. In week nine the client asks for a reminder feature, and the team replies that the signed list settles what was agreed. Using the unit's argument, which diagnosis of the team's position is correct?
2A team writes in its report: the brief is a description of the system the client needs, so a complete description of it makes the design follow. Using Rittel and Webber (1973) and Simon (1973) as the unit sets them against each other, which diagnosis of that sentence is correct?
3A team writes in its plan: we will hold the client to the original 240-hour estimate, because the estimate measures the work and the target is to come within ten per cent of it. Using the unit's reading of Jorgensen (2004) and Flyvbjerg (2006), which diagnosis is correct?
4A team must decide how to write its agreement while the dispute between a frozen baseline and an evolving scope stays open. One member proposes writing to whichever regime has the stronger evidence, another to whichever regime the client prefers. Which course does the unit support, and why?
5Essay A student team has signed a one-page scope agreement with its client. It lists eight deliverables, a deadline and a price in hours. It contains no exclusions, no assumptions and no change route. At week six the client asks for a feature the team had not considered, and at week ten the client says the reminder emails are not what they meant. Assess the team's position using the unit's argument: say what the missing parts of the agreement would have done, what the evidence says about the team's original estimate of the work, what the literature does and does not settle about holding a frozen baseline, and what the team should do now. Name the sources you rely on at each step.