Requirements and user stories: from wishes to decisions a test can check

The third unit turns from the organisation to the decisions it has to make. A backlog is a list of sentences, and most of them are wishes. "As a claimant I want to check my eligibility so that I do not lose a half-completed form" names an actor and a reason, but it does not say when the work is done. A requirement earns its place when it names a decision and a condition under which somebody can check that decision. The value of a story format therefore lies in the acceptance criterion it forces somebody to write.

If the criterion is the load-bearing part, the form of the sentence is negotiable. The use case tradition shows what the template leaves out. If the criterion has no outcome evidence, the argument for insisting on one must rest on what it forces a team to decide. The literature supports the first and not the second.

The claim: a decision and a condition, not a sentence and a template

Wake (2003) sets out the INVEST criteria. Their last letter is that a good story is testable, which he defends as an implicit promise. Writing a card promises that the author understands the requirement well enough to write a test for it. A customer who cannot say how something would be tested has revealed an unclear story. Cohn (2004) splits a story into three parts: a written description for planning, a conversation that supplies the detail, and tests that determine when the story is complete. The Product Owner specifies the acceptance criteria. The decision sits in the confirmation, not on the card.

Lucassen and others (2015) turn the promise into an assessable standard. Their Quality User Story framework gives fourteen criteria for a well-formed story, organised as syntactic, semantic and pragmatic. "Well-formed" requires a role and a means, and "atomic" requires exactly one feature. Only "conflict-free", "unique", "uniform", "independent" and "complete" apply to a set. A backlog can be well formed story by story and still describe a product whose parts do not fit.

The format is valued; the product is not measured

The best evidence about story quality is about belief. Lucassen and others (2016) surveyed 182 practitioners and followed up with 21 interviews. They reported twelve findings. Practitioners agreed that stories, a template and guidelines such as INVEST improve their productivity and the quality of their deliverables. Of the adopters, 94 per cent used stories with Scrum. The paper states that no scientific evidence existed on the effectiveness of user stories, and it describes its goal as exploring perception. Users of the INVEST guidelines were significantly more positive about the impact on productivity and deliverable quality. Technical stakeholders were less positive than non-technical ones.

What the story gives up: goal level and the record of failure

Cockburn (2001) writes use cases at three named goal levels: summary, user goal and subfunction. Each carries a main success scenario and an exhaustive list of extension conditions for the failures and alternatives the system must handle. His rule is that a condition should say what was detected. A story asks for no goal level, and it requires no failure paths. A set of stories can therefore be complete and silent about what happens when the input is wrong, or when two people hold the same slot. An acceptance criterion is the right place for an exception condition, but the template does not build that recovery in. The claimant on a metered phone or a screen reader is one such path.

Where requirements come from: elicitation is discovery, not collection

Zowghi and Coulin (2005) define elicitation as seeking, uncovering, acquiring and elaborating requirements. Ramesh, Cao and Baskerville (2010) analysed data from sixteen organisations. They found that agile requirements practices left two risks intractable: a customer who cannot decide, and the neglect of non-functional requirements such as security and scalability. Mendez Fernandez and others (2017) analysed data from 228 companies in ten countries. They assessed 21 problems from the literature for relevance and criticality, and they presented the work as a step to ground the field on empirical observation rather than conventional wisdom.

Does the requirements work pay off? One case, one review, one gap

Damian and Chisan (2006) collected evidence over a 30-month case study of a large project undergoing requirements process improvement. An effective requirements process early in the project had positive outcomes throughout the lifecycle. One case explains a mechanism, not a magnitude. Inayat and others (2015) reviewed literature published between 2002 and June 2013. They identified 21 papers and extracted seventeen practices. Those practices overcame five problems and posed eight challenges. The review asks for more empirical results, and it names non-functional requirements and self-organising teams.

Quality is partly machine-checkable, and the residue needs a person

Ferrari and others (2018) applied detection patterns, implemented in the GATE tool, to 1866 railway requirements already annotated for defects. Two domain experts inspected the output for false positives. They also ran an ambiguity tool, SREE, over the same set. The approaches were complementary, and they disagreed in company-specific ways. Luitel, Hassani and Sabetzadeh (2024) used BERT's masked language model to predict terminology withheld from requirements, evaluated on forty specifications from the PURE dataset. The predictions highlighted missing terminology, and a learned noise filter improved them. A prediction of missing terminology is a question for a person who knows whether the omission matters. This is detection on a dataset, not evidence that teams receiving such alerts write better requirements.

The company behind the first result was a single railway signalling manufacturer, and an engineer of the firm wrote the detection patterns for its own defect classes. The analysis therefore describes one company's documents and vocabulary (Ferrari and others, 2018). The forty specifications come from PURE, a public corpus of requirements documents. The objects are fixed, and the test is whether the model recovers withheld terminology, not whether a team judged the requirement complete (Luitel and others, 2024).

Contested question: is a testable condition the right gate, or does it select for what is easy to test?

Different literatures hold three positions. The first treats testability as constitutive. Wake (2003) calls an untestable story a symptom of unclarity, and Lucassen and others (2015) make parts of the standard assessable. The second holds that the gate excludes the requirements that hurt most. Ramesh and others (2010) found non-functional requirements such as security and scalability among the risks agile practice could not address. Inayat and others (2015) name non-functional requirements as an area still needing evidence. Mendez Fernandez and others (2017) find requirements problems persisting in practice. A criterion that a story can satisfy within one iteration selects for what is testable within one iteration. The properties that decide whether a system is safe or fast are the ones nobody writes a sentence for.

The third position moves the disagreement onto the artefact. Lucassen and others (2016) report that the simple structure of stories builds a common understanding of the requirement. The same tradition treats the card as a token that promises a conversation (Wake, 2003). If the value lies in the conversation, the criterion is a device for making that conversation end in a decision. One study would settle it. That study would treat the criterion as the variable: one product, a criterion required for every story including its non-functional and failure-path content, and the format held constant. It would read outcomes from artefacts such as post-release defects and rework. Ferrari and others (2018) come closest, in one company. Nobody has run the design.

What to do while the dispute is open

The evidence is uneven. The case for the gate rests on a practitioner standard and on a survey of 182 practitioners reporting their own impressions, which the authors describe as perception rather than measurement (Wake, 2003; Lucassen and others, 2016). The case against rests on one case study, one review that asks for more data, and a survey of 228 companies; each shows non-functional requirements neglected, and none shows the criterion causing the neglect (Damian and Chisan, 2006; Inayat and others, 2015; Mendez Fernandez and others, 2017). The strongest objection to this unit's thesis is that a criterion can be met by a sentence that passes its own test and still describes the wrong product. While the dispute is open, a designer should write the criterion for what the story is silent about, and attach the failure path to the same card as the happy path.

Where the argument is heading

Dalpiaz and Niu (2020) place the field. Artificial intelligence has been applied to requirements since knowledge representation in the 1980s, then through natural language processing and machine learning. If a model can draft the criterion, the obligation to decide moves to the reviewer. The named work points to that division of labour (Ferrari and others, 2018; Luitel and others, 2024). That direction sits on top of a practitioner standard still being assembled in the 2000s (Wake, 2003; Lucassen and others, 2015). The sentence is cheap, the criterion is the work, and the outcome evidence has not been collected.

The sources this unit rests on

The unit plan records the IEEE Software article as Dalpiaz and Ferrari (2020); the published article is by Dalpiaz and Niu and is cited here under its published authorship. One publisher title appears below in British spelling where the original uses an American form.

• Wake, W. (2003). INVEST in Good Stories, and SMART Tasks. https://xp123.com/articles/invest-in-good-stories-and-smart-tasks/ Wake sets out the INVEST criteria, including testability, and treats an untestable story as a symptom of an unclear or worthless requirement. He tells teams to operationalise tests for non-functional properties such as performance.
• Cohn, M. (2004). User Stories Applied: For Agile Software Development. Addison-Wesley, Boston, MA. https://archive.org/details/userstoriesappli0000cohn Cohn splits a story into a written description for planning, a conversation that supplies the detail, and tests that decide when it is complete. The Product Owner specifies the acceptance criteria.
• Cockburn, A. (2001). Writing Effective Use Cases. Addison-Wesley, Boston, MA. https://archive.org/details/writingeffective0000cock Cockburn writes use cases at three named goal levels, with a main success scenario and an exhaustive list of extension conditions for failures and alternatives. It shows which fields the story template does not require.
• Lucassen, G., Dalpiaz, F., van der Werf, J. M. E. M. and Brinkkemper, S. (2015). Forging High-Quality User Stories: Towards a Discipline for Agile Requirements. In 2015 IEEE 23rd International Requirements Engineering Conference (RE 2015). IEEE, 126-135. doi:10.1109/RE.2015.7320415 The paper proposes the Quality User Story framework, fourteen criteria for a well-formed story across syntactic, semantic and pragmatic categories. Most criteria are properties of one story and only a minority of a set.
• Lucassen, G., Dalpiaz, F., van der Werf, J. M. E. M. and Brinkkemper, S. (2016). The Use and Effectiveness of User Stories in Practice. In Requirements Engineering: Foundation for Software Quality (REFSQ 2016), Lecture Notes in Computer Science 9619. Springer, 205-222. doi:10.1007/978-3-319-30282-9_14 A survey of 182 practitioners with 21 follow-up interviews produced twelve findings, among them that 94 per cent of adopters pair stories with Scrum. The authors state that no scientific evidence existed on the effectiveness of user stories, so the study measures perception.
• Zowghi, D. and Coulin, C. (2005). Requirements Elicitation: A Survey of Techniques, Approaches, and Tools. In Aurum, A. and Wohlin, C. (eds) Engineering and Managing Software Requirements. Springer, 19-46. doi:10.1007/3-540-28244-0_2 The chapter surveys elicitation techniques and argues that requirements are elicited rather than merely captured or collected. It makes a story the late product of a discovery rather than its record.
• Ramesh, B., Cao, L. and Baskerville, R. (2010). Agile requirements engineering practices and challenges: an empirical study. Information Systems Journal 20(5), 449-480. doi:10.1111/j.1365-2575.2007.00259.x Data from sixteen organisations yielded six agile requirements practices and seven challenges they create. Two risks were intractable: a customer who cannot decide, and the neglect of non-functional requirements such as security and scalability.
• Mendez Fernandez, D., Wagner, S., Kalinowski, M., Felderer, M., Mafra, P., Vetro, A. and others (2017). Naming the pain in requirements engineering. Empirical Software Engineering 22(4), 2298-2338. doi:10.1007/s10664-016-9451-7 The authors analyse data from 228 companies in ten countries, assessing 21 problems for relevance and criticality. They present the work as a step to ground the field on empirical observation rather than conventional wisdom.
• Damian, D. and Chisan, J. (2006). An Empirical Study of the Complex Relationships between Requirements Engineering Processes and Other Processes that Lead to Payoffs in Productivity, Quality, and Risk Management. IEEE Transactions on Software Engineering 32(7), 433-453. doi:10.1109/TSE.2006.61 A 30-month case study of a large project undergoing requirements process improvement found that an effective early requirements process improved later processes and product quality. One longitudinal case explains a mechanism rather than estimating an effect.
• Inayat, I., Salim, S. S., Marczak, S., Daneva, M. and Shamshirband, S. (2015). A systematic literature review on agile requirements engineering practices and challenges. Computers in Human Behaviour 51, 915-929. doi:10.1016/j.chb.2014.10.046 The review covered literature from 2002 to June 2013 and identified 21 papers, extracting seventeen practices, five problems they overcame and eight challenges they posed. It asks for more empirical results, naming non-functional requirements and self-organising teams.
• Ferrari, A., Gori, G., Rosadini, B., Trotta, I., Bacherini, S., Fantechi, A. and Gnesi, S. (2018). Detecting requirements defects with NLP patterns: an industrial experience in the railway domain. Empirical Software Engineering 23(6), 3684-3733. doi:10.1007/s10664-018-9596-7 Patterns in the GATE tool were applied to 1866 requirements of a railway signalling manufacturer, already annotated for defects, with two experts checking false positives and an ambiguity tool, SREE, run over the same set. The approaches were complementary and disagreed in company-specific ways.
• Luitel, D., Hassani, S. and Sabetzadeh, M. (2024). Improving requirements completeness: automated assistance through large language models. Requirements Engineering 29(1), 73-95. doi:10.1007/s00766-024-00416-3 BERT's masked language model predicted terminology withheld from requirements, evaluated on forty specifications from the PURE dataset, and the predictions highlighted missing terminology. A learned filter reduced noise, so the model acts as a critic with human judgement retained.
• Dalpiaz, F. and Niu, N. (2020). Requirements Engineering in the Days of Artificial Intelligence. IEEE Software 37(4), 7-10. doi:10.1109/MS.2020.2986047 The editorial traces artificial intelligence in requirements engineering from knowledge representation in the 1980s through natural language processing and machine learning. It is a position piece about where the field is heading, and it supplies the unit's closing question.