Handover and the test that outlasts the mark: maintainability, documentation and knowledge loss

This is the ninth of ten units, and the first to look past the assessment: the other eight judge the project while the team is still inside it. This one asks what remains when the team has gone. The test is whether a competent stranger, given the repository alone, can run and maintain the product without asking anyone who built it, and that stranger may read with a screen reader. The evidence is uncomfortable. Developers consult formal prose last and repair it least (Lethbridge et al., 2003). The code's explanatory layer drifts out of step (Wen et al., 2019). The answers a stranger needs sit in fewer heads than the team believes (Mockus et al., 2002). The task is to make the code self-explaining and to document only what the code cannot carry.

The claim rests on three literatures that rarely cite each other. Software evolution explains why maintenance rather than delivery is the real test (Lehman, 1980). Documentation research shows what developers read (Forward and Lethbridge, 2002; Aghajani et al., 2020). Ownership studies show how uneven the loss is (Avelino et al., 2016; Rigby et al., 2016).

The test is maintainability, and the standard names its parts

Lehman (1980) classified programs by their relationship to their environment. He argued that evolutionary pressure makes maintenance a never-ending activity, not a phase that finishes, and stated laws of program evolution from quantitative studies. If maintenance never ends, the moment assessment stops is arbitrary, and the product's life after the mark is where its quality is spent.

ISO/IEC 25010 defines maintainability as the degree of effectiveness and efficiency with which its intended maintainers can modify a product. The 2023 edition replaced the 2011 version and keeps the decomposition into modularity, reusability, analysability, modifiability and testability within a model of nine characteristics, renaming usability as interaction capability and portability as flexibility (International Organization for Standardization, 2023). That makes the stranger test checkable: a repository a maintainer cannot analyse, modify or test has not been handed over. The standard defines rather than demonstrates.

A handover record is a reconstruction, and a rushed one leaves debt with a name

Parnas and Clements (1986) rejected deriving a program systematically from a precise statement of requirements. The useful response, they argued, is documentation presenting the design as though it had been. The record is written afterwards and tidied into a rationale, so it reconstructs reasons a maintainer cannot recover from the code rather than logging events. It fits the thesis only if that prose is the minimum carrying the decisions.

Cunningham (1992) introduced the technical-debt metaphor in an experience report on a portfolio system; Alves and colleagues (2016) later mapped the literature, evaluating 100 studies published between 2010 and 2014 into a taxonomy of debt types, indicators and management strategies. Debt names what a rushed handover leaves: missing or outdated documentation, tests planned and never written, structure that must be untangled later. The evidence base is thin and the interventions are unvalidated, so the vocabulary should make a deferral explicit rather than attach a number to it.

Knowledge concentrates, so turnover is a project-level risk

Mockus, Fielding and Herbsleb (2002) examined Apache and Mozilla through change histories and problem reports. In Apache the top 15 developers contributed more than 83 per cent of modification requests and deltas, 88 per cent of added lines and 91 per cent of deleted lines. The same core produced only 66 per cent of the fixes, because repair drew on a wider group than new functionality did.

Avelino and colleagues (2016) turned the concentration into a metric. They estimated the truck factor, the number of developers who must leave before a project stalls, for 133 popular GitHub projects. They found 65 per cent had a truck factor of two or less. A survey of developers from 67 of the systems confirmed the ranked authors as the main authors in 84 per cent of valid answers. Only 53 per cent accepted the estimated truck factors themselves.

Rigby and colleagues (2016) quantified what turnover costs: in Chrome and at Avaya, losses more than three times expected historically and more than five times expected in simulations. Recommending a successor named the right person 34 to 48 per cent of the time and cut expected loss by at most 15 per cent. Zhou and Mockus (2015) add the timing. A newcomer with one issue fixed in the first month doubled the odds of staying, so a handover should design that month.

Documentation is what developers read last, and the code's own explanatory layer decays too

Lethbridge, Singer and Forward (2003) reported three studies of how engineers use and update documentation, with two results pulling in opposite directions. Engineers do not update documentation as timely or completely as process advisers assume, and yet out-of-date documentation remains useful. Forward and Lethbridge (2002) had earlier surveyed professionals, who wanted tools that extract knowledge from the code and its tests and reported that maintenance rarely happens. Aghajani and colleagues (2020) sharpened the problem. Documentation is valued in principle and suffers content that is inadequate, obsolete and ambiguous in practice, so task needs and document content come apart.

The claim therefore concerns the marginal prose document, not documentation as such: stale documentation is still consulted.

Wen and colleagues (2019) mined 1.3 billion syntax-tree-level changes from the full history of 1,500 systems. They analysed 500 commits into a taxonomy of the comment inconsistencies developers fix. Comments, the cheapest explanatory layer of the code, drift out of step with it. Some changes are likelier than others to leave them behind, because a comment is a claim that nothing tests. Making the code self-explaining cannot mean trusting its comments; it means moving explanation into forms the build checks.

Better architecture does not buy out the risk

Borowa, Ratkowski and Verdecchia (2025) studied a system of more than 100 microservices serving more than 15,000 locations through static analysis, a focus group and an architect interview. Loose coupling had not delivered the maintainability it promises. Inadequate communication contributed significantly to debt, architectural and organisational misalignment made it accumulate, and the system cycled rapidly between accruing and clearing it. The drivers were properties of the team, not the platform.

What the case does not show is that another architecture would have prevented it, since one system cannot separate the two.

Contested question: should the answer be more documentation or less?

The disagreement is not about whether maintainers need help. Parnas and Clements (1986) hold that the design rationale must be written down and that no quantity of code recovers it. Forward and Lethbridge (2002) and Lethbridge, Singer and Forward (2003) supply the opposite emphasis. Engineers depend on code, colleagues and informal sources, and what they want is automation that extracts knowledge from code and tests rather than more prose. Aghajani and colleagues (2020) locate the failure in the fit between document and task rather than in documentation as such. Wen and colleagues (2019) make the dispute harder for both. Comments are documentation that decays, so moving the burden onto the code moves it onto another unreliable layer.

What would settle it is a controlled handover study with three groups. One would receive a prose specification and a decision log. A second would receive the same repository with self-explaining names and no prose. A third would receive both. The outcome would be time to the first correct change and the number of questions a maintainer asked. The closest evidence is the successor experiment of Rigby and colleagues (2016), which shows the transfer channel unreliable whatever the medium. No such study exists at student-team scale, and the debt literature offers mostly unvalidated proposals (Alves et al., 2016). Until it exists, the defensible position is better-placed documentation rather than less: the decisions written down, the rest carried by code and tests.

The two cases rest on different kinds of evidence. The documentation case is an argument from rationality, with no data behind it (Parnas and Clements, 1986). The code case is three studies of how engineers work (Forward and Lethbridge, 2002; Lethbridge et al., 2003; Aghajani et al., 2020). A mining study of comment decay makes the dispute harder (Wen et al., 2019), and all of it is observational. The strongest objection is that this middle position is a preference, since no handover study has compared prose against code.

Where the argument is heading

The direction of travel is from producing documentation to placing information where the maintainer looks. Aghajani and colleagues (2020) point towards recommenders that supply what a task needs. Borowa and colleagues (2025) close on continuous monitoring and incremental rewrites beside the communication practices their case study found decisive. For a student team the reading stays modest. Write down the decisions a stranger cannot recover from the code, test the behaviour the code should have, and keep the decision log beside the code.

The sources this unit rests on

• Aghajani, E., Nagy, C., Linares-Vasquez, M., Moreno, L., Bavota, G., Lanza, M. and Shepherd, D. C. (2020). Software documentation: the practitioners' perspective. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 590-601. doi:10.1145/3377811.3380405 Surveyed practitioners on what documentation is for and why it falls short. Found it valued in principle while suffering inadequate, obsolete and ambiguous content, locating the failure in the fit between document and task.
• Alves, N. S. R., Mendes, T. S., de Mendonca, M. G., Spinola, R. O., Shull, F. and Seaman, C. (2016). Identification and management of technical debt: a systematic mapping study. Information and Software Technology, 70, 100-121. doi:10.1016/j.infsof.2015.10.008 Mapped 100 studies published between 2010 and 2014 into a taxonomy of debt types, indicators and management strategies. Supplies the unit's debt vocabulary and its caution that most interventions remain unvalidated.
• Avelino, G., Passos, L., Hora, A. and Valente, M. T. (2016). A novel approach for estimating truck factors. 2016 IEEE 24th International Conference on Program Comprehension (ICPC), 1-10. doi:10.1109/ICPC.2016.7503718 Estimated truck factors for 133 popular GitHub projects and surveyed developers from 67 of them. Found 65 per cent had a truck factor of two or less; in 84 per cent of valid answers developers confirmed the ranked authors as their system's main authors, and in 53 per cent they accepted the estimated truck factors. It quantifies the concentration the unit describes.
• Borowa, K., Ratkowski, A. and Verdecchia, R. (2025). The technical debt gamble: a case study on technical debt in a large-scale industrial microservice architecture. Journal of Systems and Software, 230, 112547. doi:10.1016/j.jss.2025.112547 Studied a system of more than 100 microservices through static analysis, a focus group and an architect interview. Loose coupling did not protect maintainability and communication failures drove debt, so the drivers are team properties.
• Cunningham, W. (1992). The WyCash portfolio management system. Addendum to the proceedings of OOPSLA 1992. doi:10.1145/157709.157715 Introduces the technical-debt metaphor from an experience report on a portfolio system. It is the primary source of the debt vocabulary; its limit is that it is one project's account, with no measure of the cost.
• Forward, A. and Lethbridge, T. C. (2002). The relevance of software documentation, tools and technologies. Proceedings of the 2002 ACM symposium on Document engineering, 26-33. doi:10.1145/585058.585065 Surveyed software professionals on documentation and the tools that maintain it. Maintenance rarely happened, and respondents wanted technologies that extract knowledge from source and test code, which supports placing explanation close to the code.
• International Organization for Standardization (2023). ISO/IEC 25010:2023 systems and software engineering: SQuaRE product quality model. ISO. https://www.iso.org/standard/78176.html Defines a product quality model of nine characteristics, maintainability among them, decomposed into modularity, reusability, analysability, modifiability and testability. It gives the stranger test a standard form. The 2023 edition cancels and replaces the 2011 version, renames usability as interaction capability and portability as flexibility, and adds safety.
• Lehman, M. M. (1980). Programs, life cycles, and laws of software evolution. Proceedings of the IEEE, 68(9), 1060-1076. doi:10.1109/PROC.1980.11805 Classifies programs by their relationship to their environment and argues that evolutionary pressure produces never-ending maintenance, with laws stated from quantitative studies. It is the ground for treating maintainability as the product's real test.
• Lethbridge, T. C., Singer, J. and Forward, A. (2003). How software engineers use documentation: the state of the practice. IEEE Software, 20(6), 35-39. doi:10.1109/MS.2003.1241364 Reported three studies of how engineers use and update documentation. Engineers update it less than process advisers assume while out-of-date documentation stays useful, which limits the caution.
• Mockus, A., Fielding, R. T. and Herbsleb, J. D. (2002). Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology, 11(3), 309-346. doi:10.1145/567793.567795 Quantified participation and code ownership in Apache and Mozilla. The top 15 Apache developers produced more than 83 per cent of modification requests and deltas while the same core produced only 66 per cent of the fixes.
• Parnas, D. L. and Clements, P. C. (1986). A rational design process: how and why to fake it. IEEE Transactions on Software Engineering, SE-12(2), 251-257. doi:10.1109/TSE.1986.6312940 Argue that a program cannot be derived systematically from a precise requirements statement in practice, but that documentation can present the design as if it had been. It defines the unit's handover record and sets up the contested question.
• Rigby, P. C., Zhu, Y. C., Donadelli, S. M. and Mockus, A. (2016). Quantifying and mitigating turnover-induced knowledge loss: case studies of Chrome and a project at Avaya. Proceedings of the 38th International Conference on Software Engineering, 1006-1016. doi:10.1145/2884781.2884851 Measured susceptibility to developer turnover, finding losses over three times expected. Correct successor recommendation reached 34 to 48 per cent and cut expected loss by at most 15 per cent.
• Wen, F., Nagy, C., Bavota, G. and Lanza, M. (2019). A large-scale empirical study on code-comment inconsistencies. 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC), 53-64. doi:10.1109/icpc.2019.00019 Mined 1.3 billion syntax-tree-level changes across 1,500 systems and analysed 500 commits into a taxonomy of comment inconsistencies. Comments drift out of step with code, so the code's explanatory layer is not self-maintaining.
• Zhou, M. and Mockus, A. (2015). Who will stay in the FLOSS community? Modeling participant's initial behavior. IEEE Transactions on Software Engineering, 41(1), 82-99. doi:10.1109/tse.2014.2349496 Modelled newcomer behaviour in the issue trackers of Mozilla and Gnome. A newcomer with one issue fixed in the first month doubled the odds of staying, so a handover should design the successor's first month.