Load, queues and the response floor: the mathematics of latency

A product team writes a budget for a search screen: results within 400 milliseconds, timed in isolation on a workstation with one user. The arithmetic is correct and the budget unsupported, since 400 milliseconds describes one request against a system that is rarely idle. This unit argues that perceived latency is a queuing property of the whole system, so a budget computed from the software alone mispredicts the experience whenever arrival rate or service-time variance is high. The mathematics that fixes such a budget is queuing theory: Little (1961) proved that the mean number of units in a system equals the arrival rate multiplied by the mean time a unit spends there, so the wait a person feels rises with the number of requests in flight. Unit 9 bounded what a measurement on people can claim; this unit describes the wait the system imposes.

A response time belongs to the system, not to the request

Little's relation holds across queue disciplines and service distributions. It needs only finite means, a strictly stationary process and an arrival process metrically transitive with non-zero mean (Little, 1961). A request that takes 50 milliseconds against an idle server and 500 at the ninety-fifth percentile under 900 requests a second is the same request: the mean number in the system changed and no line of code did. Gross, Shortle, Thompson and Harris (2008) develop the Markovian models that make the dependence explicit, and note that near saturation waiting grows beyond engineering control as utilisation approaches one.

The budget has a history, and it is a list rather than a number

Miller (1968) opened the question by refusing it:

The literature concerning man-computer transactions abounds in controversy about the limits of "system response time" to a user's command or inquiry at a terminal.

His diagnosis was that the controversy came from asking "response time to what?", and his answer was seventeen response categories, each with its own requirement: control activation no more than 0.1 second, a routine acknowledgement two seconds, a complex graphical inquiry ten (Miller, 1968). Shneiderman (1984) found the same decomposition, with an addition the folklore has dropped: frequent users prefer below one second for most tasks, but error rates rise when response is too short as well as too long. Dannenbring (1984) separated two variables a single budget runs together, measuring a delay and its variability in two tasks built to maximise any adverse effect of either, finding the queue largely controls the variability.

The queue, not the code, sets the floor

The dependence can be made numerical. Take a single server with a mean service time of 10 milliseconds and a service rate of 100 requests a second, treated as the single-server Markovian queue Gross and colleagues (2008) develop; the arithmetic is a constructed case rather than a measurement. At 80 requests a second, utilisation is 0.8 and the mean time in the system is 1 divided by (100 minus 80) seconds, or 50 milliseconds; at 95 requests a second, utilisation is 0.95 and the mean time is 1 divided by 5, or 200 milliseconds. A rise of 19 per cent in load has multiplied the wait by four, and Little's relation gives the mean number in flight as 95 multiplied by 0.2, or 19. Variance matters as much: the same text's general-service model waits on the second moment of service time, so a pipeline with occasional slow paths waits longer than one with uniform work. The server is one station in the pipeline and the link is another, so the same service time is a longer wait over a slow connection. A budget timed on a workstation with one user has excluded the link by assumption.

What people notice sits below what the pipeline delivers

Ng, Lepinski, Wigdor, Sanders and Dietz (2012) measured the latencies people notice: participants experienced touch latencies from 50 to 200 milliseconds down to about 1 millisecond and greatly preferred lower values, with improvement continuing below 10 milliseconds. Jota, Ng, Dietz and Wigdor (2013) moved the outcome from preference to performance, measuring the effect of the up-to-125-millisecond delay commercial devices then presented on direct-touch tasks such as physical manipulation. Kaaresoja, Brewster and Lantz (2014) varied the touch-to-feedback delay from 0 to 300 milliseconds across three modalities, finding perceived quality falling significantly between 70 and 100 milliseconds for tactile and audio and between 100 and 150 for visual.

The threshold depends on the channel carrying the feedback

Kurzweg, Willms, Chaffangeon Caillet and Wolf (2024) built resonant sticker buttons in three sizes and varied the feedback delay. They measured perceived button weight rather than reported simultaneity: higher latency made the button feel heavier, and larger buttons felt lighter. Elfleet and Chollet (2024) reached the same conclusion under a delay the system could not remove. The quantity to model is perceived latency, which the interface constructs as well as the pipeline causes.

The budget is enforced against a distribution

The Core Web Vitals define three field metrics with thresholds: largest contentful paint within 2.5 seconds of the page starting to load, interaction to next paint at 200 milliseconds or less, and cumulative layout shift at 0.1 or less, assessed at the 75th percentile of page loads from real visits (Google, 2020, updated 2024). Doherty and Sorenson (2015) argue that when response time pulls attention back to the technology the user leaves the absorbed state of working without thought, and that no metric yet correlates satisfaction with response time. Delaney, Ward and McLoone (2006) set out the trade-off that limits how far latency can be removed: tightening the shared state and shortening the response pull against each other.

A field experiment gives the threshold a measured counterpart. Google injected server-side delay into its search results and compared searches per user against a control group over four to six weeks, finding that a delay of 100 to 400 milliseconds reduced daily searches per user by 0.2 to 0.6 per cent. The effect grew with exposure, and users who had seen the 400-millisecond delay were still searching 0.21 per cent less five weeks after it was withdrawn (Brutlag, 2009). The measurement is small, cumulative and partly persistent, so it is not a psychophysical threshold, and it was applied to the page rather than to the response to an input.

Contested question: whether one latency number can be right

The literature disagrees about what a latency budget is a budget for. One position holds that a threshold is meaningful only per transaction class, and Miller (1968) treats a single limit across purposes as the source of the controversy. A second holds that a few operational thresholds are the only workable instrument: the Core Web Vitals apply one threshold per metric to every page on the web (Google, 2020, updated 2024), and the touch literature agrees, since Ng and colleagues (2012) and Jota and colleagues (2013) both set their thresholds below 150 milliseconds. The two positions rest on different evidence: Miller's list is an argument from purpose with no measurement attached, while the Core Web Vitals thresholds come from a vendor's field data published as living documentation, not peer-reviewed findings.

A sharper disagreement is whether less latency always helps. Ng and colleagues (2012) and Kaaresoja and colleagues (2014) found preference and quality improving with lower delay. Against this, Tosic, Hoffman and Balram (2021) measured simulator sickness at 18 and 28 milliseconds and found individual differences but no sample-wide significant difference, noting that earlier studies used much higher latencies. A reduction that improves preference need not change an outcome measure, and Shneiderman (1984) found error rates rising at very short response times too.

Yang and colleagues (2025) supply half of what the unit asks for, measuring in virtual reality the latency people notice across tasks of differing difficulty and interaction strategy and estimating a threshold between 130 and 170 milliseconds that moved with both. Users grew more sensitive to delay when the strategy emphasised speed or the task became harder, and the effect of latency on performance was not linear. The threshold therefore tracked the task rather than the display, which supports the first position, but a figure far above the touch thresholds leaves open which literature an interface should be held to. What would settle it is a second study, and it has not been run. Meanwhile, set the budget per transaction class and per feedback channel, state the load it holds under, and measure it as a distribution at a fixed percentile.

Where the argument is heading

The direction of travel is from a number measured once to a distribution measured in the field. Google (2020, updated 2024) specifies field metrics at the 75th percentile, and Yang and colleagues (2025) treat the perceptual threshold itself as a property of the task rather than of the device. The second study is still lacking: several transaction classes on one pipeline, with performance and preference recorded apart.

The sources this unit rests on

• Little, J. D. C. (1961). A proof for the queuing formula L = lambda W. Operations Research, 9(3), 383-387. doi:10.1287/opre.9.3.383 Proves that in a stationary queuing process the mean number of units in the system equals the arrival rate times the mean time a unit spends there, whatever the service distribution.
• Miller, R. B. (1968). Response time in man-computer conversational transactions. Proceedings of the AFIPS Fall Joint Computer Conference, 267-277. doi:10.1145/1476589.1476628 Classifies seventeen response categories, from 0.1 second for control activation to ten seconds for a complex graphical inquiry, and originates the case against a single threshold.
• Shneiderman, B. (1984). Response time and display rate in human performance with computers. ACM Computing Surveys, 16(3), 265-285. doi:10.1145/2514.2517 Reports that frequent users prefer response times below one second, and that error rates rise when response time is too short as well as too long. Finds no predictive model yet.
• Dannenbring, G. L. (1984). System response time and user performance. IEEE Transactions on Systems, Man, and Cybernetics, SMC-14(3), 473-478. doi:10.1109/TSMC.1984.6313241 Measures the effect of delays on performance and satisfaction in two tasks built to maximise any adverse effect of delay or its variability.
• Gross, D., Shortle, J. F., Thompson, J. M. and Harris, C. M. (2008). Fundamentals of Queueing Theory. Wiley, 4th edition. doi:10.1002/9781118625651 A standard text developing the Markovian queueing models, including the single-server model behind this unit's constructed case and the general-service model whose wait depends on the second moment of service time.
• Delaney, D., Ward, T. and McLoone, S. (2006). On consistency and network latency in distributed interactive applications: a survey, part I. Presence: Teleoperators and Virtual Environments, 15(2), 218-234. doi:10.1162/pres.2006.15.2.218 (companion part II: 15(4), 465-482, doi:10.1162/pres.15.4.465) A two-part survey of consistency and latency in distributed interactive applications that sets out the trade-off between consistency and prompt response.
• Ng, A., Lepinski, J., Wigdor, D., Sanders, S. and Dietz, P. (2012). Designing for low-latency direct-touch input. Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST). doi:10.1145/2380116.2380174 Participants experienced touch latencies from 50 to 200 milliseconds down to about 1 millisecond, and greatly preferred lower latency, with improvement continuing below 10 milliseconds.
• Jota, R., Ng, A., Dietz, P. and Wigdor, D. (2013). How fast is fast enough? A study of the effects of latency in direct-touch pointing tasks. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). doi:10.1145/2470654.2481317 Reports that commercial devices then presented up to 125 milliseconds, and measures the effect on direct-touch tasks such as physical manipulation.
• Kaaresoja, T., Brewster, S. and Lantz, V. (2014). Towards the temporally perfect virtual button: touch-feedback simultaneity and perceived quality in mobile touchscreen press interactions. ACM Transactions on Applied Perception, 11(2), 1-25. doi:10.1145/2611387 Varied the touch-to-feedback delay from 0 to 300 milliseconds across three modalities: quality fell significantly between 70 and 100 milliseconds for tactile and audio and between 100 and 150 for visual.
• Doherty, R. A. and Sorenson, P. (2015). Keeping users in the flow: mapping system responsiveness with user experience. Procedia Manufacturing, 3, 4384-4391. doi:10.1016/j.promfg.2015.07.436 Argues that when response time pulls attention back to the technology the user leaves the absorbed state of working without thought, and that no metric yet correlates satisfaction with response time.
• Tosic, I., Hoffman, D. and Balram, N. (2021). Effect of latency on simulator sickness in smartphone virtual reality. Journal of the Society for Information Display, 29(7), 561-572. doi:10.1002/jsid.999 Tested simulator sickness at 18 and 28 milliseconds in smartphone virtual reality and found individual differences but no sample-wide significant difference.
• Brutlag, J. (2009). Speed matters for Google web search. Google technical report, June 2009. https://services.google.com/fh/files/blogs/google_delayexp.pdf Injected 100 to 400 milliseconds of server-side delay into search results and measured daily searches per user against a control group over four to six weeks: 0.2 to 0.6 per cent fewer searches, growing with exposure and persisting after withdrawal.
• Google (2020, updated 2024). Web Vitals. web.dev (Google documentation, living page, last updated 31 October 2024). https://web.dev/articles/vitals Defines three field metrics with thresholds, assessed at the 75th percentile of page loads: largest contentful paint at 2.5 seconds, interaction to next paint at 200 milliseconds and cumulative layout shift at 0.1, the last-named replacing first input delay.
• Elfleet, M. and Chollet, M. (2024). Investigating the impact of multimodal feedback on user-perceived latency and immersion with LLM-powered embodied conversational agents in virtual reality. Proceedings of the ACM International Conference on Intelligent Virtual Agents (IVA), 1-9. doi:10.1145/3652988.3673965 Eighteen participants interacted with a virtual-reality conversational agent with and without added multimodal feedback, which significantly improved presence, immersion and perceived response time.
• Kurzweg, M., Willms, J., Chaffangeon Caillet, A. and Wolf, K. (2024). Resonant sticker buttons: the effect of button size and feedback latency on perceived button weight and vibrotactile feedback strength. Proceedings of the International Conference on Mobile and Ubiquitous Multimedia (MUM), 210-227. doi:10.1145/3701571.3701591 Built resonant sticker buttons in three sizes and varied the feedback delay: higher latency made the button feel heavier, while larger buttons felt lighter.
• Yang, S., Yue, K., Gao, H., Guo, M., Liu, Y. and others (2025). Dynamic changes of latency perception threshold in virtual reality. IEEE Transactions on Visualization and Computer Graphics, 31(12), 10340-10355. doi:10.1109/TVCG.2025.3596919 Measured in virtual reality the latency people notice across tasks of differing difficulty and interaction strategy, estimating a threshold of 130 to 170 milliseconds that moved with both. The effect of latency on performance was not linear, and electroencephalogram signals detected delays participants did not report.