A request is a sum of delays: propagation, transmission, processing and queueing

A person waiting for a page, a search or a saved document experiences one number: the time between acting and being answered. That number is a sum with four terms. Propagation delay is the time for a signal to cross the distance between the two ends. Transmission delay is the time to push a packet's bits onto each link. Processing delay is what an endpoint spends on the request. Queueing delay is the wait behind other work. This unit argues that only two of the four are genuine design variables. They are the queueing a transaction may incur and the number of round trips it needs. A latency figure that names neither the transaction nor the dominant term is not a budget.

The sources supply the parts. De Schepper, Bagnulo and White (2023) record that much effort has gone into reducing propagation delay by placing servers closer to users. Queueing remains a major and intermittent component, with spikes of hundreds of milliseconds not uncommon even under state-of-the-art queue management. Cardwell and colleagues (2016) reduce the physical limits on a connection to two measured quantities: the round-trip propagation time and the bottleneck bandwidth. They treat every other increase in the round trip as queueing noise. The three that follow take the sum apart, from the name that opens it.

The four terms, and the one relation that links a queue to a wait

Little (1961) proved that in a stationary queueing process with finite means, the average number of units in the system equals the arrival rate multiplied by the average time a unit spends there. The consequence is blunt. At a fixed arrival rate, a queue allowed to stay full is delay, and throughput does not buy that delay back. A report that a service is slow can be rewritten, without loss, as a report that the queue is long.

A service receiving 200 requests a second with 20 requests in flight is holding each of them for 0.1 second, by L equals lambda times W. Halving the number in flight to 10 halves the mean wait at the same arrival rate. Raising the arrival rate to 400 without shortening the queue doubles that wait. This is arithmetic on the theorem rather than a measurement, and it holds only while the process is stationary. Stationarity is exactly the condition a busy server breaks.

A budget that does not name the transaction is not a budget

The rules actually applied come from Nielsen (1993), who restates three limits as design rules. They are about 0.1 second for the feeling of instant reaction, 1 second for an uninterrupted flow of thought, and 10 seconds for keeping attention on the task. Miller (1968) had already observed that the controversy about a single limit came from asking the wrong question, since different human purposes have different acceptable times.

The measurement literature disagrees with the tidiness of those numbers. Galletta and colleagues (2004) assigned 196 undergraduates one constant delay from 0 to 12 seconds across nine search tasks, and they recorded performance, attitudes and behavioural intentions. Published advice, they note, already spans 2 seconds to 12 seconds or more, and their own data flatten at about 4 seconds for intentions and 8 seconds for attitudes. That sample is undergraduates, so the budget the curve licenses describes one population: readers in later life and readers on a slow link are outside it. Claypool (2005) supplies the sharpest case. Induced latency in three popular real-time strategy games correlated only modestly with performance for exploration and very weakly for building and combat. Even very high latency had a negligible effect on the outcome, which led the author to class those games nearer web browsing than first-person shooters. The same second is a different requirement in each transaction.

Queueing is the term a designer can move

Gettys (2011) named the condition that lets queueing dominate: bufferbloat. The term describes excessively large buffers in routers, gateways and broadband equipment. Without active queue management, those buffers fill and stay full, and they destroy the timely packet drops congestion avoidance depends on. Nichols and Jacobson (2012) supply the mechanism and the measurement. The standing queue appears when a sender's window exceeds what the path can hold, and they cite independent measurements of edge queueing delays from hundreds of milliseconds up to multiple seconds. Their remedy, CoDel, acts on how long packets have waited rather than how many are queued. Hoeiland-Joergensen and colleagues (2018) separate the flows, hashing traffic into distinct queues and running CoDel on each, so that one aggressive flow cannot impose its delay on the rest. They record that this is already the default discipline in several Linux distributions. The queueing term is a control loop, and a control loop can be read.

The control law is a published choice, measured on other people's links

The remaining large term is set by endpoints a team does not own. Jacobson (1988) reported the October 1986 congestion collapse, in which throughput between two sites 400 yards apart fell from 32 Kbps to 40 bps. He set out the conservation-of-packets principle with the slow-start and increase-decrease algorithms added to 4BSD TCP. Xu and colleagues (2023) record that CUBIC, which replaces the linear window increase with a cubic function, became the default in the Linux, Windows and Apple stacks before it was standardised. Cardwell and colleagues (2016) contest the signal itself. They argue that loss-based control keeps large buffers full and so produces the very latency Gettys describes, while with small buffers it misreads loss as congestion. Their alternative measures bottleneck bandwidth and round-trip propagation time directly.

The sum changes shape with path length and fan-out

Mathis and colleagues (1997) built and validated a model that predicts a sustained connection's bandwidth from its loss rate and round-trip time. They tested it in simulation and against live internet measurements, so a fixed amount of data takes longer over a longer path. Dean and Barroso (2013) show that when one request fans out to many servers, a small fraction of slow responses sets the latency the user sees. Hedged requests buy a shorter tail with extra work.

Contested question: is the fault the queue or the sender's controller?

The first disagreement concerns the origin of the queueing term. Gettys (2011) locates it in the network, in oversized buffers that hold latency high under load. De Schepper, Bagnulo and White (2023) locate it in the endpoints. They state that the root cause of queueing delay is in the capacity-seeking controllers of senders rather than in the queue itself, so changing the network alone can achieve only so much. Nichols and Jacobson (2012) sit on the bufferbloat side, but they put the remedy where the queue can be observed, at the bottleneck gateway, signalling the sender. Even the two queue-centred sources differ on where the repair belongs. One reading makes the queueing term a deployment problem owned by the network. The other makes it an endpoint problem owned by the protocol. What would settle it is a change of one term with all else held fixed: an AQM-only change with unmodified senders, or a sender-only change with unmodified queues. The nearest published comparison, from De Schepper, Bagnulo and White (2023), reports 1 to 2 milliseconds at the 99th percentile under their own mechanism against 5 to 20 milliseconds on average under Classic control. It is the authors' own evaluation, not an independent test.

The second disagreement concerns the budget. Miller (1968) and Claypool (2005) show that the acceptable time is a property of the transaction. The limits of Nielsen (1993) are carried into practice as though one set of numbers covered every transaction. A paired experiment would settle it: one population, one ladder of delays across two transactions, with both a performance and a preference measure. Galletta and colleagues (2004) come closest, but they vary task content rather than comparing two transactions at the same delays.

The bufferbloat diagnosis rests on independent measurements of edge queueing (Nichols and Jacobson, 2012) and on a named condition (Gettys, 2011). The sender-side position rests on the authors' own evaluation of their own mechanism (De Schepper et al., 2023) rather than on an independent test. While the dispute is open, the designer measures which term dominates the transaction at hand and changes the term the team owns. The strongest objection is that this unit separates the queue from the controller more cleanly than the evidence allows, since both remedies are proposed and measured by the same parties, and no field comparison holds one term fixed while changing the other.

Where the argument is heading

The direction of travel is to define the queueing term as a service rather than tune it. De Schepper, Bagnulo and White (2023) specify a service in which congestion is signalled by explicit marking so that queues stay short while throughput scales. Their target is below 1 millisecond on average and about 2 milliseconds at the 99th percentile. They note that end-to-end delay above 50 milliseconds begins to feel unnatural for demanding interactive work. Cardwell and colleagues (2016) are the sender-side version of the same move, and Hoeiland-Joergensen and colleagues (2018) show the queue-side version in place.

The sources this unit rests on

• Little, J. D. C. (1961). A proof for the queuing formula: L = lambda W. Operations Research. doi:10.1287/opre.9.3.383 Proved that in a stationary queueing process with finite means the average number of units in the system equals the arrival rate times the average time a unit spends there. It is the unit's one equation: a queue allowed to stay full is time that throughput does not buy back.
• Miller, R. B. (1968). Response time in man-computer conversational transactions. AFIPS Fall Joint Computer Conference. doi:10.1145/1476589.1476628 Surveyed terminal response times and argued that the controversy about a single limit comes from asking the wrong question, since different human purposes have different acceptable times. It supplies the unit's sharpening: a budget must name the transaction before it can name a number.
• Nielsen, J. (1993). Response times: the three important limits. Nielsen Norman Group. https://www.nngroup.com/articles/response-times-3-important-limits/ Restated the widely used thresholds as practical design limits: about 0.1 second for direct reaction, 1 second for an uninterrupted flow of thought and 10 seconds for holding attention. It shows the norm a designer is held to as though universal.
• Galletta, D., Henry, R., McCoy, S. and Polak, P. (2004). Web site delays: how tolerant are users? Journal of the Association for Information Systems. doi:10.17705/1jais.00044 Assigned 196 undergraduates one constant delay from 0 to 12 seconds across nine search tasks and measured performance, attitudes and behavioural intentions. Published advice, the authors note, spans 2 seconds to 12 seconds or more, and their data flatten from about 4 seconds for intentions and 8 seconds for attitudes.
• Claypool, M. (2005). The effect of latency on user performance in real-time strategy games. Computer Networks. doi:10.1016/j.comnet.2005.04.008 Induced latency in three popular real-time strategy games and measured performance in exploration, building and combat. Correlations were modest for exploration and very weak for building and combat, and even very high latency had a negligible effect on the outcome.
• Gettys, J. (2011). Bufferbloat: dark buffers in the internet. IEEE Internet Computing. doi:10.1109/mic.2011.56 Argued that network equipment has acquired excessively large buffers which, without active queue management, fill and stay full and invalidate the timely packet drops that congestion avoidance relies on. It places the fault in the equipment rather than in the senders.
• Nichols, K. and Jacobson, V. (2012). Controlling queue delay. Queue. doi:10.1145/2208917.2209336 Diagnosed the standing queue as a mismatch between the sender's window and what the path can hold, cited independent measurements of edge queueing delays from hundreds of milliseconds up to multiple seconds, and proposed CoDel, which acts on how long packets have waited.
• Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, J. and Dumazet, E. (2018). The flow queue CoDel packet scheduler and its applicability to the low latency low loss scaling working group recommendations. RFC 8290. doi:10.17487/rfc8290 Specifies FQ-CoDel, which hashes flows into separate queues and runs CoDel on each so one aggressive flow cannot impose its delay on the rest, and records that it is the default queueing discipline in several Linux distributions.
• Jacobson, V. (1988). Congestion avoidance and control. ACM SIGCOMM. doi:10.1145/52324.52356 Reported the October 1986 congestion collapse, in which throughput between two sites fell from 32 Kbps to 40 bps, and set out the conservation-of-packets principle with the slow-start and increase-decrease algorithms added to 4BSD TCP.
• Xu, L., Ha, S., Rhee, I. and Goel, V. (2023). CUBIC for fast and long-distance networks. RFC 9438. doi:10.17487/rfc9438 Specifies CUBIC, which replaces the linear congestion-window increase with a cubic function to improve scalability over fast and long-distance paths, and records that it is the default congestion control in the Linux, Windows and Apple stacks.
• Cardwell, N., Cheng, Y., Gunn, C. S., Yeganeh, S. H. and Jacobson, V. (2016). BBR: congestion-based congestion control. Queue. doi:10.1145/3012426.3022184 Argues that loss-based control keeps large buffers full and misreads loss when buffers are small, and builds a controller that estimates bottleneck bandwidth and round-trip propagation time instead. The authors attribute round-trip times of seconds to buffers that grew beyond the path's bandwidth-delay product.
• Mathis, M., Semke, J., Mahdavi, J. and Ott, T. (1997). The macroscopic behaviour of the TCP congestion avoidance algorithm. ACM SIGCOMM Computer Communication Review. doi:10.1145/263932.264023 Built and validated a model that predicts a sustained connection's bandwidth from its loss rate and round-trip time, testing it by simulation and against live internet measurements. It links a path's length to the time a fixed amount of data takes.
• Dean, J. and Barroso, L. A. (2013). The tail at scale. Communications of the ACM. doi:10.1145/2408776.2408794 Show that when one logical request fans out to many servers a small fraction of slow responses sets the latency the user sees, and catalogue techniques such as hedged requests that buy a shorter tail with extra work.
• De Schepper, K., Bagnulo, M. and White, G. (2023). Low latency, low loss and scalable throughput (L4S) internet service: architecture. RFC 9330. doi:10.17487/rfc9330 Defines a service in which congestion is signalled by explicit marking so queues stay short while throughput scales, sets a target below 1 millisecond on average and about 2 milliseconds at the 99th percentile, and states that the root cause of queueing delay lies in the senders' controllers.