Stereo displays break a coupling the eye relies on

A stereo head-mounted display is bought on the numbers that describe a flat screen: pixels per degree, refresh rate, colour gamut. None of them settles whether the depth looks right. Depth is manufactured from binocular disparity, and the visual system does not read disparity alone. It couples the angle at which the eyes converge with the distance at which the lens focuses, and it has learned that the two agree (Watt, Akeley, Ernst and Banks, 2005; Banks, Hoffman, Kim and Wetzstein, 2016). A conventional stereo display breaks that agreement, because every object is drawn on one physical surface while the eyes are told to converge at the depicted depth. This fourth of ten units argues that the broken coupling, not insufficient resolution, drives much reported discomfort and misperceived depth, the first perceptual cost of the display the third described.

Focus is a depth cue, so breaking the coupling changes what is seen

An ordinary stereo display holds accommodation fixed at the screen while disparity drives vergence to the depicted depth, so the two cues disagree. Whether that reaches perception, or exhausts itself in comfort, was the question. Watt, Akeley, Ernst and Banks (2005) answered it by manipulating accommodation and the gradient of retinal blur independently of disparity. Their observers judged depth differently when the focus cues changed. That is the result this unit rests on. Banks, Hoffman, Kim and Wetzstein (2016) argue that a realistic depth experience requires correct focus cues, which most deployed displays do not provide. A display that gets it wrong therefore distorts the depth percept rather than only tiring the eye.

The conflict has thresholds, and they are small

The literature on how much is too much is unusually concrete. Lambooij, Fortuin, Heynderickx and IJsselsteijn (2009) reviewed the causes of discomfort in stereoscopic displays. Discomfort rises sharply once disparity exceeds about one degree, with viewing time, blur, crosstalk and object position all contributing, so the conflict is the field's organising explanation rather than its only comfort factor. Kooi and Toet (2004) put numbers on the wider set of imperfections, rating brief binocular image pairs that carried one error at a time. Their limits run to about two to three prism dioptres of horizontal shift, one dioptre of vertical shift, 30 arc minutes of disparity and five per cent crosstalk for high-disparity images, with vertical disparity, crosstalk and blur the strongest factors.

The most useful single result for design is the zone of comfort. Shibata, Kim, Hoffman and Banks (2011) measured discomfort at two viewing distances and at both signs of the conflict. They derived the region of vergence and focal distance where viewing is tolerable.

The conflict costs performance as well as comfort

Discomfort is easy to dismiss as a nuisance, but the stronger claim is that wrong focus cues make the user worse at the job. Hoffman, Girshick, Akeley and Banks (2008) built a display that presented focus cues correct or nearly correct for the scene. Against an ordinary stereo display in four experiments, participants under the corrective display identified a stereoscopic stimulus faster, showed better stereoacuity, and reported less distortion and less fatigue. That turns the thresholds above from a comfort rule into a performance rule, and it names the remedy the next section sets out.

Correct focus cues can be built, and the prototype sets the specification

If the conflict is the problem, the remedy is a display that presents focus cues at the right distance. Akeley, Watt, Girshick and Banks (2004) built a prototype from two volumetric displays. It was a proof of concept, but it shows the conflict is a limit of current displays, not of optics.

Ravikumar, Akeley and Banks (2011) turned the prototype into a specification. They measured how observers respond when a limited number of focal planes approximates a continuous depth range. They report the plane spacing and count needed for the cues to read as continuous. The technique has a known ceiling. MacKenzie, Hoffman and Watt (2010) found that accommodation followed the simulated distance accurately for image-plane separations up to about one diopter and broke down beyond that, which bounds how completely the technique can remove the conflict.

Where the optics cannot be fixed, the content can be managed

A product cannot wait for correct focus cues, so the workable response is to make the conflict less costly while the optics catch up. Carnegie and Rhee (2015) applied a dynamic depth-of-field effect in a head-mounted display, blurring content according to its depicted depth, and report less discomfort than a conventional rendering. March, Krishnan, Mantiuk and Watt (2024) found the same shortcut reduced depth realism rather than restoring it, which is why the mitigation manages the viewer's expectation rather than curing the problem.

Konrad, Padmanaban, Molner, Cooper and Wetzstein (2017) take the opposite route, designing the display so accommodation stops mattering. Their accommodation-invariant near-eye display presents a retinal blur matched to the viewer's accommodation state instead of a true focal plane. It gives accommodation invariance from a compact design, at the cost of accuracy elsewhere.

The contested question: is the depth error optical, or a property of the display generation?

The thesis is a causal claim with a serious rival. On the conflict position, wrong focus cues distort perceived depth. The error is therefore optical and the remedy is correct cues (Watt et al., 2005; Hoffman et al., 2008; Banks et al., 2016). On the generation position, much of the depth error belongs to the display and the renderer rather than to the conflict. Correcting the optics would therefore leave it standing.

Kelly (2023) supports the second position. His meta-analysis pooled 137 samples from 61 publications across 20 head-mounted displays, confirming the underperception of distance in virtual reality and reporting that judged distance is positively associated with field of view and resolution. That places part of the classic error in the hardware generation, and Krajancich, Kellnhofer and Wetzstein (2020) lend the rendering side the same argument, correcting stereo rendering for the gaze-dependent shift of the eye's no-parallax point and reporting improved depth perception without touching focus cues. Hibbard, Haines and Hornsey (2017) add that stereo cues improve the precision of depth judgements more reliably than the accuracy of large depth judgements.

What would settle the dispute is a design that separates the two effects inside one system. It would present the same scene under four conditions, combining correct or incorrect focus cues with a gaze-contingent or a conventional render. It would hold field of view and resolution constant, and measure judged distance and precision against a real object. If the conflict position is right, the focus-cue manipulation moves the outcome and the rendering one does not. If the generation position is right, the reverse holds. No study in this unit's list runs that design.

The two sides rest on different kinds of evidence. The conflict position rests on a purpose-built corrective display and four controlled experiments (Hoffman et al., 2008), evidence of mechanism rather than of prevalence, while the generation position rests on a meta-analysis of 137 samples (Kelly, 2023), evidence of size rather than of cause. The unit's own thesis is the conflict position, and its strongest counter-argument is that a fix aimed at the coupling would leave most of the measured depth error standing. What a designer should do while the dispute is unresolved: keep disparities inside the conditional zone of comfort, keep the monocular depth cues in place for a viewer without binocular vision, and correct the renderer as well as the optics (Shibata et al., 2011).

Where this is heading

The direction of the field is a convergence on correct focus cues rather than on more pixels. Banks and colleagues (2016) frame correct focus cues as the requirement most deployed displays fail. Konrad and colleagues (2017) show a compact computational route to accommodation invariance. Kelly (2023) supplies the caution: judged distance tracks field of view and resolution, so a fix aimed only at the conflict will not clear the whole error.

Sources

• Akeley, K., Watt, S. J., Girshick, A. R. and Banks, M. S. (2004). A stereo display prototype with multiple focal distances. ACM SIGGRAPH 2004 Papers. doi:10.1145/1186562.1015804 Built a prototype from two volumetric displays that generate near-correct focus cues without tracking eye position, because light comes from sources at the correct physical distances, and demonstrated it with three image planes. The authors describe it as a proof of concept rather than a product; it establishes that the conflict is a limit of current displays, not of optics.
• Watt, S. J., Akeley, K., Ernst, M. O. and Banks, M. S. (2005). Focus cues affect perceived depth. Journal of Vision. doi:10.1167/5.10.7 Tested whether the visual system uses focus cues for depth by manipulating accommodation and the gradient of retinal blur independently of disparity. Focus cues did affect perceived depth, so the wrong cues in a conventional stereo display distort the depth percept rather than only causing fatigue.
• Hoffman, D. M., Girshick, A. R., Akeley, K. and Banks, M. S. (2008). Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. Journal of Vision. doi:10.1167/8.3.33 Compared a display presenting near-correct focus cues against an ordinary stereo display in four experiments. Participants identified a stereoscopic stimulus faster and showed better stereoacuity and less fatigue under the corrective display, which converts a comfort complaint into a measured performance deficit.
• Lambooij, M., Fortuin, M., Heynderickx, I. and IJsselsteijn, W. (2009). Visual discomfort and visual fatigue of stereoscopic displays: a review. Journal of Imaging Science and Technology. doi:10.2352/j.imagingsci.technol.2009.53.3.030201 Reviewed the causes of visual discomfort and fatigue in stereoscopic displays. Discomfort rises sharply once disparity exceeds about one degree, and viewing time, blur, crosstalk and the position of objects relative to the screen all contribute, giving design constraints rather than a single cause.
• Kooi, F. L. and Toet, A. (2004). Visual comfort of binocular and 3D displays. Displays. doi:10.1016/j.displa.2004.07.004 Showed observers brief binocular image pairs carrying one imperfection each, from shifts and magnification to crosstalk and disparity, and derived thresholds for the onset of discomfort, including one prism dioptre of vertical shift, 30 arc minutes of disparity and five per cent crosstalk for high-disparity images. Vertical disparity, crosstalk and blur were the strongest factors.
• MacKenzie, K. J., Hoffman, D. M. and Watt, S. J. (2010). Accommodation to multiple-focal-plane displays: implications for improving stereoscopic displays and for accommodation control. Journal of Vision. doi:10.1167/10.8.22 Measured accommodation responses to depth-filtered stimuli on a multiple-focal-plane display. Accommodation followed the simulated focal distance accurately for image-plane separations up to about one diopter and broke down at larger separations, which limits how completely the technique can remove the conflict and constrains how far apart the planes may sit.
• Ravikumar, S., Akeley, K. and Banks, M. S. (2011). Creating effective focus cues in multi-plane 3D displays. Optics Express. doi:10.1364/oe.19.020940 Measured how the visual system responds to multi-plane displays that approximate a continuous depth range with a limited number of focal planes. They report the plane spacing and number needed for the focus cues to be treated as continuous, which converts the prototype into a specification.
• Shibata, T., Kim, J., Hoffman, D. M. and Banks, M. S. (2011). The zone of comfort: predicting visual discomfort with stereo displays. Journal of Vision. doi:10.1167/11.8.11 Measured visual discomfort at two viewing distances and for both signs of the vergence-accommodation conflict, and derived a zone of comfort plotted in diopters. The boundaries depend on viewing distance and on the sign of the conflict, so the safe region is conditional rather than a single disparity limit.
• Carnegie, K. and Rhee, T. (2015). Reducing visual discomfort with HMDs using dynamic depth of field. IEEE Computer Graphics and Applications. doi:10.1109/mcg.2015.98 Applied a dynamic depth-of-field effect in a head-mounted display, blurring content according to its depicted depth, and compared discomfort against a conventional rendering. Discomfort fell, and the authors argue that where the conflict cannot be removed, blur that manages the viewer's expectation is the practical mitigation.
• Banks, M. S., Hoffman, D. M., Kim, J. and Wetzstein, G. (2016). 3D displays. Annual Review of Vision Science. doi:10.1146/annurev-vision-082114-035800 Reviewed three-dimensional display technology and the perceptual requirements it must meet, covering binocular and focus-cue approaches. They argue that a realistic depth experience requires correct focus cues, which most deployed displays do not provide.
• Hibbard, P. B., Haines, A. E. and Hornsey, R. L. (2017). Magnitude, precision, and realism of depth perception in stereoscopic vision. Cognitive Research: Principles and Implications. doi:10.1186/s41235-017-0062-7 Assessed how binocular disparity affects the magnitude and the precision of perceived depth and how each relates to the sense of three-dimensionality. Stereo cues improved precision more reliably than the accuracy of large depth judgements, so stereo does not simply make depth realistic.
• Konrad, R., Padmanaban, N., Molner, K., Cooper, E. A. and Wetzstein, G. (2017). Accommodation-invariant computational near-eye displays. ACM Transactions on Graphics. doi:10.1145/3072959.3073594 Presented an accommodation-invariant computational near-eye display that uses a retinal blur matched to the viewer's accommodation state instead of a true focal plane. The approach produces accommodation invariance from a compact optical design, at the cost of accuracy elsewhere.
• Krajancich, B., Kellnhofer, P. and Wetzstein, G. (2020). Optimising depth perception in virtual and augmented reality through gaze-contingent stereo rendering. ACM Transactions on Graphics. doi:10.1145/3414685.3417820 Introduced gaze-contingent stereo rendering that accounts for the gaze-dependent shift of the eye's no-parallax point. Depth perception improved against conventional stereo rendering, which shows that part of the depth error is a rendering problem rather than an optical one.
• Kelly, J. W. (2023). Distance perception in virtual reality: a meta-analysis of the effect of head-mounted display characteristics. IEEE Transactions on Visualization and Computer Graphics. doi:10.1109/tvcg.2022.3196606 Meta-analysed 137 samples from 61 publications to describe egocentric distance perception across 20 head-mounted displays. The analysis confirms the underperception of distance but finds judged distance positively associated with field of view and resolution, placing part of the error in the hardware generation.
• March, J., Krishnan, A., Mantiuk, R. K. and Watt, S. J. (2024). Impact of focus cue presentation on perceived realism of 3-D scene structure. Journal of Vision. doi:10.1167/jov.24.2.13 Compared depth realism with correct and incorrect focus cues on a multiple-focal-planes display. Depth realism was consistently lower with incorrect cues for isolated random-dot stimuli, but the benefit of correct cues was less clear for realistic scenes, and gaze-contingent depth-of-field rendering reduced realism rather than restoring it.