MECH5720 Chap.8 Light Fields and Depth
Light Fields and Depth
A light field records ray position and direction, allowing views to be shifted and integrated after capture. Under a two-plane parameterisation, scene points form lines in epipolar-plane images, and calibrated slope or disparity provides depth information. Refocusing selects a shear that aligns one depth plane while misaligning others.
Spatial-angular sampling, occlusion, exposure variation, motion and non-Lambertian reflection limit the ideal model and should be visible in the validation evidence. A two-plane parameterisation indexes a ray by where it intersects an aperture plane and an image plane. In ideal free space, radiance remains constant along the ray, so four coordinates are sufficient after removing position along the ray and time.
A lenslet camera samples position and direction simultaneously. Pixels beneath one lenslet distinguish directions, while the lenslet array distinguishes spatial locations, creating a tradeoff under a fixed detector budget. Sub-aperture views rearrange those samples into images observed from different aperture positions. A scene point moves across those views according to disparity.
Holding one spatial and one angular coordinate forms an epipolar-plane image, where a Lambertian point traces a line. The line slope relates to depth only after view spacing, focal scale, axis orientation and sign convention are known. Calibration with planar targets establishes that mapping and reveals distortion that can curve an otherwise straight trace. Digital refocusing tests a candidate depth.
Shift every sub-aperture view by the displacement predicted for that plane and average the aligned samples. Points at the selected depth add coherently, while other depths remain displaced and blur. The same crop and boundary policy must be used for every candidate shear so padding cannot determine the sharpness score. Record effective aperture and retained views because both affect depth discrimination.
Metric depth commonly follows Z = fB/d for rectified geometry. Focal length and disparity must share pixel units, while baseline supplies physical scale. Depth sensitivity grows as disparity becomes small, so one-pixel error has a larger metre effect on distant points. Confidence should reflect texture, matching-cost ambiguity, repeatability and agreement across multiple view baselines.
Missing depth is appropriate where the data do not identify a match. Occlusion breaks a trace because a point is invisible from part of the aperture. Specular reflection changes brightness or apparent feature position with view, violating the simple Lambertian assumption. Sequential synthetic-aperture capture also turns object motion into broken or curved structure.
Diagnose these effects with sub-aperture montages, labelled epipolar slices and focal stacks, not one refocused frame. A strong result shows known planes sharpening in calibrated order, reports depth uncertainty and excludes regions whose ray consistency is not supported.
What this chapter covers
- 01
Two-plane ray parameterisation
- 02
Spatial-angular sampling
- 03
Epipolar-plane images
- 04
Shift-and-sum refocusing
- 05
Disparity and depth
- 06
Occlusion and view dependence
Worked example · free
Infer depth from calibrated disparity
- +1Use Z = fB/d with focal length and disparity both in pixels.
- +1Compute fB = 700 times 0.08 = 56 pixel-metres.
- +1Compute Z = 56/14 = 4.00 m.
- +1At disparity 13 pixels, the estimate becomes 56/13 = 4.31 m.
- +1Conclude that small-disparity depth is sensitive to pixel error.
Key terms
- Light field
- A representation of rays indexed by position and direction.
- Sub-aperture view
- An image formed from a selected region or position in the sampled aperture.
- Refocusing
- Shearing and integrating light-field views to align a chosen depth plane.
- Disparity
- The displacement of corresponding scene features between views.
- Occlusion
- Loss of visibility of a scene point from part of the aperture.
Light Fields and Depth FAQ
What does a lenslet array trade?
A fixed detector divides samples between spatial position and ray direction. More pixels beneath each lenslet improve angular sampling but leave fewer independent spatial samples across the scene.
How does computational refocusing work?
Each sub-aperture view is shifted by the disparity predicted for a candidate depth and then integrated. Features on that plane align and reinforce, while features at other depths remain displaced and blur.
Why can an epipolar line break?
Occlusion can hide a point from some views, motion can change the scene during sequential capture, and specular reflection can vary radiance with direction. Calibration or correspondence errors can also disrupt the ideal line.
Exam move
Draw the two-plane coordinates and state which axis represents view and which represents sensor position. Practise converting disparity to depth with compatible units, then explain how uncertainty grows at small disparity. Use focal stacks and epipolar slices together to diagnose sign error, distortion, occlusion and motion. Start refocusing exercises with two views and symmetric shifts before extending to an aperture array.
Keep one known front plane and back plane as sign checks. Fit an epipolar line across several views and mark which observations an occluder removes. Recompute depth after a one-pixel disparity change at near and far ranges. End by listing the evidence needed to distinguish a true depth edge from specular, moving or poorly rectified structure.
Compare focal stacks formed from narrow and wide synthetic apertures and explain the resolution tradeoff.