MECH5720 Chap.4 Multiplexing and Computational Relighting
Multiplexing and Computational Relighting
Multiplexing combines several source contributions into coded captures and separates them by solving a linear system. The coding matrix describes which sources contribute to each exposure. Rank establishes whether separation is possible, while singular values and condition number predict noise amplification. Computational relighting then recombines recovered source images under new weights.
The method can improve efficiency in some noise regimes, but only if calibration, linear response, code timing and scene stability support the algebra. Each row of a coding matrix describes one capture and each column describes one source contribution. A one indicates that a source participates under a binary code, while signed or weighted codes use other coefficients. The ordering in software must match the physical sequence.
Swapping two capture files or two source labels applies a mathematically valid inverse to the wrong system and creates structured cross-talk. Sufficient rank is the first requirement for separation. If columns are dependent, distinct source combinations make identical data. Full rank provides a unique noiseless answer but does not guarantee stability.
Unequal singular values mean that some source combinations are measured weakly, so inversion enlarges sensor noise, background error and timing drift along those directions. Orthogonal or well-separated codes reduce that amplification. A useful acquisition test activates one source at a time.
The recovered vector should concentrate in the expected channel, and applying the forward matrix to the result should reproduce the recorded sequence. Repeating at several intensity levels checks whether illumination, camera response and exposure remain linear. Dark and ambient measurements identify contributions that the coding matrix did not include. Multiplex advantage depends on noise statistics.
Combining sources can use exposure time efficiently when additive read noise dominates, but photon noise grows with the admitted flux and can reduce the expected gain. Saturation caps useful brightness, while scene motion violates the assumption that one stable set of source images generated every coded frame. The operating regime must accompany any signal-to-noise claim. Computational relighting follows after separation.
New source weights are applied to the recovered impulse images and summed, allowing illumination direction or balance to change without another capture. Negative or extreme weights may amplify recovery errors and produce a result outside physically captured lighting. Preserve source scale and colour calibration when a photometric interpretation matters.
Validate a relit image by reconstructing a lighting combination that was also captured directly. Agreement tests the full chain of code, inverse and recombination. Disagreement should be localised with residuals, single-source trials and repeat sequences before the matrix design is blamed. A sound conclusion specifies which noise regime, motion condition and intensity range support the chosen multiplex strategy.
What this chapter covers
- 01
Linear coding matrices
- 02
Rank and invertibility
- 03
Conditioning and noise
- 04
Hadamard-style codes
- 05
Demultiplexing
- 06
Computational relighting
Worked example · free
Demultiplex two source responses
- +1Add the equations to obtain 2x1 = 14.
- +1Compute x1 = 7.
- +1Subtract the second equation from the first to obtain 2x2 = 6.
- +1Compute x2 = 3 and verify both original measurements.
Key terms
- Multiplexing
- Combining several signals into coded measurements for later separation.
- Coding matrix
- The linear map from individual source contributions to captures.
- Matrix rank
- The number of independent measurement directions in a matrix.
- Demultiplexing
- Recovering individual contributions from coded measurements.
- Relighting
- Synthesising illumination by recombining recovered source images.
Multiplexing and Computational Relighting FAQ
Is an invertible code automatically a good code?
No. An invertible matrix can still have nearly dependent rows or columns, causing strong noise amplification. Inspect singular values or condition number and test recovery under expected noise and calibration errors.
When can multiplexing lose its advantage?
Advantages depend on the dominant noise regime and stable linear measurements. Shot noise, saturation, timing drift, scene motion and code miscalibration can remove the expected gain or create cross-talk between recovered sources.
How is a relit image produced?
After recovering the image associated with each source, assign desired source weights and add the weighted images. The result assumes linear response and that the scene did not change across the coded capture sequence.
Exam move
Translate every coding description into y = Mx + n. Check dimensions, rank and code order before calculating an inverse. Practise small systems by elimination and matrix form, then predict how one noisy measurement or moving object appears after demultiplexing. Finish by verifying the recovered sources through the forward model.
Compare an orthogonal code with two nearly repeated measurements and explain the expected difference before computing it. Keep a table separating rank failure, poor conditioning, saturation, motion and source-order error. For each symptom, nominate a single-source or repeat-capture test. Relight one recovered vector with several weight choices and check which combinations remain within calibrated intensity.
Always reconstruct the original coded captures as a final algebra and implementation check. State the assumed noise regime before arguing that a multiplex code improves measurement efficiency.