§ The Plenoptic Function

• What can we see because of light?
• Key idea: at each point $(x, y, z)$, we should be able to know, for all wavelenghts $\lambda$, the intensity of the wavelength in all directions $(\theta, \phi)$. Even more generally, this can vary with time $t$.
• Intuition: we should be able to reproduce at all points in spacetime, what happens if one builds a camera!
• This function $P(\theta, \phi, \lambda, t, x, y, z)$ is called as the plenoptic function .
• Notice that when one builds a pinhole camera, what one is doing is to, in fact, use the pencil of rays at that point to capture an image! Thus, the plenoptic function contains all possible pinhole images at all positions.
• The key conjecture of the paper "The plenoptic function and the elements of early vision" is that the visual cortex is extracting local changes / derivatives of the plenoptic function.

§ Crash Course Radiometry

• Irradiance at a point: density of radiant flux (power) per unit surface area.
• Radiance at a point in a direction: density of radiant flux (power) per unit surface area per unit solid angle.

§ Light field rendering

• See that if we restrict to only radiance of light at a fixed time $t_0$, then we have $(x, y, z, \theta, \phi)$, a 5 dimensional function.
• Also note that if there is no obstruction, then the radiance does not change along lines. So we can quotient $(x, y, z)$ to get a lower dimensional 4D field, given by $(\texttt{pos}_\theta, \texttt{pos}_\phi, \texttt{look}_\theta, \texttt{look}_phi)$.
• This 4D field is called as a light field.
• Alternatively, we can parametrize these by $(x_1, y_1)$ and $(x_2, y_2)$, and the paper canonically calls these as $(u, v, s, t)$. This coordinate system they call a light slab , and represents light starting from the point $(u, v)$ at the first plane and ending at $(s, t)$ at the second plane.