Scene radiance - ISET/isetcam GitHub Wiki

• Core Pipeline overview
• Scene tutorials
• Scene test targets
• Scene spectral basis representations

The ISETCam scene structure describes the spectral radiance of the world in front of the camera: how much light of each wavelength leaves each point in the scene, in each direction toward the lens. It is the first structure in the pipeline — nothing is computed to produce it from an earlier ISETCam stage; you create it directly, either synthetically or by reading data from a file. The scene is simplified in one important way: each point is assumed to emit light uniformly in every direction that reaches the camera aperture, so the full seven-parameter light field collapses to a single spectral radiance value at every spatial position. (For the complete, direction-dependent description, see Light field.)

Radiance is stored in scene.data.photons in units of photons/sec/sr/nm/m². The scene structure also carries:

• Depth: scene.depthMap, the distance from the observer to each point (or scene.distance for a single planar distance).
• Illumination: the spectral radiance of the light source, either a single spectral vector (uniform illumination) or a space-varying illuminant map.

Scene (radiance) --oiCompute--> Optical Image (irradiance) --sensorCompute--> Sensor --ipCompute--> Image Processor --> Display

The scene is the pipeline's starting point. It has no sceneCompute counterpart to oiCompute/sensorCompute/ipCompute — you create or read a scene, and the next stage, optical image formation, computes the irradiance an optical system would produce from it.

Like other ISETCam structures, the scene has the standard set of methods:

• sceneCreate — produces many different types of scenes for testing
• sceneSet / sceneGet — set and get scene parameters (field of view, mean luminance, distance, illuminant, ...)
• scenePlot — plot scene radiance, illuminant, and derived quantities
• sceneWindow — open the interactive scene GUI, which can hold a collection of scenes

scene = sceneCreate;                 % Default Macbeth Color Checker scene
scene = sceneCreate('slanted bar');  % A named test target
scene = sceneSet(scene,'fov',10);    % Set the horizontal field of view (deg)
fov   = sceneGet(scene,'fov');       % Read it back
scenePlot(scene,'illuminant photons');
sceneWindow(scene);

Typing scene<TAB> at the MATLAB prompt lists the full set of scene functions. See Scene test targets for a gallery of the built-in test images sceneCreate can produce.

In addition to the synthetic targets from sceneCreate, sceneFromFile reads an ordinary image and returns an estimated scene radiance structure. This lets you test camera features using almost any photograph as simulation input.

sceneFromFile treats the RGB (or multispectral) data as if it were shown on a calibrated display, and treats the display's white point as the scene illuminant. By default it uses a theoretical display (data/displays/reflectance-display.mat) calibrated so that RGB values are rendered under a D65 illuminant using the first three principal components of naturally occurring surface reflectances; examples/display/s_displaySurfaceReflectance.m explains how that calibration was built. Because the true scene depth and illuminant are unknown, scenes built this way are a useful approximation, not a physically precise reconstruction.

d = displayCreate('LCD-Apple');
scene = sceneFromFile('eagle.jpg','rgb',100,d);   % 100 cd/m^2 mean luminance
sceneWindow(scene);

The resulting radiance depends on which display model you assume — the same file rendered through two different displays produces two different estimated scenes:

d = displayCreate('LCD-Apple');
scene = sceneFromFile('eagle.jpg','rgb',100,d);
rgb1 = sceneGet(scene,'rgb'); imshow(rgb1);

d = displayCreate('CRT-Dell');
scene = sceneFromFile('eagle.jpg','rgb',100,d);
rgb2 = sceneGet(scene,'rgb'); imshow(rgb2);

sceneFromFile can also read multispectral and hyperspectral data files saved in ISETCam format. A few are included under data/images/multispectral; many more are available for download with ieWebGet — see Spectral Scene Data. Large spectral files are often stored as a compressed linear-model representation (coefficients plus a small set of basis functions) rather than as full wavelength-by-wavelength data; see Scene spectral basis representations for how that compression works and how sceneFromBasis reconstructs the radiance.

• Scenes and light — the scene-object tutorials, including t_sceneIntroduction.
• Examples — s_sceneFromMultispectral reads multispectral and hyperspectral scene files with sceneFromFile.

The online Foundations of Image Systems Engineering (FISE) develops scene radiance, radiometry, and light measurement in more depth as conceptual background.

Everything above describes a planar scene: every point emits uniformly toward the aperture, as if all objects were far away or arranged on a flat surface. For three-dimensional effects — occlusion, depth of field and bokeh, multi-element lenses, or a complete light field — ISETCam scene computations are not enough. ISET3D uses physically based ray tracing (PBRT) on 3D graphics models to compute these effects; when it uses a pinhole camera model with diffraction turned off, ISET3D returns an ordinary ISETCam scene structure. With a real lens model, it returns an ISETCam optical image instead — see Optics and Optical Images.