Core Pipeline - ISET/isetcam GitHub Wiki

Core Pipeline

ISETCam's main computations follow a physical and computational pipeline, from scene radiance to a rendered display image, with every intermediate quantity kept in real physical units:

Scene (radiance) --oiCompute--> Optical Image (irradiance) --sensorCompute--> Sensor --ipCompute--> Image Processor --> Display

Each stage below has a consistent page shape: what the representation means physically (with units), where it sits in the pipeline, the basic *Create/*Get/*Set/*Compute/*Plot/*Window calls, a short current code example, and links to the relevant tutorials, examples, and FISE background.

Stage Page Covers
1. Scene Scene Radiance Spectral radiance, illumination, depth; creating scenes with sceneCreate and sceneFromFile.
2. Optics Optics and Optical Images Spectral irradiance at the sensor; shift-invariant (wavefront/diffraction-limited) and ray-trace optical models.
3. Sensor Sensor The photodetector array, color filter array, pixel electronics, and noise that produce pixel voltages.
4. Image Processing Image Processing Demosaicking, color balance, and illuminant correction that turn sensor voltages into a rendered image.
5. Display Display The calibrated display model used to view — and, run in reverse, to help create — a scene.

If you haven't already, Getting Started walks through a five-line script that runs all five stages once. For a hands-on tour instead of the conceptual pages above, see Tutorials and Examples.

Supporting concepts

Color science, image-quality metrics, and full camera-system simulation build on top of this core pipeline rather than being a stage within it:

  • Metrics — image-quality measures (spatial, color, and noise) computed from the ip output.

Boundaries

The core pipeline models a planar scene under a shift-invariant or ray-trace 2D optical model. For three-dimensional effects — depth of field, bokeh, occlusion, complex multi-element lenses, and light field cameras — see ISET3D, which is required on the MATLAB path alongside ISETCam for that work. For human visual encoding (cone mosaics, eye movements, retinal responses) built on top of this same pipeline, see ISETBio.