Image processing - ISET/isetcam GitHub Wiki

What the image processor represents

The image processor (ip) structure turns raw sensor voltages into a displayed image: demosaicking, color balance, illuminant correction, and the transform into a display-referred color space. It is the fourth stage of the ISETCam pipeline, after scene, optical image, and sensor.

The ip is different in kind from the other core structures. Scene, optical image, and sensor are all tied to physics, optics, or hardware; the ip is tied only to software choices. For most ISETCam users, the image processing steps in a real camera are entirely within their own control, and the ip methods exist mainly to organize and clearly label those choices — and to support the image-quality metrics (spatial, color, and noise) that measure the result. See Metrics.

Where the image processor fits in the pipeline

Scene (radiance) --oiCompute--> Optical Image (irradiance) --sensorCompute--> Sensor --ipCompute--> Image Processor --> Display

The key ip routines follow the usual pattern: ipCreate, ipCompute, ipSet/ipGet, ipWindow, and ipPlot.

scene  = sceneCreate;                scene  = sceneSet(scene,'fov',10);
oi     = oiCreate('wvf');            oi     = oiCompute(oi,scene,'crop',true);
sensor = sensorCreate('imx363');     sensor = sensorSet(sensor,'fov',10,oi);
sensor = sensorCompute(sensor,oi);
ip     = ipCreate;
ip     = ipCompute(ip,sensor);
ipWindow(ip);

The key ip parameters appear on the right side of the window, including the names of the methods used at each processing step — for example, the demosaicking method and the sensor-to-internal-color-space conversion method:

ipGet(ip,'demosaic method')          % ans = 'bilinear'
ipGet(ip,'internal color space')     % ans = 'stockman'

See Image processing methods for the specific demosaicking, sensor-conversion, and illuminant-correction methods available and how to select them.

There are also many plotting functions for analyzing image quality, including the ISO 12233 slanted-edge standard, CIELAB, spatial CIELAB, and noise metrics:

ipPlot(ip,'horizontal line',[1 160]);

image

Recommended tutorials and examples

  • Image processing, color, metrics, and displays — t_ip and t_ipDemosaic cover the pipeline directly.
  • Examples — s_ipSRGB demonstrates the sRGB display calculation, the last step before an image reaches the screen.
  • GUI utilities — Macbeth Color Checker (MCC) selection and other window tools frequently used alongside image-processing analysis.

Related FISE concepts

The online Foundations of Image Systems Engineering (FISE) develops demosaicking, color balance, and color rendering as conceptual background.

Boundaries: displays and metrics

Evaluating an ip result meaningfully requires a display model — nobody looks directly at RGB values; they are always displayed, and the eye responds to the resulting spectral radiance. See Display for how ISETCam represents that step, and Metrics for the image-quality measures built on top of the ip output.