ISETCam

The Image Systems Engineering Toolbox for Cameras models complete imaging systems in physical units. By insisting on keeping track of the units, the software connects the absolute light levels in a scene to the physical properties of optics and sensors, through digital computations, and to a final displayed image or to the digital values used by an AI. By sticking with the true physical parameters you can analyze not only whether a system works, but you can also trace the success or failure to specific system components.

New to ISETCam? Start with Getting Started.

Why image systems engineering?

Cameras bring together light, lenses, electronics, algorithms, perception, and inference. That combination matters in consumer imaging, medicine, robotics, automotive systems, and scientific measurement. ISETCam provides a place to make the assumptions explicit, compute their consequences, and learn how the parts of an image system interact.

What can ISETCam compute?

ISETCam models the physical and computational pipeline of an imaging system. Scene spectral radiance data can be measured directly or data can be synthesized from 3D graphics models in ISET3D, and then analyzed through optics, sensor capture, image processing algorithms, and display simulation in ISETCam.

Explore the pipeline stages: scene radiance, optics and optical images, sensors, image processing, and displays.

Device and spectral scene data

ISETCam includes an extensive library of device models and calibration data in its data/ directory, including lenses, optical prescriptions, image sensor color filter arrays (CFAs), display spectral emissions, illuminant spectra, and test charts.

As part of its broader computational ecosystem, ISETCam also provides access to a large collection of spectral scene radiance data. These datasets encompass:

• Measured scene data: Hyperspectral, multispectral, and high dynamic range (HDR) photographic measurements of natural landscapes, objects, materials, and human faces.
• Simulated scene data: Calibrated spectral scenes rendered from 3D computer graphics models using physically based ray tracing (PBRT).

Built-in utilities (ieWebGet and sceneSDR) automatically discover, download, and cache these datasets from curated Stanford Digital Repository (SDR) deposits. The datasets in ISETCam are part of a larger, shared data ecosystem that connects with 3D computer graphics scenes and assets in ISET3D and retinal mosaic models in ISETBio.

Explore the available collections, formats, and download tools in Spectral Scene Data.

Real units matter

ISETCam keeps the quantities that connect a camera to the world—spectral scene radiance, optical-image irradiance, and sensor photoelectrons and voltages—in physical units. This makes it possible to reason about changes in illumination, scene distance, field of view, aperture, exposure, pixel design, and processing choices. By comparing simulations with lab measurements, you can connect a camera model to a real device and real device components.

Simulations that capture the physical device parameters, usually called a 'digital twin', are particularly helpful for system design and component selection.

Learn the ideas and use the software

The evolving online Foundations of Image Systems Engineering (FISE) resource is a conceptual companion to ISETCam. Use it to develop the scientific picture, then use this wiki's tutorials and examples to run and adapt the software. FISE is a work in progress; the ISETCam source repository is the authority for current runnable workflows.

A foundation for the ISET ecosystem

ISETCam is a required MATLAB-path dependency for both ISET3D, which adds physics-based 3D rendering, and ISETBio, which adds models of human visual encoding. General scene, optical-image, sensor, image-processing, display, color, and metric guidance belongs here; the specialized 3D and biological workflows belong in those projects.

Open, reproducible image science

ISETCam is open source so that image-system results can be inspected, reproduced, taught, and extended. It grew from ImageVal work and Stanford teaching; read History for that story, contributors, and detailed citation guidance.

If ISETCam contributes to published work, cite the repository and an appropriate foundational or validation paper:

• J. E. Farrell, P. B. Catrysse, B. A. Wandell (2012). “Digital camera simulation”, Applied Optics, Vol. 51, Iss. 4, pp. A80–A90.
• H. Lyu, J. Goossens, B. A. Wandell, and J. E. Farrell (2022). “Validation of Physics-Based Image Systems Simulation With 3-D Scenes”, IEEE Sensors Journal, Vol. 22, No. 20, pp. 19287–19297.

See History for additional publications and historical context.