Optics and Optical Images - ISET/isetcam GitHub Wiki

• Core Pipeline overview
• Optics and optical image tutorials
• Ray trace optics
• Wavefront and PSF representations
• Light field
• Zemax and Code V lens import

ISETCam uses the term optical image (oi) for the light that has passed through the optics and arrives at the sensor surface — the spectral irradiance at that surface. Where scene radiance is measured in photons/sec/sr/nm/m², optical image irradiance is measured in photons/sec/nm/m² (the steradian term drops out because the light has already been collected by the aperture). The oi* functions operate on this structure: oiCreate, oiSet/oiGet, oiCompute, oiWindow, and oiPlot.

Scene (radiance) --oiCompute--> Optical Image (irradiance) --sensorCompute--> Sensor --ipCompute--> Image Processor --> Display

oiCompute takes a scene and an optics model and produces the optical image:

scene = sceneCreate;
oi    = oiCreate('wvf');
oi    = oiCompute(oi,scene);
oiWindow(oi);

Typing oi<TAB> at the MATLAB prompt lists the full set of optical-image functions. The image shown in oiWindow is an sRGB rendering computed from the spectral irradiance (via ieXYZFromPhotons and then xyz2srgb), not the underlying data itself — you can inspect the spectral data directly, for example plotting the irradiance along one row:

oiPlot(oi,'irradiance hline',[1 63]);

The oi structure carries an optics sub-structure that describes the imaging lens:

>> oi.optics

ans =

  struct with fields:

             type: 'optics'
             name: 'wvf'
          offaxis: 'cos4th'
       vignetting: 0
              wvf: [1x1 struct]
            model: 'shiftinvariant'
          fNumber: 4.0000
      focalLength: 0.0039
              OTF: [1x1 struct]
    transmittance: [1x1 struct]
           cos4th: [1x1 struct]

>> oiGet(oi,'optics f number')

ans =

     4

ISETCam includes two optical models, both designed for planar scenes (objects at a roughly uniform distance, or far away). For depth-dependent effects, see Boundaries below.

The default and most commonly used model treats the optics as shift-invariant: the point spread function (PSF) is the same across the visual field, though it varies with wavelength. oiCreate('wvf') (or the default oiCreate) builds this model. The PSF is not stored directly; ISETCam stores the wavefront aberration as Zernike polynomial coefficients (defocus, coma, astigmatism, and so on) and computes the PSF from them on the fly inside oiCompute, at the spatial sampling resolution the current scene requires. See Wavefront and PSF representations for why ISETCam moved to this representation and how the computation works.

Diffraction-limited optics is the important special case of an aberration-free wavefront (all Zernike coefficients zero). ISETCam computes this two ways: by setting the wavefront to all zeros, or with the closed-form diffraction-limited point spread formula (the Airy disk), which depends only on the lens f-number. The two methods agree almost exactly; the closed-form version is retained mainly because it is fast, though it is very slightly less accurate for high-dynamic-range scenes with flare (see isethdrsensor). oiCreate with no arguments, or oiCreate('diffraction limited'), builds this model directly.

The second model allows the PSF to vary with both wavelength and position in the visual field (field height) — a ray trace calculation, together with the lens's geometric distortion. The data for this model are extracted from a lens-design program such as Zemax or Code V; see Zemax and Code V lens import. See Ray trace optics for the calculation itself.

• Optics and optical images — optical image and wavefront tutorials, including t_oiIntroduction and t_wvfOverview.
• Examples — s_opticsMicrolens models the microlens array above a sensor, a practical camera-design calculation that builds on the optical image.

The online Foundations of Image Systems Engineering (FISE) develops optical image formation, diffraction, and wavefront optics as conceptual background.

The models above only describe optical image formation for a planar scene. For three-dimensional effects — depth of field, bokeh, occlusion, complex multi-element lenses — ISETCam's two-dimensional models are not enough. ISET3D takes 3D graphics models and multi-element lens prescriptions as input and uses physically based ray tracing (PBRT) to compute the spectral irradiance at the sensor, correctly handling occlusion and depth of field. It can also place microlenses at the sensor surface to model dual-pixel autofocus and light field cameras — see Light field. When ISET3D returns a result for a real lens model, the result is an ISETCam optical image structure, usable by the rest of the ISETCam pipeline exactly like one computed by oiCompute.