Overview - RenderToolbox/RenderToolbox4 GitHub Wiki
RenderToolbox is a set of Matlab tools that facilitates working with 3D models and rendering with physically-based renderers.
A particular focus is on easy manipulation of surface spectral reflectances, surface materials, and illuminant spectral power distributions. One target application for the toolbox is visual psychophysics, where often manipulating these variables is of experimental interest.
Above, the renderer PBRT rendered a dragon scene, 4 times, with the dragon's material as a variable.
Workflow
RenderToolbox prescribes a workflow that includes 3D modeling of a parent scene, batch processing of the scene to produce a family of renderer-native scene files, rendering with a physically-based renderer to produce a family of renderings, and image processing of the renderings to produce viewable images and do other analyses.
The workflow involves a few different applications and files.
3D Modeling
Most scenes start life in a 3D modeling application, which allows the user to define a 3D scene which contains a camera, lights, shapes, and materials. The modeling application also lets the user save a scene file.
Blender is one nice, open-source 3D modeling application.
RenderToolbox uses Assimp to read scene files into memory. Assimp supports a wide variety of input formats and a handful of output formats.
Batch Processing
RenderToolbox has a "batch renderer", which runs in Matlab. The batch renderer can manipulate a scene loaded by Assimp and produce a family of renderer-native scene files, each of which may differ by parameter values. Parameters and values are defined in a conditions file and assigned to parts of the scene. UPenn uses mappings file to coordinate with the conditions; Stanford uses remodeler functions for this purpose. Both are means that enable RenderToolbox4 to modify the PBRT file to align with the conditions file.
Examples of values that can be assigned to the scene are multi-spectral reflectances and illuminant spectra.
Rendering
RenderToolbox supports two different physically-based renderers: PBRT and Mitsuba. Both are open-source. The RenderToolbox batch processing utilities can produce native scene files for either renderer.
RenderToolbox works with sampled spectra, as opposed to RGB colors. This requires special multi-spectral builds of the renderers. We have made such builds available via Docker.
The RenderToolbox rendering utilities produce mat-files that store multi-spectral images in units of radiance. This facilitates comparisons of renderings produced by the two renderers, and other analyses.
It would be possible to add additional renderers as long as they support spectral rendering and produce renderings in a spectral image format.
Image Processing
Multi-spectral image files are rich in information, but difficult to visualize. RenderToolbox includes image processing utilities for converting multi-spectral data to sRGB images and montages, and other "sensor images" that calculated with arbitrary color matching functions. These utilities are largely based on utilities from the Psychophysics Toolbox.
Example: ColorChecker Dragon
A montage of 24 "Color Checker" dragons represents the typical RenderToolbox workflow:
- 3D Modeling The scene was modeled in Blender and saved in a Blender file, which included a camera, lights, a floor, walls, and a 3D dragon model.
- Batch Processing A conditions file specified 24 "Color Checker" colors. A mappings file associated the colors with the dragon's surface material. Batch processing utilities produced a family of 24 renderer-native scene files, one for each color.
- Rendering Each renderer produced 24 multi-spectral data files, one for each color.
- Image Processing For each renderer, the 24 multi-spectral data files were combined into a single sRGB montage.
Above, PBRT rendered a dragon scene, 24 times, with the dragon's color as a variable.
Above, Mitsuba rendered a dragon scene, 24 times, with the dragon's color as a variable.