Enhancing‐Tone‐Mapping‐Fidelity - ApertureViewer/Aperture-Viewer GitHub Wiki

Abstract

This document details a proposed enhancement to the postDeferredTonemap.glsl fragment shader within the Second Life Viewer codebase. The current implementation of the tonemap_mix uniform, intended to allow users to blend between the fully tone-mapped High Dynamic Range (HDR) image and a linear representation, introduces a subtle artifact by prematurely clamping HDR data before blending. This results in a loss of dynamic range and visually incorrect transitions when the mix factor is less than 1.0. The proposed solution modifies the blending logic to operate correctly in linear space, mixing the tone-mapped result with the properly exposed, unclamped linear input color before the final clamp to Low Dynamic Range (LDR). This refinement preserves HDR information during the blend, yields a visually smoother and more intuitive transition, and increases the overall fidelity of the tone mapping stage.

1. Introduction and Background

Modern real-time rendering pipelines, such as that employed by the Second Life Viewer, increasingly utilize High Dynamic Range (HDR) lighting and rendering. This allows for light intensities far exceeding the typical [0, 1] range representable by standard displays, capturing greater realism in highlights and overall scene illumination.

A critical component of an HDR pipeline is Tone Mapping, the process of converting the scene-referred HDR color values into the Low Dynamic Range (LDR) suitable for display, typically [0, 1]. The postDeferredTonemap.glsl shader performs this crucial step, incorporating exposure controls (manual exposure uniform and automatic via exposureMap) and offering multiple tone mapping operators (PBRNeutralToneMapping, toneMapACES_Hill).

To provide artistic control and flexibility, the shader includes a tonemap_mix uniform. The apparent intent of this parameter is to allow users to linearly interpolate between the result of the chosen tone mapping operator (mapping HDR to LDR) and a non-tone-mapped, linear representation of the scene, effectively controlling the "strength" of the tone mapping effect.

2. Problem Analysis: Premature Clamping in Blend Operation

The current implementation of the blending logic within the toneMap function (and similarly in toneMapNoExposure) effectively performs the following steps:

// Original logic within toneMap function (simplified conceptual flow)
vec3 color = // ... initial HDR color ...
float exp_scale = texture(exposureMap, vec2(0.5,0.5)).r;
color *= exposure * exp_scale; // Apply exposure

// PROBLEM: HDR color is clamped BEFORE being used in the mix source ('a')
vec3 clamped_color = clamp(color.rgb, vec3(0.0), vec3(1.0));

vec3 tonemapped_color = color; // Initialize
switch(tonemap_type)
{
    // ... apply selected tonemapper to 'color', result in 'tonemapped_color' ...
}

// INCORRECT MIX SOURCE: Mixes tonemapped result with the prematurely clamped color
color = mix(clamped_color, tonemapped_color, tonemap_mix);

// Final clamp (redundant if mix=0, potentially hides issues otherwise)
color = clamp(color, 0.0, 1.0);

The core issue lies in the calculation and use of clamped_color. By clamping the exposed HDR color (color) to the [0, 1] range before it is used as the starting point (a) in the mix(a, b, mix_factor) operation, all high dynamic range information (values > 1.0) present in the scene's highlights is discarded whenever tonemap_mix is less than 1.0.

The linear interpolation formula is output = a * (1.0 - mix_factor) + b * mix_factor. When tonemap_mix < 1.0:

  • b is the correctly tone-mapped LDR color.
  • a should represent the linear HDR color, appropriately scaled by exposure, corresponding to the "zero effect" state.
  • Instead, a is the clamped LDR color.

This leads to several undesirable outcomes:

  • Loss of HDR Detail: As the user decreases tonemap_mix from 1.0, instead of smoothly blending back towards the original scene appearance (with visible detail in bright highlights), the blend incorporates clipped highlights from clamped_color.
  • Incorrect Visual Transition: The transition does not accurately represent a linear blend in perceived intensity or appearance. Highlights may abruptly "pop" back to clipped white instead of smoothly re-emerging.
  • Counter-Intuitive Control: The slider does not behave as a simple intensity control for the tone mapping curve; it actively changes the perceived dynamic range in a non-linear, potentially artifact-prone way.

3. Proposed Solution: Linear Space Blending Prior to Final Clamp

The proposed solution ensures that the blending operation occurs between two correctly represented states before the final clamp to the LDR display range. The blend should occur between the fully tone-mapped LDR result and the appropriately scaled, unclamped linear HDR input.

Implementation

The modification involves storing the original linear color, applying exposure consistently, performing the tone mapping, and then mixing the tone-mapped result with the exposed linear input before the final clamp.

Revised toneMap Function:

vec3 toneMap(vec3 color)
{
#ifndef NO_POST
    // Store original linear color BEFORE exposure application
    vec3 linear_input_color = color;

    // Calculate exposure scale
    float exp_scale = texture(exposureMap, vec2(0.5,0.5)).r;
    float final_exposure = exposure * exp_scale;

    // Apply exposure to a working variable
    vec3 exposed_color = color * final_exposure;

    // Apply the selected tone mapping operator to the exposed color
    vec3 tonemapped_color = exposed_color; // Initialize
    switch(tonemap_type)
    {
    case 0:
        tonemapped_color = PBRNeutralToneMapping(exposed_color);
        break;
    case 1:
        tonemapped_color = toneMapACES_Hill(exposed_color);
        break;
        // Consider adding other cases if needed (e.g., Narkowicz)
    }

    // --- CORRECTED BLENDING LOGIC ---
    // Calculate the correctly exposed linear color for mixing target 'a'
    vec3 exposed_linear_input = linear_input_color * final_exposure;

    // Mix between the exposed linear input ('a') and the tonemapped result ('b')
    color = mix(exposed_linear_input, tonemapped_color, tonemap_mix);
    // --- END CORRECTION ---

    // Final clamp to LDR for display AFTER mixing
    color = clamp(color, 0.0, 1.0);
#else
    // If NO_POST, apply exposure and clamp (consistent behavior)
    color *= exposure * texture(exposureMap, vec2(0.5,0.5)).r;
    color = clamp(color, 0.0, 1.0);
#endif

    return color;
}

Revised toneMapNoExposure Function:

vec3 toneMapNoExposure(vec3 color)
{
#ifndef NO_POST
    vec3 linear_input_color = color; // Store original linear color
    vec3 tonemapped_color = color;   // Initialize

    // Apply the selected tone mapping operator to the original color
    switch(tonemap_type)
    {
    case 0:
        tonemapped_color = PBRNeutralToneMapping(color);
        break;
    case 1:
        tonemapped_color = toneMapACES_Hill(color);
        break;
    }

    // --- CORRECTED BLENDING LOGIC ---
    // Mix between the ORIGINAL linear input ('a') and the tonemapped result ('b')
    color = mix(linear_input_color, tonemapped_color, tonemap_mix);
    // --- END CORRECTION ---

    // Final clamp AFTER mixing
    color = clamp(color, 0.0, 1.0);
#else
     // Simple clamp if no post
     color = clamp(color, 0.0, 1.0);
#endif
    return color;
}

4. Code Comparison Snippets

The essential change within the toneMap function's mix logic:

Original toneMap Mix Logic:

// ... exposure applied to 'color' ...
vec3 clamped_color = clamp(color.rgb, vec3(0.0), vec3(1.0)); // HDR clipped here
// ... tonemapping applied, result in 'tonemapped_color' ...
color = mix(clamped_color /* Incorrect 'a' */, tonemapped_color, tonemap_mix);
// ... final clamp ...

Improved toneMap Mix Logic:

// ... exposure calculated and stored in 'final_exposure' ...
// ... original linear color stored in 'linear_input_color' ...
// ... tonemapping applied to exposed color, result in 'tonemapped_color' ...
vec3 exposed_linear_input = linear_input_color * final_exposure; // Correct 'a' target
color = mix(exposed_linear_input /* Correct 'a' */, tonemapped_color, tonemap_mix);
// ... final clamp ...

(A similar principled change applies to toneMapNoExposure)

5. Visual Comparison (Illustrative)

The visual impact of this change is most apparent when tonemap_mix is set to values between 0.0 and 1.0 in scenes with significant dynamic range (bright highlights alongside darker areas).

(Insert images here using standard Markdown syntax)

Image 1: Legacy Shader (tonemap_mix = 0.7) Snapshot_032

Caption: Scene rendered with legacy postDeferredTonemap.glsl (tonemap_mix = 0.7). Observe potential loss of detail or abrupt clipping in highlight areas due to premature clamping before the mix.

Image 2: Improved Shader (tonemap_mix = 0.7) Snapshot_033 Caption: Same scene rendered with improved postDeferredTonemap.glsl (tonemap_mix = 0.7). Highlight areas should exhibit smoother transitions and better preservation of detail compared to the legacy version, demonstrating the benefit of correct linear space blending.

6. Benefits of the Proposed Enhancement

Implementing this change yields several significant advantages:

  1. Preservation of Dynamic Range: The blend operation no longer prematurely discards HDR information. Details in highlights are correctly preserved and blended when tonemap_mix < 1.0.
  2. Accurate Linear Interpolation: The mix occurs between two appropriately scaled values – the exposed linear input and the final tone-mapped output – resulting in a mathematically and perceptually correct interpolation.
  3. Smoother Visual Transition: Reducing tonemap_mix provides a smooth visual fade of the tone mapping effect towards the linear scene representation, without the jarring reintroduction of clipped highlights.
  4. Intuitive User Control: The tonemap_mix slider behaves more predictably as an intensity control for the tone mapping curve, directly affecting the mapping without introducing unrelated clipping artifacts during the blend.
  5. Increased Fidelity: The overall visual fidelity and correctness of the HDR rendering pipeline are improved by ensuring operations occur in the appropriate color/luminance space.
  6. Code Elegance & Correctness: The revised logic represents a more mathematically sound approach to blending in an HDR pipeline.

7. Implementation Considerations

  • Performance: The computational cost of this change is negligible. It primarily involves storing one additional vec3 and performing one vec3 multiplication before the mix operation.
  • Self-Contained: This improvement is localized to the tone mapping shader (postDeferredTonemap.glsl) and does not necessitate changes in other parts of the rendering pipeline or C++ codebase.
  • Compatibility: It corrects the behavior of the existing tonemap_mix uniform without changing its intended purpose or requiring UI modifications. Existing user settings for tonemap_mix will simply produce a more accurate visual result.

8. Conclusion

The proposed modification to the blending logic within postDeferredTonemap.glsl represents a targeted enhancement that significantly improves the visual fidelity and user experience related to the tonemap_mix parameter. By ensuring the blend operation occurs between the correctly exposed linear input and the tone-mapped output before the final LDR clamp, this change preserves High Dynamic Range detail during interpolation, provides a smoother visual transition, and results in more intuitive control over the tone mapping effect strength. This refinement aligns the implementation more closely with the best practices for HDR rendering pipelines, enhances the quality of the final rendered image, and demonstrates a commitment to technical accuracy and visual excellence. We recommend its implementation.