Guide: Using RenoDX to Upgrade HDR in Games - clshortfuse/renodx GitHub Wiki

1. Introduction to RenoDX and HDR

RenoDX is a powerful tool designed to enhance HDR (High Dynamic Range) rendering in games, enabling improvements even for titles originally developed for SDR (Standard Dynamic Range). HDR delivers a richer visual experience by expanding the color gamut and increasing luminance ranges, resulting in more vibrant colors and realistic brightness.

This guide provides a technical breakdown of how RenoDX upgrades HDR using swapchain proxies, resource upgrades, and shader adjustments.

2. Understanding the Limitations of RGBA8U Swapchains

Games that use RGBA8U swapchains (8 bits per channel, unsigned) face constraints that hinder HDR implementation:

  1. Color Space Restriction: RGBA8U confines values to the 0-1 range, limiting the rendering pipeline to the BT709 color space. Without support for negative values, out-of-gamut colors in wider gamuts like BT2020 cannot be expressed.

  2. Luminance Cap: Luminance levels are capped at 1.0, which corresponds to SDR reference white. This prevents the display of HDR brightness levels required for highlights, specular reflections, and other high-luminance details.

3. Transitioning to Floating-Point Formats

To overcome these limitations, the rendering pipeline can adopt floating-point formats, such as RGBA16F. These formats enable:

  • Expanded Color Gamut: Floating-point formats support negative values, making it possible to represent BT2020 colors that are outside the BT709 gamut.
  • Higher Luminance Levels: Values exceeding 1.0 allow for accurate rendering of HDR highlights far beyond SDR's brightness range.

However, using a floating-point swapchain isn’t always necessary, as explained below.

4. Using Proxy Resources with HDR Swapchains

HDR upgrades can be achieved by introducing a proxy resource while retaining a practical swapchain format. The process works as follows:

  • Intermediate or Proxy Resource: Rendering occurs in a high-precision RGBA16F texture or buffer, enabling the use of floating-point precision for luminance and color gamut expansion.

  • Swapchain Format Options: The final output is tone-mapped and encoded to a format like RGB10A2, suitable for HDR10 with PQ (Perceptual Quantizer) encoding, or linear scRGB for displays supporting that standard.

This approach allows floating-point operations without requiring a full floating-point swapchain, balancing performance and compatibility.

5. Impact of Moving the Texture Pipeline to Floating-Point

When the texture pipeline uses floating-point formats, the rendering operations gain critical advantages:

  1. Values Beyond 1.0:
    All rendering stages can output values greater than 1.0, preserving highlight details that would otherwise be clipped in SDR pipelines.

  2. Detail Preservation:
    Render data remains in floating-point representation throughout the pipeline. Even after tonemapping, values above 1.0 (representing higher-than-SDR luminance) are preserved until the final display output.

  3. Support for Negative Values (BT2020):
    Floating-point formats allow for negative values, which represent colors outside of the BT709 color gamut. This enables HDR rendering to incorporate BT2020 colors, while the majority of the render pipeline can remain in the original BT709 rendering color space defined by the game.

  4. Retention of Gamma Space:
    The floating-point pipeline ensures that most rendering operations can continue to function in gamma space, minimizing the need for extensive changes to the game's native pipeline and preserving compatibility with existing systems.

  5. Reduced Banding:
    Transitioning from 8-bit to 16-bit floating-point formats increases precision, significantly reducing visual banding in gradients and subtle color transitions. This enhances the quality of rendered visuals, especially in scenes with smooth lighting or color variations.

  6. Intermediate Precision:
    The floating-point environment acts as a working space for all rendering and post-processing steps before conversion to the final display format.

6. Tonemapping SDR Games for HDR

Tonemapping plays a critical role in adapting SDR games to HDR displays:

  1. User-Defined Peak Nits: Instead of tonemapping to SDR's reference white (1.0), shaders are modified to target peak luminance values based on user settings. This allows for accurate scaling of bright highlights.

  2. Selective Gamma Encoding: Modified shaders tonemap to HDR values and finalize in gamma space, enabling seamless blending with unmodified components like UI elements, which continue rendering in traditional gamma space.

  3. Utilizing Perceptual Color Spaces:
    By leveraging perceptual color spaces like BT2020 instead of limiting tonemapping to BT709, it becomes possible to achieve faithful, high dynamic range tonemapping. This approach expands the color gamut and utilizes wider dynamic ranges to represent varying levels of contrast and luminance accurately.

    • The goal is not to overcolor or oversaturate the original image but to deliver more perceptually accurate results. This ensures natural rendering of colors and brightness levels, enhancing the game's visual fidelity without altering its artistic intent.

  4. Final Format Conversion: Before the intermediate or proxy texture writes to the swapchain, gamma-encoded values are converted to the proper format for output:

    • Linear scRGB for wide gamut linear HDR.
    • PQ HDR10 for HDR10 displays using the Perceptual Quantizer curve.

7. Swapchain Proxy Pixel Shader for Format Conversion

The swapchain proxy uses a dedicated pixel shader to handle the conversion from the floating-point gamma representation to the final swapchain format. This shader performs several key roles:

  1. Gamma Representation Mapping:
    The proxy pixel shader processes floating-point gamma values, ensuring proper handling of HDR luminance and color gamut before encoding into the swapchain format.

  2. Handling Gamma Mismatches:

    • Any gamma mismatches within the pipeline, such as sRGB (standard gamma curve) and Gamma 2.2, are addressed here.
    • The shader adjusts mismatched gamma values, ensuring consistent output and preventing unintended luminance values caused by discrepancies in gamma encoding.

  3. Output Format Encoding:
    After handling gamma mismatches, the shader encodes the processed data into the appropriate HDR format, such as RGB10A2 for HDR10 or linear scRGB, depending on the target display.

8. Independent Tonemapping and Scaling for Game Render and UI

A crucial aspect of adapting SDR games for HDR is managing how tonemapping and scaling are applied to the game render and UI independently. This separation provides the following key benefits:

  1. Isolated Tonemapping for Game Render:
    Tonemapping is applied exclusively to the game render early in the pipeline, ensuring that color grading and luminance adjustments do not affect UI elements. This preserves the intended visual consistency of the UI.

  2. Independent Scaling and Compositing:
    The game render and UI can be scaled independently before being composited. This allows precise control over their individual brightness and ensures the combined image transitions smoothly to the swapchain proxy for final HDR format conversion (e.g., linear scRGB or PQ HDR10).

9. Extra Considerations: Clipped or Tonemapped Game Renders

In certain cases, the game render may already be clipped or tonemapped by the time it is passed to the swapchain. This can limit the data's suitability for HDR output, as highlight details and extended luminance ranges may have been altered. To address this, you must ensure that the render is not prematurely clipped or tonemapped before being passed to the swapchain proxy. Here’s how:

  1. Avoiding Premature Clipping or Tonemapping:

    • Traverse through shaders and textures in reverse rendering order to identify and adjust any points where clipping or tonemapping might have been applied prematurely.
    • Ensure that resource formats and shaders preserve full luminance and color range for HDR-critical data.

  2. Maintaining Precision Throughout the Pipeline:

    • Use high-precision resources (e.g., RGBA16F textures) to carry the render data through all stages of the pipeline, preventing luminance or color information loss.
    • Ensure high-precision data persists until the swapchain proxy shader processes it for final HDR formatting.

  3. Addressing Specific Resource or Shader Limitations:

    • Limited Resource Formats: Upgrade resource formats (e.g., from 8-bit to 16-bit floating-point) to preserve extended luminance and color range.
    • Shader Modifications: Modify or replace shader code to avoid premature tonemapping or to retain full luminance information until the proxy stage.

10. Code Implementation

To enable the swapchain proxy in RenoDX, specific configurations and shaders must be added to your project. For detailed code examples and explanations, visit:
[Link Placeholder for Code Explanations]

This will guide you through:

  • Configuring the addon.cpp file.
  • Vertex shader implementation for full-screen rendering.
  • Pixel shader functionality for processing and encoding HDR values.
  • Options related to tonemapping configurations, including perceptual color space adjustments for accurate HDR rendering.
  • Fine-tuning the swapchain proxy configuration, such as resource cloning and shader adjustments to ensure optimal HDR output.