Gamma correction in X3D and glTF - michaliskambi/x3d-tests GitHub Wiki

What is gamma correction

Nice summary:

https://blog.molecular-matters.com/2011/11/21/gamma-correct-rendering/

Short version: To be correct (this is called "linear workflow"):

  1. you should do "pow(x, 2.2)" when reading colors from inputs (like textures)
  2. you should do "pow(x, 1 / 2.2)" as last step before storing pixel color in color buffer

AD 1 - Note that it can be avoided if you know that the input is already provided in proper space for "linear workflow" (i.e. they represent real physical values, not caring how would they display on a typical monitor). Some software (Blender) has per-texture settings to tell "this texture is already in linear space, no need to convert it".

AD 1 - In theory, you could also do pow(x,2.2) for inputs from explicit vector parameters like X3D Material.diffuseColor or glTF baseColorFactor (which will be PhysicalMaterial.baseColor in X3Dv4). It depends on whether you expect these values to be specified in "sRGB color space" or in "linear color space". In practice, both X3DOM and glTF sample implementation assume that such explicit parameters (like X3D Material.diffuseColor or glTF baseColorFactor) are already in "linear space", so they don't apply pow(x,2.2) on them.

AD 1 - Of course this operation only applies to RGB components of colors (because they are physically-incorrect in most images, to display properly on typical monitors). Alpha should be left unmodified.

AD 1 and 2 - There are OpenGL extensions to avoid doing these operations (texture formats, FBO formats), by declaring texture/buffer format as SRGB. See https://en.wikipedia.org/wiki/SRGB for links to OpenGL extensions.

More info:

Note that tone mapping is often related, but it has a different purpose: "gamma correction" is about making lighting calculations correct, while "tone mapping" is about enhancing the result in a visually-pleasing way. See https://en.wikipedia.org/wiki/Tone_mapping .

Pseudo-code

This is lighting calculation (Phong lighting model) without gamma correction:

main()
{
  md = Material.diffuseColor * texture2D(Material.diffuseTexture);
  ms = Material.specularColor * texture2D(Material.specularTexture); // specularTexture is new in X3Dv4
  me = Material.emissiveColor * texture2D(Material.emissiveTexture); // emissiveTexture is new in X3Dv4
  result = me;
  for i = 0 to lights.count - 1 do 
  {
    result += md * lights[i].color * diffuseFunction(normal, point - light[i].position);
    result += ms * lights[i].color * specularFunction(normal, point - light[i].position, point - camera);
  }
  gl_FragColor = result;
}

Below is lighting calculation (Phong lighting model) with gamma correction, assuming that textures need the sRGB transfer (pow(x,2.2)) applied. As far as I understand, this is what X3DOM does for gamma="linear". As you can see, the modifications boil down to adding appropriate pow calls at the very beginning and very end of the calculations.

main()
{
  md = Material.diffuseColor * pow(texture2D(Material.diffuseTexture), 2.2);
  ms = Material.specularColor * pow(texture2D(Material.specularTexture), 2.2);
  me = Material.emissiveColor * pow(texture2D(Material.emissiveTexture), 2.2);
  result = me;
  for i = 0 to lights.count - 1 do 
  {
    result += md * lights[i].color * diffuseFunction(normal, point - light[i].position);
    result += ms * lights[i].color * specularFunction(normal, point - light[i].position, point - camera);
  }
  gl_FragColor = pow(result, 1 / 2.2);
}

X3D

X3D spec doesn't say anything about whether you should do gamma-correction (TODO grep for sRGB, gamma, linear).

X3DOM

https://doc.x3dom.org/tutorials/lighting/gamma/index.html

Important: All textures and RGB colors are assumed to be gamma coded (the norm for 99.9% of image material).

Later analysis of X3DOM source code by Andreas Plesch reveals that actually the conversion is applied only to textures, not to material vectors like diffuseColor.

From Andreas Plesch, analysis what happens in shader when Environment.gammaCorrectionDefault="linear" is used:

Castle Game Engine

We implement gamma correction following the ideas outlined on this page.

See Gamma Correction in Castle Game Engine documentation.

A detailed algorithm:

  • When loading a color texture (like PhysicalMaterial.baseTexture or Material.diffuseTexture), we convert it using pow(..., 2.2)

  • Note that color values given explicitly (not from texture), like PhysicalMaterial.baseColor or Material.diffuseColor or UnlitMaterial.emissiveColor, are not converted through pow(..., 2.2). This is compatible with both glTF and X3DOM.

  • Then we calculate the lighting as usual.

  • Then we optionally apply a "tone mapping" operator.

  • Then we do pow(..., 1.0 / 2.2) before displaying the color on the screen.

The core of the implementation happened in these commits: https://github.com/castle-engine/castle-engine/compare/47b078f0ff73...ec960a262bcf

glTF

glTF specification prose, 1

glTF specification says:

https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#materials

  • "RGB color values use sRGB color primaries."

    What does it mean? Investigating glTF sample implementation, stuff like baseColorFactor is assumed to be already provided in linear space (not sRGB).

  • Textures should use sRGB transfer function:

    • "The baseColorTexture uses the sRGB transfer function and must be converted to linear space before it is used for any computations."
    • emissiveTexture: "This texture contains RGB components encoded with the sRGB transfer function."

    Investigating glTF sample implementation, they do SRGBtoLINEAR (which does pow(x,2.2) basically) on all colors sampled from color textures (like baseColorTexture).

I think above means: you should use gamma correction, assuming textures are in sRGB.

  • this is consistent with X3DOM "linear" approach for textures
  • this is not consistent with X3DOM "linear" approach for parameters like Material.diffuseColor

glTF sample implementation closer look

sRGB is practically "power 2.2", see https://blog.molecular-matters.com/2011/11/21/gamma-correct-rendering/

glTF sample implementation: https://github.com/KhronosGroup/glTF-Sample-Viewer/

sRGB conversion functions are in tone mapping GLSL code: https://github.com/KhronosGroup/glTF-Sample-Viewer/blob/master/src/shaders/tonemapping.glsl

Shader code, when accessing texture, does SRGBtoLINEAR, like SRGBtoLINEAR(texture2D(u_BaseColorSampler, getBaseColorUV())) (hint: grep glTF-Sample-Viewer code for SRGBtoLINEAR).

See e.g. glTF-Sample-Viewer code src/shaders/metallic-roughness.frag:

#ifdef HAS_BASE_COLOR_MAP
    baseColor = SRGBtoLINEAR(texture2D(u_BaseColorSampler, getBaseColorUV())) * u_BaseColorFactor;
#else
    baseColor = u_BaseColorFactor;
#endif

( This has since been modified in their code, it's now in https://github.com/KhronosGroup/glTF-Sample-Viewer/blob/master/src/shaders/pbr.frag . The intention is the same. )

And SRGBtoLINEAR in practice just does "pow(x, 2.2)".

Observe: glTF does not adjust baseColorFactor. It assumes that it's already in "linear" space.

At the end (right at wring gl_FragColor.rgb they convert color using LINEARtoSRGB. Possibly using tone-mapping function toneMap, all tone-mapping in practice also call LINEARtoSRGB at end.

#ifndef DEBUG_OUTPUT // no debug

   // regular shading
    gl_FragColor = vec4(toneMap(color), baseColor.a);

#else // debug output

// ... skipped various debug features

    #ifdef DEBUG_BASECOLOR
        gl_FragColor.rgb = LINEARtoSRGB(baseColor.rgb);
    #endif

// ... skipped various debug features

    gl_FragColor.a = 1.0;

#endif // !DEBUG_OUTPUT

glTF specification prose, 2

glTF also says:

Implementation Note: Color primaries define the interpretation of each color channel of the color model, particularly with respect to the RGB color model. In the context of a typical display, color primaries describe the color of the red, green and blue phosphors or filters. The same primaries are also defined in Recommendation ITU-R BT.709. Since the overwhelming majority of currently used consumer displays are using the same primaries as default, client implementations usually do not need to convert color values. Future specification versions or extensions may allow other color primaries (such as P3) or even provide a way of embedding custom color profiles.

I think the above paragraph just means "most existing displays use CRT". It doesn't really say whether the renderer should or not should not do gamma correction.

So, as far as I understand, the above paragraph doesn't say anything particularly interesting for us.

glTF specification prose, 3

glTF does not say explicitly "you must do linear workflow". It simply says stuff like "this texture should use sRGB transfer" which means that, to have correct results, you should do linear workflow.

https://github.com/KhronosGroup/glTF/tree/master/specification/2.0

It seems their lighting spec doesn't explicitly say to use "linear workflow" (i.e. convert colors), looking at

Unity

https://docs.unity3d.com/Manual/LinearRendering-LinearOrGammaWorkflow.html

They have "linear" and "none" ("none" seems to be called just "gamma") workflows. The description seems to be complicated by the fact that they do "linear" using proper hardware help (I assume this means using GPU features for sRGB textures/buffers).

See also:

TODO: What is default?

Summary

  • treatment of textures matches between "glTF sample implementation" and "X3DOM with Environment.gammaCorrectionDefault=linear. They both apply pow(x,2.2) to it.

  • treatment of explicit parameters (like diffuseColor in X3D, baseColorFactor in glTF) also matches between "glTF sample implementation" and "X3DOM with Environment.gammaCorrectionDefault=linear. Neither X3DOM nor glTF do pow(x,2.2) on them.

What to do in X3Dv4?

  • Be compatible with glTF, be compatible with X3DOM tested approach:
    • textures should use sRGB transfer function
    • material parameters should not use sRGB transfer function
  • What should be the default? Do gamma or not? In CGE, I decided that default is "do gamma correction for PhysicalMaterialonly", see https://castle-engine.io/manual_gamma_correction.php .