Image Based Lighting (EnvironmentLight node) - michaliskambi/x3d-tests GitHub Wiki

Introduction

This is a new, upcoming light source type. It is using cubemaps to describe light source intensity ("environment") around the shape. It can use any X3D cubemap node to describe the environment.

This is a feature I didn't have the time to properly finalize, in a way that satisfies all my own requirements, for X3D 4.0.

It is however partially done, and even partially implemented, in Castle Game Engine and view3dscene. My plan is to work on it as CGE extension in 2021. When ready I can submit it to X3D, for inclusion in the next X3D version.

Node definition

The current definition (implemented in CGE), extremely straightforward:

EnvironmentLight : X3DLightNode {
  SFNode  [in,out]  diffuseTexture   NULL  [X3DEnvironmentTextureNode]
  # MFFloat [in,out]  diffuseCoefficients TODO -- we will probably add this, see below
  SFNode  [in,out]  specularTexture  NULL  [X3DEnvironmentTextureNode]
}

Simple testcases

See https://github.com/michaliskambi/x3d-tests/tree/master/pbr/environment_light . Open them with view3dscene already.

Why call it EnvironmentLight, not ImageBasedLight?

• Because it's a much better name :)

  Image-based light name would emphasize that it's a light defined by textures. Which is not really the crucial idea behind this. In fact, diffuseCoefficients, consistent with glTF extension for IBL, means that you do not have to define diffuse light by textures.

  Environment light emphasizes that it's a light that shines from all possible directions, like a sphere with infinite radius surrounding the shape. This is the main idea of this light.

• Also using the word environment is consistent with how X3D refers to cubemaps, calling them "environment textures" all around. So calling this an "environment light" means that our usage of term "environment" is consistent.

• Timo Sturm paper also proposed name PhysicalEnvironmentLight. We just drop Physical prefix, because it is not only for PBR (not only for PhysicalMaterial). It can work with Phong (Material) as well.

• What is glTF doing? They also hesitate about it, but it seems that "environment" wins:

  • Older extension for this is called EXT_lights_image_based

  • Newer extension for this (work in progress) is called KHR_lights_environment

• Downside: We realize some people are just familiar with the term "image-based lighting" more.

Design goals

• Allow to use X3D X3DEnvironmentTextureNode for the specification of environment. The X3DEnvironmentTextureNode is a powerful concept in X3D to express cubemaps (from images or auto-generated by rendering), we want to integrate with it.

• Have a straightforward conversion from glTF EXT_lights_image_based.

Some open questions

• Allow diffuse as float coefficients, or cubemap, or both?

  EXT_lights_image_based presents an approach to define diffuse lighting part by coefficients (this would be MFFloat in X3D), unlike specular (which is done by cubemaps).

  Even glTF-Sample-Viewer has a different approach to it, where both diffuse and specular are from cubemaps.

  Also it seems more natural to allow both diffuse and specular from cubemaps.

  Timo Sturm paper shows the more natural approach with diffuse and specular from cubemaps too:

  <PhysicalEnvironmentLight>
    <ImageCubeMapTexture url="diffuse_env.dds" containerField="diffuse" />
    <ImageCubeMapTexture url="specular_env.dds" containerField="specular" />
    <ImageTexture url="brdf.png" containerField="brdf" />
  </PhysicalEnvironmentLight>
  

  I think we will want to support both, so we'll have

  EnvironmentLight : X3DLightNode {
    SFNode  [in,out]  diffuseTexture   NULL  [X3DEnvironmentTextureNode]
    MFFloat [in,out]  diffuseCoefficients   TODO # used when diffuseTexture = NULL, must be 3 x 9 array of floats then
    SFNode  [in,out]  specularTexture  NULL  [X3DEnvironmentTextureNode]
  }
  

• I'd like the know the glTF folks rationale behind having coefficients. Both seem to make sense to make (and diffuseTexture approach seems more common, but diffuseCoefficients much better).

  See new glTF IBL proposal: https://github.com/KhronosGroup/glTF/pull/1956

  Answer from Richard: "I think it will settle on having the option of providing either or both of SH coefficients and cubemap texture."

• Note that glTF has additional field rotation. Do we need it, or is this case handled OK by just transforming the light node?

  We may still need it anyway, for global=FALSE case, when it is impossible to transform only the light source without breaking the scope.

• Should diffuse above by irradiance, like irradianceCoefficients and irradianceTexture?