What is PBR and why do we want it in X3D? - michaliskambi/x3d-tests GitHub Wiki

This page is the 1st part (out of 3) of the Include PBR (PhysicalMaterial and related concepts) in the official X3D specification series. It describes PBR, how others integrate PBR, and the reasons behind it.

Table of Contents:

What is Physically-based Rendering?

It's a different lighting model. It uses terms like "albedo" (or, equivalent, "baseColor"), "metallic", "roughness".

In comparison, the current (non-PBR) Phong lighting model uses terms like "diffuse" and "specular". There is no simple 1-1 correspondence between these two lighting models. There is no automatic conversion possible -- equations are just different, neither is a generalization of the other.

Metallic-roughness terms

This image visualizes the meaning of "metallic" and "roughness" terms. (For the exact mathematical equations, see the glTF 2.0 materials specification.)

PBR metallic-roughness terms meaning

This image was made with an environment of uniform medium gray. An interactive version of this is available, where you can change the environment map.

This display was built using XSeen, which is built on top of THREE.js. Thank you to Leonard Daly for creating this!

Useful resources to learn about PBR

Paper presenting PBR in X3D

In the context of X3D, this paper proposes new X3D nodes PhysicalMaterial and PhysicalEnvironmentLight: "A unified GLTF/X3D extension to bring physically-based rendering to the web" (by Timo Sturm, Miguel Sousa, Maik Thöner, Max Limper) from the Web3D 2016 conference.

If you don't have time to read all the above references, then read only this

Why do we want it?

  • Looks better.

    • Because it is more physically-accurate (although the mere usage of PBR equations doesn't make you physically-accurate, you still have to choose correct parameters for everything).
    • Because "everything is shiny", which means that things have more realistic (and interesting) reflections. Typically, you need for this environment textures (cubemaps), which in the simplest case could be ImageCubeMapTexture (although of course could be also GeneratedCubeMapTexture, if you want dynamic environment).

  • Is a standard:

    • In glTF 2.
    • In Unity3d 5.
    • In Unreal Engine 4.

  • It can be easily used in other software too:

    • Blender (although defaults still present Phong lighting model).

Compatibility

If we don't want to break backward compatibility, then the old Material node must still be kept in X3D specification, along with it's lighting equations. Using PhysicalMaterial instead of Material is just an option for new models. So Apperance.material is either Material or PhysicalMaterial.

It's not that weird having both PBR and non-PBR lighting models possible.

Unity3d has it too. Unity3d 5 still offers old (non-PBR) materials from Unity3d <= 4. It also does not convert non-PBR materials to PBR when importing a project (as there's no perfect conversion).

Is this critical to include in X3D?

Yes and no...

It is standard in some popular technologies (glTF, and at least two big proprietary game engines) nowadays.

That said, the old lighting model also still works, and is successfully used in existing applications (including games).

  • For applications that don't strive to have maximum realism, PBR is not really critical.
  • Unity3d 5 still offers old (non-PBR) materials from Unity3d <= 4. It also does not convert non-PBR materials to PBR when importing a project (as there's no perfect conversion). So the projects started in Unity 4 still use non-PBR materials, even if opened in Unity 5, unless you manually update all the materials (adjusting parameters) -- which is generally a substantial work for the graphic artist.

Personal summary (Michalis)

  • The old model (non-PBR) must be kept anyway. It is normal to have both non-PBR and PBR possible.

    I do not think that X3D 4.x can break backward-compatibility, like glTF 2.0 did. We have more history, more data to preserve.

    So we should do it like Unity: new models can use PBR (but do not have to). Existing models keep using non-PBR if you don't manually update them.

  • The new model (PBR) is well-researched and specified. We have glTF spec, we have spec how to do it in X3D.

The new model (PBR) is the way of the future, but I think no-one will stop using X3D because it has non-PBR materials as of today (early 2018). As long as we will make it clear that we plan to incorporate PhysicalMaterial in the near future. I'm fine with deferring it for X3D 4.1 (not doing it for X3D 4.0), if we officially acknowledge that we want it in 4.1, and we have a proposition how it could look like: "We plan to include PhysicalMaterial in X3D 4.1. In the meantime, this is how it will probably look like: " ... and point people to the paper A unified GLTF/X3D extension to bring physically-based rendering to the web (or maybe to this wiki page -- I'm trying to document here consensus reached on x3d-public).

This is sensible because:

  • X3D 4.0 is taking time to be released. Let's not burden it more than necessary.
  • X3D 4.0 may include glTF support for meshes. It seems reasonable to integrate X3D with glTF incrementally:
    • X3D 4.0 allows to use glTF for meshes (using ExternalGeometry, like X3DOM is already doing).
    • X3D 4.1 allows to use glTF for lighting too (using Inline).
    • This allows existing X3D implementors to also incrementally integrate with glTF.

How does this affect my other proposals?

  • The need for CommonSurfaceShader (see Include CommonSurfaceShader in the official X3D specification) is no longer critical. (Assuming that we want to encourage everyone to switch to PBR eventually, like glTF does.) Many features of CommonSurfaceShader extend the Phong lighting model, and have no use for PBR model.

    But note that some features of CommonSurfaceShader are useful for all lighting models: normalmaps, displacement maps. PBR still benefits from normalmaps, just like Phong lighting model. So we should "extract" these features and allow them in some new way. So normalmaps should be specified outside of CommonSurfaceShader, outside of Material, outside of PhysicalMaterial. Displacement maps can probably be rejected for now from the X3D spec (only X3DOM implements them).

    The answer to this question is dealt with more detail in How to add PBR to X3D?.

    Here, in summary:

    1. To upgrade the Phong lighting model, let's extract from CommonSurfaceShader useful concepts, like diffuseTexture, specularTexture, emissiveTexture, and add them to the normal Material. This will also make Material consistent with PBR's PhysicalMaterial, as PhysicalMaterial already does incorporate mechanism to parametrize everything using textures (albedoFactor, albedoMap).

    2. We need to invent a way to add normalmaps (and maybe displacement maps) for PhysicalMaterial, as the need for normalmaps is the same for old and new (non-PBR and PBR) lighting models. X3D standard currently does not allow to reliably provide normalmaps (without writing your own shaders), which is a pity -- normalmaps are an absolute norm in modern 3D content creation. CommonSurfaceShader was a solution for this, but only in case of non-PBR. If we want to include normalmaps for both non-PBR and PBR, we need to design a new solution for this. For starters, I propose a new simple field on Appearance like Appearance.normalMap, like this view3dscene/Castle Game Engine extension: https://castle-engine.io/x3d_implementation_texturing_extensions.php#section_ext_bump_mapping

  • The need to Make RGB and grayscale textures treatment consistent remains.

    1. Although the problem described there (inconsistent RGB and grayscale texture treatment) applies only to the old lighting model, it still remains, since the old lighting model stays. Existing browsers treat it inconsistently, let's make it consistent in X3D 4.0.

    2. Of course, let's not repeat the same mistake when designing PBR model. Let's make RGB and grayscale texture treatment consistent, from the start, in PBR workflow. If we just use the same equations as glTF, we'll be fine.

Metallic-roughness or specular-glossiness workflow?

There are actually two ways to express PBR parameters, the

  1. metallic-roughness
  2. specular-glossiness

The 2nd version (specular-glossiness) has parameters that resemble the Phong model a bit more, and may be easier to upgrade (so Phong lighting model -> PBR with specular-glossiness parameters). But it seems that it only means "easier to upgrade for a graphic artist", and not "so easy to upgrade that it can be done automatically".

See e.g.

However, it seems to me that X3D should adopt the "metallic-roughness" model of PBR. Reasons:

  • This is what glTF did. That's the most important reason. glTF only uses the metallic-roughness version. So if we want easy and perfect interoperability with glTF 2.0 (and I think we should), then we need to also adopt the metallic-roughness version, or (like e.g. Unity3d) allow to choose between metallic-roughness and specular-glossiness .

  • Allowing both PBR models (metallic-roughness and specular-glossiness) means more complication in the spec. If the glTF decided that only metallic-roughness is enough, then it's likely also true for X3D.

  • Even though Unity allows to choose between these two approaches, they still do not convert non-PBR materials to PBR with specular-glossiness. This suggests that the automatic conversion is not possible, at least not in a way that would be acceptable to all the models in the wild.

  • Some more arguments are in https://jmonkeyengine.github.io/wiki/jme3/advanced/pbr_part1.html (see the "pros" / "cons" in "Metalness workflow" / "Specular workflow" there. To me, it seems that the metallic-roughness workflow is a winner: less texture space, easier color concept. The only advantage of specular-glossiness is that it allows easier upgrade for graphic artists, but since this upgrade cannot be automated anyway, this advantage is not big.

    Also note that in X3D standard (if you don't consider CommonSurfaceShader, which is not a standard node), there is no way to specify specular maps in X3D. So upgrading from "Phong lighting model -> PBR metallic-roughness" may pose a similar difficulty as upgrading "Phong lighting model -> PBR specular-glossiness" for the artists. TODO: This is something that would be nice to check in practice, of course. I hope to check it once I implement PBR in view3dscene/Castle Game Engine.