Textures - SethKitchen/ThreeMojo GitHub Wiki

Textures

render/texture.mojo and render/texture_store.mojo. A Texture is an RGBA image with a wrap mode, a filter, a color space, an alpha mode and an optional mip chain. It holds bytes, or floats for an HDR image. A material names one by id.

Two checkerboard cubes turn, nearest beside bilinear

three.js: Texture, DataTexture, DepthTexture, CompressedTexture, KTX2Loader, KTXLoader, DDSLoader, CubeTexture, Data3DTexture, DataArrayTexture, WebGLRenderTarget.texture. Their settings are wrapS, wrapT, magFilter, minFilter, generateMipmaps, flipY, mapping, colorSpace, anisotropy, type and channel. The tools are RGBELoader, EXRLoader, TextureUtils and DataUtils.

Make a texture

checkerboard(size, squares, first, second, wrap=CLAMP, filter=BILINEAR, mipmapped=True, alpha=COVERAGE)
texture_from(image, wrap=CLAMP, filter=BILINEAR, color_space=None, mipmapped=True, alpha=COVERAGE)
Texture(width, height, pixels, wrap=CLAMP, filter=BILINEAR, color_space=SRGB, mipmapped=True, alpha=COVERAGE)
data_texture(width, height, numbers, channels=4, wrap=CLAMP, filter=NEAREST, mipmapped=False, alpha=COVERAGE)
texture_of(framebuffer, wrap=CLAMP, filter=BILINEAR, mipmapped=True, alpha=COVERAGE)
depth_texture_of(framebuffer, wrap=CLAMP)
float_texture(width, height, floats, wrap=CLAMP, filter=BILINEAR, mipmapped=False, alpha=COVERAGE)
float_texture_from(hdr_image, wrap=CLAMP, filter=BILINEAR, mipmapped=False, alpha=COVERAGE)

Every texture clamps its edges by default, as three.js's ClampToEdgeWrapping does. Pass REPEAT to tile a texture. The glTF, OBJ, FBX and Collada loaders repeat by default, as their three.js loaders do.

checkerboard builds a test pattern. texture_from takes a DecodedImage from the PNG reader. Texture takes row-major RGBA bytes from the top. The next three are below. The float textures are in HDR images.

From numbers

data_texture is three.js's DataTexture: a texture from raw numbers rather than from an image file. Each number is a fraction from zero to one and becomes one byte, quantized without any transfer function. The texture is LINEAR, so the byte samples back as the fraction it was. A number outside zero to one is clamped.

var ramp = data_texture(3, 1, [0.35, 0.7, 1.0], channels=1, alpha=IGNORED)
var cel = toon_material(Color(255, 170, 60), gradient_map=assets.textures.add(ramp^))
channels three.js Each texel holds
1 RedFormat Red. Green and blue are zero, alpha is one.
2 RGFormat Red and green. Blue is zero, alpha is one.
3 RGBFormat Red, green and blue. Alpha is one.
4 RGBAFormat All four. The default.

The numbers fill the channels in order, as WebGL samples these formats. A one-channel texture is red, not gray. A toon ramp reads red alone, so one channel serves it. A gray image needs three equal numbers a texel.

The defaults are three.js's own for a DataTexture: nearest, no mip chain, and the edges clamped. Data is read where it was written. The length must be width * height * channels, and every number must be finite.

From a render

texture_of takes a Framebuffer and returns the texture that holds it: three.js's WebGLRenderTarget.texture. A framebuffer holds eight-bit sRGB with unassociated alpha, which is what a color texture holds, so nothing is converted. RenderTarget.texture() resolves a target and does the same in one call. A later draw can then sample what an earlier one drew.

var target = RenderTarget(160, 120, Color(30, 60, 90))
renderer.render_into(target, stage, props, studio_camera)
var picture = assets.textures.add(target.texture())
var screen = Material(Color(255, 255, 255), picture, kind=BASIC)

The texture is a copy, not a view. Drawing into the target again changes nothing the texture holds. The edges are clamped by default, as three.js clamps a render target's texture.

A 3D, array or cube target reads back as a Data3DTexture, a DataArrayTexture or a CubeTexture. See Layered render targets. To copy part of one texture into another, or part of a frame into a texture, see Texture copies.

It is a snapshot in bytes. Light above one was clamped or tone mapped when the image was resolved, and a later exposure cannot bring it back. That suits a picture on a screen in the scene. A linear texture that keeps a render's range for a later pass is a different thing, and this is not it.

depth_texture_of takes the depth the framebuffer carries and returns it as a texture: a preview of three.js's DepthTexture, at eight bits. Each texel is the window-space depth a GPU stores, as one gray byte in every channel. It is zero at the near plane and one at the far plane. A pixel nothing was drawn into is at the far plane. The texture is LINEAR, ignores its alpha, and is read nearest with no chain: two depths averaged are the depth of nothing.

RenderTarget.depth_texture() does the same for a target as it stands. depth_texture_of_buffer takes a bare depth buffer.

It is a picture of the depth, not the depth. A byte holds 256 steps, and a perspective projection spends most of them near the near plane. Take planes at a tenth of a meter and a hundred. A surface one meter away is byte 230, ten meters is 253, and forty meters is 255, the far plane's own byte. three.js's DepthTexture holds a real depth format. Use this to look at a depth, not to compare or reconstruct one.

examples/television.mojo renders a box into a small target every frame and shows its picture and its depth on two screens.

A box plays on one screen and its depth on the other

From a compressed file

render/compressed_texture.mojo. compressed_texture decodes a block-compressed payload into an ordinary texture: three.js's CompressedTexture. The GPU samples a compressed texture as it is. This project's rasterizers read bytes, so the blocks are decoded once, on the host, and sampled like any other image.

var image = compressed_texture(width, height, blocks, RGBA_S3TC_DXT5_FORMAT)

compressed_texture(width, height, data, format, wrap=CLAMP, filter=BILINEAR, color_space=None, mipmapped=False, alpha=COVERAGE). The defaults are three.js's own for the class: edges clamped and no chain, since a compressed file usually carries its own levels. Only the level you pass is read. Pass mipmapped=True to build a chain from the decoded image.

The S3TC formats are below. See KTX2 and compressed formats for the other fourteen formats and for the three container files.

format three.js A 4x4 block is
RGB_S3TC_DXT1_FORMAT RGB_S3TC_DXT1_Format Eight bytes: two RGB565 colors and a two-bit index per texel. The transparent index reads as opaque black.
RGBA_S3TC_DXT1_FORMAT RGBA_S3TC_DXT1_Format The same, with the transparent index read as transparent black.
RGBA_S3TC_DXT3_FORMAT RGBA_S3TC_DXT3_Format Sixteen bytes: sixteen four-bit alphas, then a color block always read in its four-color order.
RGBA_S3TC_DXT5_FORMAT RGBA_S3TC_DXT5_Format Sixteen bytes: two alphas and a three-bit index per texel, then a color block always read in its four-color order.

The 565 channels widen to eight bits by copying their top bits down, as the hardware widens them. The blends round to nearest. An image need not be whole blocks: the texels past the edge are decoded and dropped. decode_s3tc returns the RGBA bytes of an S3TC payload without building a texture. decode_compressed does the same for every format.

compressed_texture refuses a format that is none of the twenty-three, dimensions that are not positive, and a payload whose length is not the block grid's. tests/compile_fail/ proves a bare integer is not a format.

KTX2 and compressed formats

render/compressed_texture.mojo decodes eighteen block formats. render/dds.mojo, render/ktx.mojo and render/ktx2.mojo read the three container files that three.js reads. Each container gives every level of every face, and a texture method decodes one of them. A KTX2 file can also hold Basis Universal data and Zstandard supercompression. This reader decodes UASTC, UASTC HDR 4x4, ETC1S and ETC1S video, and Zstandard.

Three panels show a UASTC image, an ETC1S image and a gradient

examples/basis.mojo draws this picture.

three.js: KTX2Loader, KTXLoader, DDSLoader, CompressedTexture and the compressed format constants.

from render.ktx2 import read as read_ktx2

var container = read_ktx2(Path("rock.ktx2").read_bytes())
var rock = assets.textures.add(container.texture(mipmapped=True))

The formats

A format that a byte cannot hold decodes to floats, and its texture is a float texture. CompressedFormat.is_float names these formats. A color format is SRGB by default. Every other format holds data, so it is LINEAR, and SRGB is refused.

format Block Decodes to
RGB_S3TC_DXT1_FORMAT, RGBA_S3TC_DXT1_FORMAT BC1, 8 bytes RGBA bytes
RGBA_S3TC_DXT3_FORMAT BC2, 16 bytes RGBA bytes
RGBA_S3TC_DXT5_FORMAT BC3, 16 bytes RGBA bytes
RED_RGTC1_FORMAT BC4, 8 bytes Red bytes
SIGNED_RED_RGTC1_FORMAT BC4 signed, 8 bytes Red floats, -1 to 1
RED_GREEN_RGTC2_FORMAT BC5, 16 bytes Red and green bytes
SIGNED_RED_GREEN_RGTC2_FORMAT BC5 signed, 16 bytes Red and green floats, -1 to 1
RGB_BPTC_UNSIGNED_FORMAT BC6H, 16 bytes RGB half floats, widened
RGB_BPTC_SIGNED_FORMAT BC6H signed, 16 bytes RGB half floats, widened
RGBA_BPTC_FORMAT BC7, 16 bytes RGBA bytes
RGB_ETC1_FORMAT ETC1, 8 bytes RGB bytes
RGB_ETC2_FORMAT ETC2, 8 bytes RGB bytes
RGBA_ETC2_EAC_FORMAT EAC alpha and ETC2, 16 bytes RGBA bytes
R11_EAC_FORMAT, SIGNED_R11_EAC_FORMAT EAC, 8 bytes Red floats
RG11_EAC_FORMAT, SIGNED_RG11_EAC_FORMAT Two EAC blocks, 16 bytes Red and green floats

A red or a red-green format gives a blue of zero and an alpha of one, as WebGL samples it. render/bptc.mojo holds the BC6H and BC7 decoders, and render/etc.mojo holds the ETC and EAC decoders.

  • BC7 blends two endpoints with an exact integer weight. Every decoder gives the same texels. A block whose first byte is zero names no mode and decodes to transparent black.
  • BC6H has fourteen modes. The endpoints are unquantized to sixteen bits, blended, and read as a half. Four reserved mode numbers decode to black.
  • ETC2 reads an ETC1 block unchanged. A difference that leaves its range selects the T, H or planar mode.
  • EAC R11 keeps eleven bits. A texel is (base * 8 + 4 + offset * multiplier * 8) / 2047, or / 1023 for the signed form.

The tests check each decoder against hand-built blocks with worked-out texels. They also check blocks of every mode against the reference decoders bcdec and texture2ddecoder.

The containers

Reader Returns Reads
render.dds.read(bytes) CompressedImage A four-character code or a DX10 header, uncompressed BGRA and BGR, levels and cubes.
render.ktx.read(bytes) CompressedImage KTX 1 in both byte orders, levels and cubes, PVRTC included.
render.pvr.read(bytes) CompressedImage PVR versions 2 and 3 of PVRTC, levels and cubes.
render.ktx2.read(bytes) KTX2Container KTX 2.0: the header, the level index, the data format descriptor, levels, cubes and arrays.

CompressedImage.mipmaps holds each face's levels in turn: mipmaps[face * levels + level]. texture(face, level, ...) decodes one of them. KTX2Container.level_data holds each level with its supercompression removed. texture(face, layer, level, ...) decodes one image of it.

The color space comes from the file. A DDS or KTX file is SRGB only for a format with sRGB in its name. A KTX2 file is SRGB when its data format descriptor names the sRGB transfer function, as three.js reads it.

A KTX2 file can also hold eight-bit, half and float texels. R8, R8G8 and R8G8B8A8 give a byte texture, in UNORM or SRGB. R16, R16G16 and R16G16B16A16 halves, and R32 to R32G32B32A32 floats, give a float texture. These are the uncompressed formats three.js reads.

KTX2 supercompression Here
0, none Read.
1, BasisLZ Read for ETC1S data through render/etc1s.mojo. BasisLZ holds only ETC1S.
2, Zstandard Read through render/zstd.mojo.
3, zlib Read through render/inflate.mojo. three.js refuses it.

Raw ASTC

KTX2 reads all six raw ASTC formats in three.js r180. Each block uses sixteen bytes. The host decoder shares the integer-sequence and endpoint decoder with Basis HDR. Its block footprint controls partition coordinates, weight interpolation and edge cropping.

Vulkan format Number Block Texture
ASTC 4x4 UNORM 157 4x4 Linear RGBA bytes
ASTC 4x4 SRGB 158 4x4 sRGB RGBA bytes
ASTC 6x6 UNORM 165 6x6 Linear RGBA bytes
ASTC 6x6 SRGB 166 6x6 sRGB RGBA bytes
ASTC 4x4 SFLOAT 1000066000 4x4 Linear RGBA floats
ASTC 6x6 SFLOAT 1000066004 6x6 Linear RGBA floats

The descriptor must name the ASTC color model, KHR_DF_MODEL_ASTC (162). This is the matching block-compressed model required by the KTX2 data format descriptor rules. Files with another model now raise an error.

The Vulkan format and descriptor transfer function must agree. sRGB stays encoded in the byte texture until sampling. UNORM endpoints expand by bit replication. sRGB endpoints expand by a shift and midpoint bias. Both LDR profiles use the top eight interpolated bits. The decoder does not convert LDR through half floats.

SFLOAT uses the ASTC half-float conversion, then widens each half exactly to a float. LDR and HDR alpha are supported. The separate Basis HDR alpha restriction does not apply.

The decoder reads every legal block mode for these footprints and all sixteen endpoint modes. It supports one to four partitions, dual planes, and bit, trit and quint sequences. HDR endpoints and HDR constant blocks in an LDR format are refused. Malformed blocks and nonfinite HDR constants raise an error instead of producing the hardware error color.

The reference is Arm astcenc 5.3.0 at commit 30aabb3f42406df45a910d8496f9bee17eeba9bb. The checked-in corpus compares 2,152 decoded blocks with its UNORM8 and FP16 outputs. It covers all 145 legal 4x4 block modes and all 370 legal 6x6 block modes. These comparisons require exact byte or half-bit equality, with zero tolerance.

Hardware can select another permitted decode precision; identical GPU texels are not promised. Another 8,192 malformed blocks must be refused. See the reproduction instructions.

Mips, cube faces, array layers, edge dimensions and Zstandard use the same container bounds as the other KTX2 formats. Decoding allocates the final byte or float image directly, with only one block of temporary texels.

Basis Universal

A KTX2 file whose data format descriptor names the UASTC, UASTC HDR or ETC1S color model holds Basis Universal data. three.js transcodes the data to a GPU format with the Basis Universal WebAssembly transcoder. Here texture decodes UASTC and ETC1S to an RGBA byte texture, with the texels of that transcoder's RGBA32 target. It decodes UASTC HDR to a float texture, with the texels of the RGBA_HALF target.

var container = read_ktx2(Path("rock_uastc.ktx2").read_bytes())
if container.is_uastc() or container.is_etc1s():
    var rock = assets.textures.add(container.texture(level=0))
var sky = read_ktx2(Path("sky_uastc_hdr.ktx2").read_bytes())
if sky.is_uastc_hdr():
    var light = assets.textures.add(sky.texture())
Data Module How it decodes
UASTC LDR 4x4 render/uastc.mojo One 16-byte block for each 4x4 texels. A prefix code names one of nineteen modes. The mode sets the subsets, the planes, the components and the endpoint and weight ranges. The endpoints unquantize and blend as ASTC blends them.
UASTC HDR 4x4 render/uastc_hdr.mojo One ASTC 4x4 block for each 4x4 texels. The decoder reads every ASTC feature that fits a 4x4 block: void extents, weight grids of 2x2 to 4x4, dual planes, one to four partitions, and all sixteen endpoint modes. Each texel becomes four half floats.
ETC1S render/etc1s.mojo The BasisLZ global data holds an endpoint codebook, a selector codebook and four Huffman tables. Each image is a slice of Huffman-coded codebook indices, with predictions from the neighbor blocks and a history of recent selectors. A file with alpha has a second slice per image, and its green is the alpha.
ETC1S video render/etc1s.mojo Each frame is a layer. A block of a P-frame can copy the same block of the frame before.

The three modules port the transcoder of Binomial's Basis Universal, which is Apache-2.0. THIRD-PARTY-NOTICES.md holds its notice. A UASTC or UASTC HDR level can be stored whole, or supercompressed with Zstandard or zlib.

UASTC HDR. Its Vulkan format is VK_FORMAT_UNDEFINED or VK_FORMAT_ASTC_4x4_SFLOAT, as the transcoder accepts. An HDR endpoint blends in ASTC's logarithmic encoding and becomes a half. An LDR endpoint blends in sixteen bits and is truncated to a half. The texture holds each half widened to a float, so the values are exact. The texture is LINEAR, as a half float texture is.

ETC1S video. A file is a video when its key and value data holds the key KTXanimData. A flat array with a P-frame is a video too. The transcoder keeps the last frame of each level, and three.js asks for the images face by face, and layer by layer in each face. So texture decodes every image that three.js decodes before the one it returns, at the same level.

The tests decode files that the Basis Universal 2.50 encoder wrote. Between them, the UASTC files use all nineteen modes. The ETC1S files have alpha, mipmaps, sizes that are not a multiple of four, and gray endpoints. The videos have P-frames, alpha and six levels. A UASTC HDR file has Zstandard, mipmaps and a size of 21x13. Every level must have the same texels as the three.js r186 transcoder gives.

A second UASTC HDR file holds 240 random ASTC blocks that the transcoder accepts. They use every endpoint mode, partition count, grid and range. The transcoder and this decoder also agree on 200,000 random blocks, valid and not.

Uncompressed DDS

A DDS file whose code names no block format is read as three.js's DDSLoader reads it: by its bit count and masks. Thirty-two bits with red, green, blue and alpha masks is BGRA. Twenty-four bits with the three color masks is BGR, with an opaque alpha. Its format is RGBA_FORMAT, three.js's RGBAFormat. As in three.js, a mask only has to overlap its byte, and the format flags are not read.

PVRTC

render/pvrtc.mojo decodes PVRTC1 at 4 and 2 bits a texel: three.js's RGB_PVRTC_4BPPV1_Format and the next three. three.js hands PVRTC to a GPU that decodes it. This decoder follows Imagination's reference decoder, PVRTDecompress.cpp, whose output the tests compare with. The RGB forms are opaque.

The width and the height must be powers of two. A level holds two words across and down at least, as three.js's PVRLoader reads it.

render.pvr.read reads a PVR file as PVRLoader does. Version 3 names its format by a pixel format, 0 to 3. Version 2 names it by its flags, 24 or 25, and an alpha mask makes it RGBA. Six faces make a cube. KTX 1 reads PVRTC by its GL formats, 0x8C00 to 0x8C03.

Zstandard

render/zstd.mojo decodes Zstandard, RFC 8878, with no compression library. zstd_decompress(bytes, limit) returns the bytes of every frame, joined. A KTX2 level with scheme 2 is one Zstandard frame, and read decompresses it to its stated size.

The decoder reads the whole format except dictionaries:

  • Raw, RLE and compressed blocks, and skippable frames.
  • Raw, RLE, Huffman and treeless literals, in one stream or in four.
  • Huffman weights, direct or coded with an FSE table.
  • Sequences with predefined, single-symbol, described and repeated FSE tables.
  • The three repeated offsets, and the checksum, an XXH64 of the content.

The tests decode frames that libzstd 1.5.7 wrote at levels -5 to 19. Frames built by hand cover the modes libzstd rarely writes. libzstd decodes the same bytes from them.

What is not ported

  • UASTC HDR 6x6, XUASTC LDR and XUBC7. three.js cannot transcode them either: its transcoder is Basis Universal 1.50, which does not know these formats. read refuses them by name.
  • UASTC HDR with alpha. three.js has no transcoder target for it and throws, so read refuses it too.
  • The transcode from UASTC, UASTC HDR or ETC1S to a GPU block format. Each image decodes to the texels of the transcoder's RGBA32 or RGBA_HALF target.
  • Zstandard dictionaries. A frame that names a dictionary is refused. three.js cannot read one either: its transcoder and its Zstandard decoder read no dictionary.
  • ASTC footprints other than 4x4 and 6x6, PVRTC, and ETC2 with punch-through alpha as Vulkan formats in KTX2. ASTC and ETC2 with punch-through alpha remain excluded in KTX 1. three.js uploads ASTC to compatible GPU hardware.
  • PVRTC2, and PVRTC1 of a size that is not a power of two.
  • The premultiplied DXT2 and DXT4, and DDS texture arrays.
  • 1D, 3D and array textures in KTX 1, and 1D and 3D textures in KTX2.
  • The GPU upload of the blocks. Each image decodes on the host and keeps RGBA in memory.
  • A file's own smaller levels in the texture. The container gives them, but a texture builds its chain from the level it decodes.
  • CompressedCubeTexture and CompressedArrayTexture. Take each face or layer with texture(face, layer).

Where the formats differ from three.js

  • three.js hands the blocks to the GPU. Here the decoded texels are what the hardware gives, to the last bit for BC6H, BC7, ETC and EAC. The BC1 to BC5 blends round to nearest, as the reference decoder rounds them. A GPU can differ by one step.
  • The eleven-bit EAC formats and the signed RGTC formats decode to floats. A byte cannot hold a negative value or eleven bits.
  • DDSLoader computes a level's size from max(4, width) / 4, which is a fraction for a width that is not a multiple of four. This reader rounds up to whole blocks.
  • A container is refused if it names more levels than its size has, or a face count other than one or six. A level that does not fit is refused too. three.js reads past such a file.
  • three.js transcodes Basis Universal data to a GPU format, and hands that to the GPU. Here UASTC and ETC1S decode to the RGBA32 texels of the same transcoder. A GPU that samples the transcoded blocks can differ from them by a step.
  • three.js transcodes UASTC HDR to BC6H when the GPU takes BC6H, and to RGBA_HALF otherwise. Here it always decodes to the RGBA_HALF texels, held as floats.
  • A UASTC HDR file with the sRGB transfer function is refused, as a half or float format is. three.js makes a half float texture of it.
  • The Basis Universal encoder can scale HDR values and store the scale as the key KTXmapRange. three.js does not apply the scale, and this reader does not apply it either.
  • A Zstandard repeated offset of zero is refused. libzstd reads it as one.
  • An ETC1S Huffman code of one symbol matches only its own bit. Binomial's decoder reads any other bit as that symbol.

Errors

  • Each container refuses a wrong identifier or magic, a size that is not positive, and a file that ends early.
  • A size that decodes to more than MAX_DECODED_BYTES is refused before any allocation. A float format counts sixteen bytes a texel.
  • A KTX2 file refuses a descriptor or a level outside the file, and a level whose length is not its size.
  • A KTX2 half or float format with the sRGB transfer function is refused.
  • A KTX2 file with ETC1S data must use BasisLZ, and BasisLZ must hold ETC1S. UASTC and ETC1S data must name no Vulkan format. UASTC HDR data must name no Vulkan format or ASTC 4x4 SFLOAT.
  • The key and value data of a Basis Universal file is checked as the transcoder checks it. An entry that is cut short, or a key with no zero byte to end it, is refused.
  • The BasisLZ global data must fit in the file and hold what its header states. A codebook or a table that is empty or malformed is refused.
  • texture refuses a UASTC block with the reserved mode or a partition its mode does not have. It refuses an ETC1S slice that predicts from outside the image or reads past its codebooks.
  • texture refuses a UASTC HDR block that the transcoder refuses. That is a reserved block mode, a weight grid larger than 4x4, or weights of fewer than 24 bits or more than 96. It is also dual planes with four partitions, endpoints that do not fit, or a malformed void extent.
  • texture refuses a level past the sixteenth of an ETC1S video, as the transcoder does.
  • zstd_decompress refuses a frame that is cut short, has a reserved bit or type, or does not match its size or its checksum. It refuses a malformed table or bitstream, and a stream that expands past limit.
  • tests/compile_fail/ proves that a bare integer is not a VkFormat, a Supercompression or an ASTC EndpointMode.

HDR images

An HDR image holds linear light with no upper limit. render/rgbe.mojo reads a Radiance .hdr file and render/exr.mojo reads an OpenEXR file. Each returns a FloatImage, and float_texture_from turns it into a float texture. Use the texture as a map, or turn it into a cube for a background or an environment.

three.js: RGBELoader (now HDRLoader), EXRLoader, DataTexture with FloatType, and WebGLCubeRenderTarget.fromEquirectangularTexture.

from render.exr import decode as decode_exr
from render.rgbe import decode as decode_rgbe

var sky = float_texture_from(decode_exr(Path("sky.exr").read_bytes()))
var cube = assets.cube_textures.add(cube_from_equirectangular(sky, mipmapped=True))
scene.background = cube_background(cube)
scene.environment = cube
var lamp = assets.textures.add(float_texture_from(decode_rgbe(Path("lamp.hdr").read_bytes())))

Float textures

float_texture(width, height, floats) takes row-major RGBA floats from the top. Its texel_type is FLOAT_TYPE, three.js's FloatType. A byte texture is UNSIGNED_BYTE_TYPE, three.js's UnsignedByteType. The floats are in data, and pixels is empty.

A float is sampled as it is. There is no ramp, no clamp and no transfer function, so the texture must be LINEAR. The filter and the mip chain average the floats premultiplied, as they average a byte texture's decoded light. A texel of 8 beside a texel of 0 filters to 4, where a byte texture gives one half.

The defaults are three.js's loader settings: CLAMP, BILINEAR and no chain. Every number must be finite: an infinity or a NaN spreads through every filter that reads it. texel(x, y) returns bytes, so a float texture refuses it. Use wrapped_texel(x, y).

Both rasterizers sample a float texture. The GPU texel buffer holds bytes, so each float crosses as four little-endian bytes. The kernel reads the bits back with float_from_bytes, and both backends make the color with float_texel. A gradient map must hold bytes, on both backends.

Read an RGBE file

render.rgbe.decode(bytes) reads a Radiance .hdr file. The header must start with #? and a program name. It must name FORMAT=32-bit_rle_rgbe and a size line -Y height +X width. Each scanline is flat, or run-length encoded one channel after another.

A channel is byte * 2^(exponent - 128) / 255, worked in doubles and stored as a float. This is three.js's arithmetic. Alpha is one. GAMMA and EXPOSURE are read past, as three.js reads past them.

Read an EXR file

render.exr.decode(bytes) reads a single-part scanline OpenEXR file. The header needs channels, compression and dataWindow. Other attributes are read past.

Compression Lines a block Note
NO_COMPRESSION 1
RLE_COMPRESSION 1 Run-length bytes, then the predictor and the split undone.
ZIPS_COMPRESSION 1 zlib through render/inflate.mojo, then the same.
ZIP_COMPRESSION 16 The same, sixteen lines at a time.
PIZ_COMPRESSION 32 A bitmap and a lookup table, a Huffman code, and a Haar wavelet per channel.
PXR24_COMPRESSION 16 Lossless half and UINT samples; reduced-precision float deltas. three.js r180.
B44_COMPRESSION 32 Lossy 4-by-4 half blocks; lossless float and UINT channels. three.js r186.
B44A_COMPRESSION 32 B44 with three-byte flat blocks. three.js r186.
DWAA_COMPRESSION 32 Lossy DCT color; lossless alpha and unknown channels. three.js r180.
DWAB_COMPRESSION 256 The same DWA decoder with larger scanline blocks. three.js r180.

A channel holds halves or floats. A half widens to the float it spells, subnormals and all. R, G and B make RGBA, with A as the alpha or one where there is none. Y alone makes gray, with an alpha of one. Other channels are read past. Rows come out from the top line of the data window.

The new codecs follow OpenEXR 3.1.5. B44 and DWA preserve each channel's pLinear flag. DWA supports versions zero, one and two, static-Huffman or DEFLATE AC data, grayscale, and separate RGB layers. Mixed half, float and ignored UINT channels keep their own sample widths. Raw fallback blocks remain supported.

The reader checks DWA counters against the channel layout before stream allocation. It rejects truncated rules, impossible channel groups, nonfinite DCT coefficients, invalid runs, and incorrect stream lengths. Scanline chunks cannot overlap or start between block boundaries. The image pixel limit is unchanged. Decoded streams cannot exceed four bytes times that pixel limit. DWA coefficient allocations use the same byte budget, including their integer element width.

EXR reference checks

tests/test_exr_codecs.mojo checks every sample of 65 fixtures against independent OpenEXR 3.1.5 output. The fixture generator uses Imath 3.1.5. The fixtures cover half, float, alpha, grayscale, mixed channels, block edges, flat blocks, both pLinear values, and DWA stream variants. assets/exr_codecs/manifest.json records every compressed and raw chunk, source error, and fixture hash. The fixture README gives the regeneration commands.

PXR24, B44 and B44A must match decoded reference samples exactly. DWA allows abs(actual - reference) <= 0.003 * max(1, abs(reference)). This preselected fixture budget permits scalar/SIMD DCT and half-rounding differences. It is not a universal numerical error bound.

The largest observed scaled difference is 0.002522, or three representable half steps in one fixture. All other fixtures match the decoded reference exactly. The fixture README explains the metric and nonlinear rounding amplification. These checks do not promise lossy source-image fidelity. The manifest records compression loss separately.

The comparison scripts also run the unmodified three.js r180 and r186 loaders. They reverse three.js's output rows and request FloatType. They record upstream errors and mismatches, rather than treating those results as reference values. Mixed sample widths and ignored UINT channels expose upstream decoder errors. DWA also exposes grayscale, pLinear, unknown-stream and legacy-version differences. The recorded results name each reproducible fixture.

Six HDR files as a cube

loaders/hdr_cube.mojo reads six Radiance .hdr files into a float cube texture. three.js: HDRCubeTextureLoader.

var paths: List[String] = ["px.hdr", "nx.hdr", "py.hdr", "ny.hdr", "pz.hdr", "nz.hdr"]
var cube = assets.cube_textures.add(read_hdr_cube_texture(paths, SEEN_FROM_OUTSIDE))
  • read_hdr_cube_texture(paths, layout, mipmapped) reads the files in px, nx, py, ny, pz, nz order. hdr_cube_texture_from(images, layout, mipmapped) takes six FloatImages.
  • Each face is a float texture: LINEAR, CLAMP, BILINEAR and no chain, the settings of three.js's loader.
  • SEEN_FROM_OUTSIDE reads the files as three.js's cube loaders read them. See cube_texture_from.
  • three.js's default type is HalfFloatType, which rounds each value to a half. This keeps the floats, as three.js's setDataType(FloatType) does.
  • tests/test_hdr_cube.mojo compares the faces with the floats that three.js 0.180 reads, in assets/hdr_cube/three.json.

Read an UltraHDR file

loaders/ultrahdr.mojo reads an UltraHDR (JPEG_R) file as an HDR image. three.js: UltraHDRLoader.

var hdr = read_ultrahdr("photo.jpg")
var sky = assets.textures.add(float_texture_from(hdr.image, mipmapped=True))

An UltraHDR file holds two JPEG files: the SDR image and a gain map. The SDR image's MPF segment says where the gain map is. The gain map's XMP holds the hdrgm attributes, which hdr.metadata keeps. The HDR image is the SDR image made brighter where the gain map says, by three.js's formula and its table of SRGBToLinear. Each value is clamped to 0 and 65,504.

  • parse_ultrahdr(bytes) reads the bytes of a file. apply_gain_map(sdr, gain, metadata) applies a gain map to decoded images.
  • render.jpeg decodes the two JPEG files. It agrees with libjpeg to one level. A browser decodes them in three.js.
  • A gain map of another size is scaled bilinearly by scale_gain_map. A browser's canvas scales it in three.js, with a filter of its own.
  • The alpha is one. three.js fills each alpha with 255.
  • An APP1 segment that is not XMP, such as EXIF data, is read past. three.js throws a TypeError.
  • A number in the metadata that is not finite, or an HDR capacity range of zero, is refused. three.js gives an image of NaNs.
  • tests/test_ultrahdr.mojo compares the images with what three.js 0.180 gives for the files in assets/ultrahdr/. The node script gives three.js the pixels that render.jpeg decodes, so the segments, the XMP and the formula are three.js's own.

An equirectangular environment or background

A panorama with an equirectangular mapping is read directly, at equirect_uv of each direction. equirect_uv is three.js's equirectUv.

var panorama = float_texture_from(decode_rgbe(bytes), mipmapped=True)
panorama.mapping = EQUIRECTANGULAR_REFLECTION_MAPPING
var sky = assets.cube_textures.add(cube_of_panorama(panorama))
var chrome = Material(Color(255, 255, 255), kind=BASIC, env_map=sky)
scene.background = texture_background(assets.textures.add(panorama^))

cube_of_panorama(image, size=None) makes the environment: a CubeTexture that holds the panorama. sample and sample_level read the panorama, and so does the GPU kernel. The cube also holds six faces from cube_from_equirectangular, for the readers that walk the texels, such as a light probe. Its mapping is the panorama's. A texture background with an equirectangular mapping is read by direction too, not stretched.

cube_from_equirectangular(image, size=None, mipmapped=None) turns a panorama into six faces. three.js does the same in WebGLCubeRenderTarget.fromEquirectangularTexture. The faces keep the panorama's texel type, color space, filter and alpha mode. The face size is the panorama's height by default, as in three.js. The faces get a chain if the panorama has one.

A standard or physical surface reads a PMREM. prefilter_environments(scene, assets) in renderers/environment.mojo builds one for each environment that needs it, as three.js's WebGLCubeUVMaps does. A panorama cube is prefiltered from the panorama. See PMREM.

What is not ported

  • Half-float storage. three.js's loaders default to HalfFloatType. Here a half widens to a float, which holds every half exactly.
  • The PMREM on first use. three.js builds it inside the renderer. Here the renderer only reads the assets, so you call prefilter_environments before the frame. The scene JSON reader calls it for you.
  • Light above one in a background. The backdrop crosses to both backends as sRGB bytes, so a background clips at one before tone mapping. A reflection keeps the floats.
  • RGBE: XYZE pixels, the old Radiance run-length scheme, and every orientation but -Y +X. The first is refused, and three.js reads none of them correctly.
  • EXR: tiled, deep and multi-part files, luminance-chroma images, subsampled channels, and UINT color. Each is refused by name.
  • EXRLoader.setOutputFormat. The output is always RGBA, three.js's default.

Where the HDR readers differ from three.js

Each reader refuses a file that three.js reads wrongly. three.js reads every EXR color channel with the type of the last one. This reader reads each channel as its own type. three.js reads past the end of a block, a Huffman table or a file. It also leaves the rows of a short RGBE file black. Both readers here refuse each of these.

Flakes

flakes_texture(width, height, seed) in render.flakes_texture is three.js's FlakesTexture: a normal map of small discs, each with a random normal. Use it as a car paint's normal map, as three.js's webgl_materials_car example does. The texture holds data, not color, and it repeats.

var flakes = assets.textures.add(flakes_texture(512, 512, seed=7))

The places, the sizes and the normals come from seed, so the same seed makes the same texture on every platform. three.js draws on a canvas with Math.random, so its flakes are different at each run.

Anisotropy

A surface seen at a glancing angle covers a footprint that is long one way and short the other. A mip level is square. The level the long axis wants blurs the short axis, and the level the short axis wants sparkles along the long one. anisotropy, three.js's Texture.anisotropy, is how many samples a fragment can take along the long axis instead, each read at the level the short axis wants.

var floor = checkerboard(64, 8, white, blue)
floor.anisotropy = 16

One, the default, is the plain trilinear read. MAX_ANISOTROPY is sixteen, which is what a desktop GPU reports and what three.js caps a texture at. Set the field after construction, as in three.js. validate() refuses a value below one or above the cap, and the GPU upload refuses it again.

The footprint is an ellipse, not two derivatives

anisotropic_footprint(along_x, along_y, width, height, anisotropy) returns a Footprint: the level, the tap count and the step between taps.

The two arguments are how far the texture coordinates move for one pixel right and one pixel down. They are the columns of a 2x2 matrix, and the footprint is the ellipse that matrix maps the unit disc onto. Its principal lengths are that matrix's singular values, which are not the lengths of the two derivatives.

Measure the derivatives instead and a rotation breaks it. Take a footprint 16 texels long and 1 across, then turn the screen's basis 45 degrees. The ellipse is unchanged, but the two derivatives now have the same length. Measuring them calls the footprint round: one tap at level 3.5, where sixteen taps at level 0 are correct. A surface blurs because of how it happens to lie against the screen axes.

_principal_axes takes the real lengths from a quadratic, and _major_direction takes the real long axis. Neither needs a decomposition library.

The level follows the short axis

Taps along the long axis filter along the long axis. They do nothing across the short one, so rounding the tap count up must not shrink the level below what the short axis needs:

effective minor = max(1, minor, major / taps allowed)
taps            = clamp(ceil(major / effective minor), 1, allowed)
level           = log2(effective minor)

Dividing the major axis by the tap count jumps instead. Take a footprint going from 16 by 16 to 16.001 by 16. It gains one tap, and it used to lose almost a whole level with it, from 4.0 to 3.0. A thousandth of a texel moved the level by one. It now gains the tap and keeps the level. The floor of one stops a tap per texel from reading any texel twice.

With an anisotropy of one the level is the log of the longer derivative. That is the number mip_level gives, and the number OpenGL's isotropic rho gives. A texture that asks for nothing reads exactly as it did.

Sampling a footprint

sample_footprint(u, v, footprint) takes the taps and averages them premultiplied. It calls Footprint.validate() first, because a Footprint is fieldwise-constructible and Footprint(0, 0, Vector2(0, 0)) builds: zero taps would divide the average by nothing. The rasterizers take _sample_footprint, which does not check. What they pass came from anisotropic_footprint, and a fragment loop is not a place to handle an error.

Both rasterizers call the same estimator and the same accumulation; see Rasterization.

Cube textures

render/cube_texture.mojo and render/cube_texture_store.mojo. A CubeTexture is six square textures, one per face of a box around the viewer, sampled by direction rather than by place. It is what a mirror reflects and what a sky is made of.

A chrome ball under a sky reflects two boxes that circle it

three.js: CubeTexture, CubeTextureLoader, WebGLCubeRenderTarget.texture.

var sky = assets.cube_textures.add(cube_texture_from(images, SEEN_FROM_OUTSIDE))
var seen = assets.cube_textures.add(renderer.render_cube(scene, assets, cube_camera))
var chrome = Material(Color(255, 255, 255), kind=BASIC, env_map=sky)
scene.background = cube_background(sky)
Builder Meaning
CubeTexture(faces) Six textures in face order, each square, all one size, all CLAMP.
cube_texture_from(images, layout=SEEN_FROM_INSIDE, filter=BILINEAR, color_space=None, mipmapped=False, alpha=COVERAGE) Six DecodedImages. three.js's CubeTextureLoader.
cube_texture_of(images, filter=BILINEAR, mipmapped=False, alpha=COVERAGE) Six Framebuffers, as a CubeCamera renders them.

The faces are held in three.js's order: POSITIVE_X, NEGATIVE_X, POSITIVE_Y, NEGATIVE_Y, POSITIVE_Z, NEGATIVE_Z. FACE_COUNT is six.

One convention

Every face is what a camera at the center of the box sees looking out along one axis. face_forward(face) is the axis and face_up(face) is the camera's up. The up is positive y for the four side faces, and the z axis for the two faces on y. These are three.js's own six ups. A CubeCamera renders the faces this way, and the sampler reads them this way, so a rendered cube needs no flip.

three.js's CubeCamera uses the same ups with a field of view of minus ninety degrees. That turns each view a half turn. WebGL then stores the render from the bottom row up.

Thus a face of three.js's cube render target is a face here with each row mirrored. three.js samples it through OpenGL's cube map table with flipEnvMap set to one. Each texel then stands for the same direction as it does here. tests/test_cube_texture.mojo checks each texel of each face against that table.

three.js reads six image files in a different layout. The px image is the view along -x, and the nx image is the view along +x. No image is mirrored. three.js gets this layout from an OpenGL cube map and a sign, flipEnvMap, that turns the x axis over.

Here cube_texture_from takes a CubeLayout. SEEN_FROM_INSIDE, the default, reads each image as it is. SEEN_FROM_OUTSIDE swaps the px and nx images on the way in. Use it for six image files that three.js's CubeTextureLoader reads.

Sampling

sample(direction) returns the color in a direction. face_of(direction) picks the face on the axis the direction leans along most. A tie goes to x, then y, then z, as OpenGL picks it. face_uv(face, direction) projects the direction onto that face and reads how far it lands across and up, in the camera's own right and up axes. Both are pure, and both rasterizers call them.

A face is read at its full size, never down a mip chain. A reflection's direction changes across a surface at a rate that is not the surface's own texture footprint. The same rule keeps a matcap out of its chain. One reader asks for the chain by a number of its own: a physical surface reads it by its roughness. sample_level(direction, level) reads a face level down, and levels() says how many there are. See Materials.

A face must be wrapped CLAMP on both axes. A coordinate past a face's edge belongs to the next face, and a flat image has no next face to read. The bilinear filter's neighbors at an edge hold that edge.

Reflection, refraction and a turn

A cube's mapping says how a basic, lambert or phong surface looks into it. CUBE_REFLECTION_MAPPING, the default, reads along reflected(toward_eye, normal). CUBE_REFRACTION_MAPPING reads along refracted(toward_eye, normal, ratio), GLSL's refract, by the material's refraction_ratio. A panorama cube takes the two equirectangular mappings. See Materials.

env_rotation(euler) gives the Basis3 that turns a lookup direction, three.js's envMapRotation uniform. three.js negates the three angles. It flips y and z back for a loaded cube and reads it through flipEnvMap.

This port swapped those images when it loaded them, so every environment turns by one matrix. Basis3.turn applies it on both backends.

CubeTextureStore

assets.cube_textures.add(cube) returns a CubeTextureId. get(id) borrows it. NO_CUBE_TEXTURE is the id of no cube texture. SCENE_ENVIRONMENT is not an id either: a material naming it reflects the scene's environment. See Materials and Scene graph.

Errors

  • A cube needs exactly six faces. Each must hold texels, be square, be the size of the others, and be wrapped CLAMP on both axes.
  • A cube with a panorama needs an equirectangular mapping, and a cube of six images a cube mapping. cube_of_panorama refuses a panorama without an equirectangular mapping.
  • env_rotation refuses an angle that is not finite and an order that is none of the six.
  • cube_texture_from refuses a layout that is neither named value, and an empty image. It refuses a file whose color space cannot be interpreted when none is given.
  • validate() refuses a face edited into nonsense after the cube was built. The GPU upload calls it again.
  • The store refuses NO_CUBE_TEXTURE, SCENE_ENVIRONMENT and any id it does not hold.

examples/mirror.mojo builds a sky from six computed faces, renders a cube camera's view every frame, and reflects it in a chrome ball.

PMREM

render/pmrem.mojo and render/cube_uv.mojo. A PMREM is an environment prefiltered for every roughness. A rough STANDARD or PHYSICAL surface reads it and sees the environment blurred by its own lobe. Without one, the surface reads the cube's box-filtered chain.

three.js: PMREMGenerator.fromCubemap, PMREMGenerator.fromEquirectangular, PMREMGenerator.fromScene, cube_uv_reflection_fragment.

var sky = pmrem_from_cube(cube_texture_from(images, SEEN_FROM_OUTSIDE))
var env = assets.cube_textures.add(sky^)
scene.environment = env
var hdr = assets.cube_textures.add(pmrem_from_equirectangular(panorama))
Builder Meaning
pmrem_from_cube(cube) A copy of cube with its PMREM in cube_uv. Byte or float faces.
pmrem_from_equirectangular(image) The faces of cube_from_equirectangular, with a PMREM read straight from the panorama.
pmrem_from_scene(renderer, scene, assets, sigma=0, near=0.1 m, far=100 m, size=256, position=origin) A scene drawn into six float faces around a point, then prefiltered. See Environments from a scene.
pmrem_from_faces(cube, sigma) pmrem_from_cube with the sharpest copy blurred by sigma first.

The result is an ordinary CubeTexture. Name it as an env map, as a scene's environment or as a background. is_prefiltered() says whether a cube holds a PMREM.

Environments from a scene

pmrem_from_scene in renderers/environment.mojo is three.js's PMREMGenerator.fromScene. It draws the scene six times from position, one face for each axis, into float targets. Light above one stays: a panel that glows at fifty stays at fifty. There is no tone mapping, as in three.js. Where nothing is drawn, a face shows the scene's color background, or else the renderer's background. sigma, an angle, blurs the sharpest copy before the other copies are made.

environments/room_environment.mojo holds two scenes made for it. room_environment(assets) is three.js's RoomEnvironment: a white room with six boxes, six glowing panels and one bulb. debug_environment(assets) is three.js's DebugEnvironment: a room with a red, a green and a blue panel.

var room = room_environment(assets)
var studio = pmrem_from_scene(renderer, room, assets, sigma=Angle(0.04, RADIAN))
scene.environment = assets.cube_textures.add(studio^)

These are the differences from three.js:

  • Each face is a CubeCamera face, and pmrem_from_faces reads it in the direction of each texel. three.js draws straight into the cube UV layout with its own face cameras. The image is the same environment.
  • three.js's blur for sigma keeps the pole of the last blur it ran. Here the pole is +y, the pole the blur starts with.
  • The room keeps the box's texture coordinates. three.js deletes them, and nothing reads them.
  • pmrem_from_faces refuses a sigma that is negative or not finite.

The layout

The PMREM is one float image in three.js's cube UV layout. Each copy of the environment is six square tiles, three across and two up. The sharpest copy is the face size, rounded down to a power of two, at the bottom left. Each copy above it is half the size, down to sixteen texels. Six more sixteen-texel copies sit beside the last one, each blurrier than the one before.

Every tile keeps a one-texel border in the directions of the next face. The bilinear filter never reads a neighbor tile, so a rough reflection has no seam.

How a roughness reads it

roughness_to_mip(roughness) is three.js's table from a roughness to a copy. Roughness one reads the blurriest copy, and a low roughness reads the sharpest. cube_uv_taps finds the two copies on either side and the fraction between them. sample_cube_uv reads both and mixes them, as three.js's textureCubeUV does.

A physical surface asks CubeTexture.sample_rough(direction, roughness). It reads the radiance at its roughness and the irradiance around its normal at roughness one, as three.js's getIBLRadiance and getIBLIrradiance do. A cube without a PMREM answers from its chain at reflection_level. Both rasterizers use this arithmetic. The GPU reads the PMREM from the row after the six faces. See GPU backend.

How it is built

The sharpest copy reads the source in the direction of each texel. Each copy after it is the one before, blurred by a Gaussian on the sphere. The blurs add up to each copy's own width: 1 / size for the halving copies, and three.js's extra_lod_sigma() for the six others. three.js chose those widths to follow the GGX lobe at each roughness.

Each blur is two passes. The first turns about a pole and the second turns toward it. The pole changes each time among ten axes of a dodecahedron, pole_axis. This is three.js's SphericalGaussianBlur, run on the host once.

Each pass finds its weights, the sine and cosine of each tap, and the position of its copy once. Each texel then only turns its direction and reads. The sums run in three.js's order, so the image is the same to the bit. See Benchmarks.

Where it differs from three.js

  • The image holds 32-bit floats. three.js renders half floats.
  • A face smaller than sixteen texels is read at sixteen. three.js's layout does not work below sixteen.
  • A mirror read of a prefiltered cube reads its faces, not the sharpest copy. The two hold the same image.
  • A sharp background reads the faces. A background with a background_blurriness above zero reads the copy at that roughness, as three.js's backgroundBlurriness does. See Scene graph.

Errors

  • pmrem_from_cube refuses a cube that CubeTexture.validate refuses.
  • pmrem_from_equirectangular refuses a blank panorama.
  • pmrem_from_scene refuses what CubeCamera refuses: a size below one, or planes out of order.
  • validate_cube_uv refuses a layout image that is not float, CLAMP and BILINEAR with no chain. It refuses a size that is not three.js's layout. CubeTexture.validate calls it, and so does the GPU upload.

3D textures

render/volume_texture.mojo. A Data3DTexture is a volume of texels, width by height by depth, sampled by three coordinates. It is three.js's Data3DTexture, read as a GLSL sampler3D reads it. A color lookup table is one: see Post-processing.

three.js: Data3DTexture, wrapR, texture and texelFetch on a sampler3D.

var image = VolumeImage.of_floats(16, 16, 16, densities, channels=1)
var volume = Data3DTexture(image^, wrap=CLAMP, filter=BILINEAR)
var id = assets.data_3d_textures.add(volume^)
var color = assets.data_3d_textures.get(id).sample(0.5, 0.5, 0.25)

The image

A VolumeImage holds the texels. It is three.js's texture.image, { data, width, height, depth }. The first coordinate changes fastest, then the second, then the third.

Builder three.js Meaning
VolumeImage.of_bytes(width, height, depth, bytes, channels=4) UnsignedByteType One through four bytes a texel.
VolumeImage.of_floats(width, height, depth, floats, channels=4) FloatType One through four finite floats a texel.

The dimensions must be positive, and their product times four must fit in Int. Builders and validate() reject an overflowing size before multiplying or allocating. The supplied storage must match the dimensions exactly. LUT image loaders also check decoded pixel storage before transposing it.

A texel with fewer than four channels fills red first. A color channel that it does not reach is zero. A missing alpha is one. That is how WebGL samples a RedFormat or an RGFormat texture. The image is stored as RGBA.

The texture

Data3DTexture(image, wrap=CLAMP, filter=NEAREST, color_space=LINEAR) takes three.js's defaults. The fields are image, wrap_s, wrap_t, wrap_r, filter and color_space. The constructor sets all three wrap modes to wrap. Set one field afterward to make the axes differ.

Member GLSL Meaning
sample(s, t, r) -> FloatColor texture(sampler3D, vec3) The color at a coordinate.
texel_fetch(x, y, z) -> FloatColor texelFetch(sampler3D, ivec3, 0) The texel at an index.
validate() Refuse a field edited into nonsense.

Under NEAREST, sample returns the texel that the coordinate lands in. Under BILINEAR, three.js's LinearFilter, it blends the eight texels around the coordinate. Texel centers are at half-integers. Each index is wrapped on its own axis by wrap_s, wrap_t or wrap_r.

A byte is read as a fraction of 255. With SRGB, the three color channels are decoded through the sRGB curve, and alpha is not. A float is read as it is.

Where the 3D textures differ from the 2D texture

  • The rows run up. three.js sets flipY to false for both kinds, and WebGL does not flip a 3D upload. Texel (0, 0, 0) is at the coordinate origin. A 2D Texture reads its first row at the top.
  • The filter is straight. GLSL blends each stored channel on its own, and so does this port. A 2D Texture blends premultiplied, because its alpha is coverage. The fourth channel of a volume is often density or data.
  • There is no mip chain. three.js turns generateMipmaps off for both kinds. One filter serves both magnification and minification.

Array textures

render/volume_texture.mojo. A DataArrayTexture is a stack of images of one size, sampled by two coordinates and a layer number. It is three.js's DataArrayTexture, read as a GLSL sampler2DArray reads it.

three.js: DataArrayTexture, texture and texelFetch on a sampler2DArray.

var image = VolumeImage.of_bytes(64, 64, 8, frames)
var stack = DataArrayTexture(image^, wrap=REPEAT, filter=BILINEAR, color_space=SRGB)
var id = assets.data_array_textures.add(stack^)
var color = assets.data_array_textures.get(id).sample(0.5, 0.5, 3)

The image is a VolumeImage. Its depth is the number of layers. DataArrayTexture(image, wrap=CLAMP, filter=NEAREST, color_space=LINEAR) takes three.js's defaults. The fields are image, wrap_s, wrap_t, filter and color_space.

Member GLSL Meaning
sample(u, v, layer) -> FloatColor texture(sampler2DArray, vec3) The color at a coordinate in one layer.
texel_fetch(x, y, layer) -> FloatColor texelFetch(sampler2DArray, ivec3, 0) The texel at an index.
layers() -> Int image.depth How many layers there are.
validate() Refuse a field edited into nonsense.

array_layer(layer, depth) finds the layer that sample reads. It rounds the coordinate to the nearest whole number, and a half rounds up. Then it holds the result inside the stack. That is the OpenGL rule. A sample never blends two layers. Within the layer, the filter and the wrap modes work as they do for a 3D texture.

The stores

assets.data_3d_textures.add(texture) returns a Data3DTextureId. assets.data_array_textures.add(texture) returns a DataArrayTextureId. get(id) borrows the texture. NO_DATA_3D_TEXTURE and NO_DATA_ARRAY_TEXTURE are the ids of no texture.

The two ids are types. A bare number does not compile. tests/compile_fail/ proves it.

Errors of the 3D and array textures

  • of_bytes and of_floats refuse a dimension that is not positive. They refuse a channel count outside one through four, and a length that does not match.
  • of_floats refuses a number that is not finite.
  • validate refuses a wrap mode, a filter, a color space or a texel type that is not named. It refuses UNKNOWN_SPACE, and a float image that is not LINEAR. It refuses an image whose length does not match its dimensions.
  • texel_fetch refuses an index outside the image. GLSL leaves that result undefined.
  • The stores refuse the "no texture" ids and any id that they do not hold.

Not ported in the 3D and array textures

  • A shader material reads neither kind, because a node program has no sampler3D and no array sampler. three.js reads them from a custom shader or from LUTPass. The LUT pass is ported, and GpuComposer also runs it on the device. See Post-processing.
  • layerUpdates, addLayerUpdate and unpackAlignment control the upload to WebGL. They have no counterpart here.
  • Mip chains, separate magFilter and minFilter, and the formats other than red, RG, RGB and RGBA.

Wrap

Each axis has its own wrap, as in three.js. wrap_s is three.js's wrapS, across. wrap_t is wrapT, up. The constructors take one wrap and set both. set_wrap(mode) sets both after construction.

Value three.js A coordinate of 1.5 reads
REPEAT RepeatWrapping The same texel as 0.5.
CLAMP ClampToEdgeWrapping The edge texel.
MIRROR MirroredRepeatWrapping The image reflected, so tiles meet without a seam. MIRRORED_REPEAT is the same value.

Under REPEAT, coordinates 0 and 1 name the same texel. Under CLAMP they name opposite edges.

Filter

A texture has two filters, as in three.js. mag_filter, three.js's magFilter, reads a magnified sample. min_filter, three.js's minFilter, reads a minified sample.

Value three.js Inside a level Level
NEAREST NearestFilter The texel the sample lands in. The full-size image.
BILINEAR LinearFilter The four nearest texels, blended by distance. The full-size image.
NEAREST_MIPMAP_NEAREST NearestMipmapNearestFilter The nearest texel. The nearest level.
LINEAR_MIPMAP_NEAREST LinearMipmapNearestFilter The four nearest texels. The nearest level.
NEAREST_MIPMAP_LINEAR NearestMipmapLinearFilter The nearest texel. The two nearest levels, mixed.
LINEAR_MIPMAP_LINEAR LinearMipmapLinearFilter The four nearest texels. The two nearest levels, mixed.

mag_filter must be NEAREST or BILINEAR, because a magnified sample reads one level. validate refuses a mipmap filter there. min_filter can be any of the six.

plan_levels(level, levels, mag_filter, min_filter) picks the levels and the filter inside them. At a level of zero or below, the sample is magnified and reads the full-size image through mag_filter. Above zero, min_filter decides. The nearest level is OpenGL's ceil(level + 0.5) - 1. Both rasterizers call plan_levels, and the kernel reads both filters from the texture table.

OpenGL changes from magnification to minification at 0.5 for a linear magFilter and a nearest-mipmap minFilter. This port changes at zero for every pair.

Mipmaps

mipmapped=True builds a chain of halved copies down to one texel. Sampling then picks the level whose texels match the pixel footprint. This stops a distant surface from shimmering.

The chain is built by default, as three.js's Texture sets generateMipmaps. The constructors take one filter and set mag_filter to it. minifying(filter, mipmapped) gives the min_filter.

BILINEAR with a chain is LINEAR_MIPMAP_LINEAR, three.js's default. NEAREST with a chain is NEAREST_MIPMAP_LINEAR. Without a chain, min_filter is the filter itself. Pass mipmapped=False or NEAREST to turn either off.

A mipmap min_filter on a texture without a chain reads its one image, with the filter's own rule inside it.

The chain costs a third more memory. It is built in premultiplied linear light, unless the texture ignores its alpha.

Color space

Value Meaning
SRGB An image. Bytes are decoded to linear light when read. The default.
LINEAR Data. Bytes are used as stored.
UNKNOWN_SPACE A decoder's answer for a file it cannot interpret. A texture refuses it.

Alpha is never decoded. It is coverage, not color.

Alpha

Value Meaning
COVERAGE Alpha hides color. Filtering and the mip chain weight each texel by it, and a sample carries it. The default.
IGNORED Alpha is not read. Every alpha byte counts as 255. Color is filtered as it is, and every sample is opaque.

An emissive map must ignore its alpha. Its alpha is not coverage. Filtered as coverage, a white texel with alpha zero turns black under BILINEAR, and the whole mip chain darkens with it. The renderer refuses an emissive map built with COVERAGE.

An alpha map must ignore its alpha too, and must be LINEAR. Its green channel is a coverage rather than a color. See Materials.

ignoring_alpha() copies a texture into the other mode and rebuilds its mip chain from the full-size image. Use it when one image is both a base map and an emissive map. The blank texture's copy is blank and ignores its alpha, so it passes as an emissive map and samples as white.

Coordinates

u runs from left to right. Rows in memory run from the top. flip_y, three.js's flipY, says which way v runs.

flip_y v of zero reads Default for
True The bottom row. An image, as in three.js.
False The first row. A glTF texture, a compressed texture and a KTX2 texture, as in three.js.

The texels do not move: row_coordinate(v, flip_y) gives the row a v lands on, and both rasterizers call it. three.js flips the image on upload instead, with the same result.

Mapping

mapping, three.js's Texture.mapping, says how a texture is laid over what reads it. Its value is three.js's constant.

Value three.js Read
UV_MAPPING UVMapping (300) By a surface's texture coordinates. The default.
CUBE_REFLECTION_MAPPING CubeReflectionMapping (301) Six faces, along the reflected view.
CUBE_REFRACTION_MAPPING CubeRefractionMapping (302) Six faces, along the refracted view.
EQUIRECTANGULAR_REFLECTION_MAPPING EquirectangularReflectionMapping (303) A panorama, along the reflected view.
EQUIRECTANGULAR_REFRACTION_MAPPING EquirectangularRefractionMapping (304) A panorama, along the refracted view.
CUBE_UV_REFLECTION_MAPPING CubeUVReflectionMapping (306) A PMREM's layout, at a roughness.

A map ignores its texture's mapping, as in three.js. A panorama with an equirectangular mapping can be a background or an environment. See An equirectangular environment or background. render.pmrem gives its layout CUBE_UV_REFLECTION_MAPPING.

channel, three.js's Texture.channel, says which set of the geometry's coordinates a texture reads. UV_CHANNEL_0, the default, reads uv. UV_CHANNEL_1 reads uv1. Any map can read either set. See Light map.

Transform

A texture can move, tile and turn on a surface. three.js: offset, repeat, rotation, center, matrix.

Field Default Meaning
offset Vector2(0, 0) How far the coordinates move, after the rest.
repeat Vector2(1, 1) How many times the texture fits across each axis.
rotation zero How far the image turns, counter-clockwise. An Angle.
center Vector2(0, 0) The point the turn and the scale are about.

Set the fields after construction, as in three.js. uv_transform() returns the matrix they make. See Math for its order.

Each map is sampled at its own coordinate, as in three.js. The texture's channel picks the raw pair, and its matrix moves that pair. The texture itself does not change. The wrap mode still decides what a coordinate past the edge reads. ignoring_alpha() copies the transform and the channel.

placement() returns a UvPlacement: the channel and the top two rows of the matrix. place(uv, uv1) gives the coordinate where the map is sampled.

Fit a texture to a surface

render/texture_utils.mojo sets repeat and offset so that the image keeps its aspect ratio on a surface of another. three.js's TextureUtils.contain, cover and fill, which are CSS's object-fit.

Function Meaning
contain(texture, aspect) Show the whole image, centered. The wrap mode decides what shows beside it.
cover(texture, aspect) Fill the surface, centered. The image is cropped.
fill(texture) Stretch the image over the surface: a repeat of one and no offset.

aspect is the surface's width over its height. The blank texture has an aspect of one, as three.js's texture with no image does. An aspect that is not a positive finite number is refused. three.js takes it and gives a fit that is not a number.

var poster = assets.textures.get(id)          # a 400 by 200 image
cover(poster, 1.5)                            # repeat (0.75, 1), offset (0.125, 0)

How the rasterizers place a map

The corners carry the geometry's two raw pairs, uv and uv1. A geometry with no uv1 carries its uv twice. Each fragment interpolates both pairs, and each map moves its own pair by its own matrix. The matrix is linear, so this is the same as moving each corner and then interpolating, to the rounding. three.js moves each corner in the vertex shader, one varying for each map.

The CPU rasterizer builds the placements once for each triangle. The GPU reads them from the texture table. The last seven columns of a texture's row are its six matrix numbers, bit for bit, and its channel. A triangle carries only its map ids, so no lane and no state column changes. See GPU backend.

A map's mip level and a normal map's tangent frame come from that map's own coordinates, one pixel right and one pixel up. The anisotropy frame uses the normal map's coordinates, or the clear coat normal map's, or the raw uv, as three.js does.

var board = checkerboard(64, 8, white, blue)
board.repeat = Vector2(4, 4)
board.rotation = Angle(45.0, DEGREE)
board.center = Vector2(0.5, 0.5)
var id = assets.textures.add(board^)

Members

Member Meaning
data_texture(width, height, numbers, channels) A texture from fractions. See From numbers.
texture_of(framebuffer), depth_texture_of(framebuffer) A texture from a render, and from its depth. See From a render.
sample(u, v) -> FloatColor The color at a coordinate, level zero, through mag_filter.
sample_level(u, v, level) -> FloatColor The color at a fractional level, through both filters. See Filter.
sample_at(u, v, level) -> FloatColor One named level, through filter_at(level).
texel(x, y) -> Color One stored texel. A float texture refuses it.
wrapped_texel(x, y, level=0) -> FloatColor One texel as light, wrapped. Floats as they are, bytes through the ramp.
is_blank() -> Bool The blank texture, which samples as opaque white.
ignoring_alpha() -> Texture A copy that ignores its alpha, with its chain rebuilt.
uv_transform() -> Matrix3 The transform on the coordinates, from the four fields above.
placement() -> UvPlacement The channel and the matrix, as the rasterizers read them.
sample_footprint(u, v, footprint) -> FloatColor One trilinear sample, or several along a footprint's long axis. See Anisotropy.
validate() Refuse a wrap, filter, mapping, color space, alpha mode, texel type or channel that is none of the named values, a mipmap mag_filter, a float texture that is not LINEAR, or an anisotropy below one or above MAX_ANISOTROPY.
wrap_s, wrap_t, set_wrap(mode) The wrap across and up. See Wrap.
mag_filter, min_filter The two filters. See Filter.
flip_y, mapping Which way v runs, and how the texture is laid. See Coordinates and Mapping.
levels, width, height, alpha, anisotropy The chain length, the base size, the alpha mode and the tap count.
channel Which coordinates the texture reads: UV_CHANNEL_0 or UV_CHANNEL_1.
texel_type, pixels, data UNSIGNED_BYTE_TYPE with bytes in pixels, or FLOAT_TYPE with floats in data. See HDR images.
offset, repeat, rotation, center The transform's fields.

Byte length

byte_length(width, height, format, type) is three.js's TextureUtils.getByteLength: how many bytes an image takes in GPU memory. TextureFormat and TextureDataType are three.js's format and type constants, with three.js's numbers as their values. TextureFormat.RGBA is RGBAFormat, 1023. TextureDataType.HALF_FLOAT is HalfFloatType, 1016.

The result is a Float64, because three.js's result is not always a whole number. A red image of UNSIGNED_SHORT_4444 texels is half a byte for each texel. The function refuses a negative size, a format or type that is none of the named values, and UNSIGNED_INT_248, which three.js's getByteLength does not know.

Half floats

render/data_utils.mojo is three.js's DataUtils. to_half_float(x) -> UInt16 and from_half_float(bits) -> Float32 give three.js's bits, bit for bit. The tests build three.js's conversion tables and compare every half, and every float exponent.

to_half_float clamps to 65504, rounds to a Float32, and then cuts the extra fraction bits. It does not round them. So 1.0007 gives 0x3C00, which is one, although 0x3C01 is nearer. An infinity gives the largest half. A NaN gives 0xFE00, which is what three.js gives in Node on x86-64. from_half_float is exact, because a Float32 holds every half.

TextureStore

assets.textures.add(texture) returns a TextureId. get(id) borrows it. NO_TEXTURE is the id of no texture, and samples as white.

Errors

  • Dimensions must be positive, and the buffer length must match.
  • A wrap, filter, mapping, color space, alpha mode or channel that is none of its named values raises. A mipmap mag_filter raises. The GPU upload checks again.
  • A checkerboard size must divide evenly by its square count.
  • A data_texture with a channel count outside one through four, a length that does not match, or a number that is not finite.
  • A float_texture with a length that does not match, or a number that is not finite. A float texture that is not LINEAR, or a texel type that is none of the two, raises in validate.
  • The RGBE and EXR readers refuse a file cut short, a malformed header, and each feature in What is not ported.
  • An anisotropy below one, or above MAX_ANISOTROPY, raises in validate.
  • A Footprint with no taps, or a non-finite level or step, raises in sample_footprint.
  • See Cube textures for what a cube refuses, and KTX2 and compressed formats for what a compressed payload or container refuses.

Why

See Why mipmaps and Why color is linear.

Shared texture payloads

Texture(copy=source) keeps an independent deep copy of the texels and mip levels. Use Texture(copy=source, share_data=True) to share the complete byte and float payload allocations. Sampling reads those allocations without a copy. The first write to a shared payload takes a private copy. Replacing or destroying the source does not invalidate another texture that shares its payload.

Each texture keeps its own repeat, offset, center, rotation, wraps, filters, color-space state, lookup ramp, mip offsets and other sampling metadata. Changes to these fields do not change another texture. Changing a base texel does not rebuild the mip chain. Call regenerate_mipmaps when required, as before.

The public pixels and data fields use TextureBuffer, not List. Indexed reads and writes, len, append, resize, and copy remain available. Use values() when a function needs a borrowed read-only List. Use copy() when it needs an independently owned List. Use mutable_values() for a writable List, Span, or pointer. That call detaches the payload before it returns.

A buffer that exposed a writable List keeps that state. Any later shared copy of that buffer takes an independent snapshot. A retained writable alias cannot change the new snapshot. The direct unsafe_ptr() is read-only. Do not cast a read-only pointer to a writable pointer or keep a pointer after its owner dies.