SurfaceTexture Transforming Matrix - tonykwok/made-mistakes-again GitHub Wiki

SurfaceTexture.getTransformMatrix(float[])

Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by the most recent call to SurfaceTexture.updateTexImage.

This transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample that location from the texture. Sampling the texture outside of the range of this transform is undefined.

The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via the glLoadMatrixf or glUniformMatrix4fv functions.

m[offset +  0] m[offset +  4] m[offset +  8] m[offset + 12]
m[offset +  1] m[offset +  5] m[offset +  9] m[offset + 13]
m[offset +  2] m[offset +  6] m[offset + 10] m[offset + 14]
m[offset +  3] m[offset +  7] m[offset + 11] m[offset + 15]

If the underlying buffer has a crop associated with it, the transformation will also include a slight scale to cut off a 1-texel border around the edge of the crop. This ensures that when the texture is bilinear sampled that no texels outside of the buffer's valid region are accessed by the GPU, avoiding any sampling artifacts when scaling.

Remember that you are transforming the texture coordinates, not the texture image. Transforming the texture image forward is the same as transforming the texture coordinates backwards.

 0.0, -1.0,  0.0,  1.0,
-1.0,  0.0,  0.0,  1.0,
 0.0,  0.0,  1.0,  0.0,
 0.0,  0.0,  0.0,  1.0

Matrix.setIdentityM(matrix, 0);
Matrix.translateM(matrix, 0, 1f, 1f, 0);
Matrix.scaleM(matrix, 0, 1, -1, 1);
Matrix.rotateM(matrix, 0, 90, 0, 0, 1);3

 0.0,  1.0,  0.0,  0.0,
-1.0,  0.0,  0.0,  1.0,
 0.0,  0.0,  1.0,  0.0,
 0.0,  0.0,  0.0,  1.0

Matrix.setIdentityM(matrix, 0);
Matrix.translateM(matrix, 0, 0, 1, 0);
Matrix.rotateM(matrix, 0, 270, 0, 0, 1);3

¹ According to Android 10 Compatibility Definition 7.5.5 Camera Orientation⁠, front and rear-facing cameras "MUST be oriented so that the long dimension of the camera aligns with the screen’s long dimension.".

² The matrix is stored in column-major order so that it may be passed directly to OpenGL ES via the glLoadMatrixf or glUniformMatrix4fv functions.

³ If you look from the positive end towards the origin of the axis the positive rotation is counter-clockwise.