What do the different material per cell options do? - pennyworth12345/A3_MMSI GitHub Wiki
(Credit to ianbanks for most of the info)
When your terrain is being rendered, the engine attempts to to send an entire land grid cell to a single DirectX call. Each DirectX call can have up to 16 textures sent to the pixel shader. The textures being passed to the DirectX call can be seen in your RVMATs in the various Stages, e.g. Stage0, Stage5, etc.
4 Materials Per Cell
In Arma 1 terrains you could have 4 surfaces max per tile, with three texture types for each individual surface, and they were mco, nopx/nohq, co. The mask tiles produced by Visitor would only have 4 colors, black: (0, 0, 0), red, green, and blue. The RVMAT generated by Visitor uses the "Terrain#" pixel shader, where # is some number 1-15 based on the number of surfaces on the tile, and they represents a value for a 4 bit number. It might look something like this for Terrain15:
Stage0: satellite tile
Stage1: mask tile
Stage2: surface 1 mco
Stage3: surface 1 nopx/nohq
Stage4: surface 1 co
Stage5: surface 2 mco
Stage6: surface 2 nopx/nohq
Stage7: surface 2 co
Stage8: surface 3 mco
Stage9: surface 3 nopx/nohq
Stage10: surface 3 co
Stage11: surface 4 mco
Stage12: surface 4 nopx/nohq
Stage13: surface 4 co
Putting the total count of textures used for this DirectX call to 14 (because it starts from Stage0, not Stage1).
6 Materials Per Cell
Arma 2 introduced having up to 6 surfaces per tile using the alpha channel in mask tiles, but at the cost of losing mco textures. So now each surface can only use 2 textures, nopx and co. Once Visitor was updated to the point of being able to use 6 surfaces per tile, your generated mask tiles would use the black, red, green, blue, 128 alpha, and 0 alpha. There are some gimmicks to how it handles the surfaces with the alpha channels, but those are unnecessary details for this. The RVMAT generated by Visitor/TB uses the "TerrainX" pixel shader, and it might look something like this:
Stage0: satellite tile
Stage1: mask tile
Stage2: mco texture used by every surface, for example: #(rgb,1,1,1)color(0.5,0.5,0.5,1,cdt)
Stage3: surface 1 nopx
Stage4: surface 1 co
Stage5: surface 2 nopx
Stage6: surface 2 co
Stage7: surface 3 nopx
Stage8: surface 3 co
Stage9: surface 4 nopx
Stage10: surface 4 co
Stage11: surface 5 nopx
Stage12: surface 5 co
Stage13: surface 6 nopx
Stage14: surface 6 co
Bringing us to a total of 15 textures used for this DirectX call (because it starts from Stage0, not Stage1).
5 Materials Per Cell
Arma 3 introduced the ability to use a normal map for your terrain, but at the cost of one surface per tile. So now you can only have 5 surfaces per tile, but with the added benefit of a normal map. Each surface still only has 2 textures, nopx and co. When using a normal map, your mask tiles generated by TB would use the black, red, green, blue, and 128 alpha. Again with some gimmicks for the alpha channel. The RVMAT uses the "TerrainSNX" pixel shader, and it might look like this:
Stage0: satellite tile
Stage1: mask tile
Stage2: mco texture used by every surface, for example: #(rgb,1,1,1)color(0.5,0.5,0.5,1,cdt)
Stage3: surface 1 nopx
Stage4: surface 1 co
Stage5: surface 2 nopx
Stage6: surface 2 co
Stage7: surface 3 nopx
Stage8: surface 3 co
Stage9: surface 4 nopx
Stage10: surface 4 co
Stage11: surface 5 nopx
Stage12: surface 5 co
Stage14: normal map tile
So TB skips Stage13 all together, and instead places the normal map for the tile on Stage14. This adds up to a grand total of 14 textures used for this DirectX call.
It would appear that BI decided when moving from A1 to A2 that the benefit from mco textures was outweighed by having the option of 6 surfaces per tile. Then from A2 to A3 they decided it was worth it to only have 5 surfaces per tile, but with the added benefit of having a normal map for each tile.