Virtual Reality Performance Test

Description

Virtual Reality has been upcoming for the last few years and a lot of different VR-headsets have been released like the Oculus Rift, HTC Vive and the PlayStation VR. A VR-headset contains two displays, one for each eye. This means that every object is rendered twice. To get a smoothly running game a lot of optimization is required. The oculus rift display refreshes at 90 frames per second (FPS). The high refresh rate is needed to prevent users from getting motion-sick (Eassa Kidwell, 27 April 2018).

With VR, every frame must be typically drawn twice, once for each eye. That typically means that every draw call is issued twice, every mesh is drawn twice, and every texture is bound twice. Since the Rift refreshes frames at 90 Hz, it can be challenging to hit the frame rate consistently (Oculus Developers, n.d)

The two biggest performance reducers of VR are:

• The number of draw calls
• The number of triangles in a scene

The implementation of the Oculus integration is done by this tutorial and the implementation of SteamVR by the documentation

Evaluation Proposal: Compare in VR (Oculus Rift) the game performance of different objects and scene setups.

• Oculus Developers - Guidelines for VR Performance Optimization https://developer.oculus.com/documentation/native/pc/dg-performance-guidelines/
• Eassa Kidwell (27 April 2018) - How to avoid motion sickness caused by VR headsets while gaming. Retrieved from https://www.windowscentral.com/how-avoid-motion-sickness-caused-by-vr-headsets-while-gaming
• Frontiers in Psychology (6 November 2018). Retrieved from: https://doi.org/10.3389/fpsyg.2018.02086
• Unity 3D - Olly’s seven stages of optimizations for mobile VR. Retrieved from https://unity3d.com/how-to/optimize-mobile-VR-games

Performing the tests

To make the testing a lot easier and to get more reliable results the testing is automated. Inside the project, there's a Test Runner with a queue that runs multiple tests in a sequence and exports the test results to Excel sheets (read more about the automated test process).

The headset that's being used in the tests is the Oculus Rift.

For the test, the following setup is used:

• CPU: Intel Core i7-7820X
• GPU: Asus ROG STRIX-GTX1080-A8G-GAMING
• RAM: 32GB (4 x 8) 3000mhz Corsair
• MOBO: Asus ROG Strix X299-E Gaming

The system is water-cooled by an NZXT Kraken X62 but all overclocking settings are turned off to factory defaults for the tests.

Comparing

Abreviations used: k = thousand / 1000, M = Million / 1.000.000

In total 130 different setups have been tested. The tests can be separated into 3 categories: test with 1k tris/mesh, 10k tris/mesh and 100k tris/mesh. In each category, we first start testing with a low number of total triangles and keep scaling up the amount until the FPS is dropped drastically.

Test results in 3 categories

The graph shows the 3 different categories and how their average frame rate goes down while the number of triangles in the scene is scaled up.

Analysis

It's really interesting to see that each graph has a breakpoint where the average FPS immediately goes down to ~43 FPS. This is caused by the Headsets that synchronize the refresh rate. When a frame takes longer than 11.1ms (90 fps) to process it skips 1 frame and then released the rendered frame in the next update. This makes the refresh rate switch to 45 FPS. If you profile (deep profiler) the performance in Unity the "XR.WaitForGPU" event is called every time it synchronizes the update rate.

This synchronization is done one purpose, it makes the VR-experience more bearable for the user. When the FPS would scale down unevenly from 90FPS to for example 80FPS it would change the refresh speed, this can cause motion sickness. The FPS now goes down wrong 90FPS to 45FPS which is at the same speed, but at a different rate.

Graph zoomed in

The graph above is the same graph as the first one but zoomed in. The breakpoints where the average FPS drop to 45 is more clear on this graph.

FPS Breakpoints

This chart shows at what points the average FPS drops below 90 and 45. The 1k tris/mesh category drops below 90fps at 4M tris but it has a lot of objects. The 100k tris/mesh category drops below 90 at 16M but only has 160 objects.

To get the most FPS in a scene it's important to find a balance between the numbers of objects and the number of triangles per object. More objects = more draw calls and that's CPU heavy. More triangles are more GPU heavy.

Test results in table

The table above shows all the results of the 130 tests that are used to generate the graphs and charts.