Debugging - dekay/vpinball-wiki GitHub Wiki
- Useful Git Commands
- Using Git Bisect to Identify Problematic Commits
- Compiling in Debug Mode
- Setting up Renderdoc
VPinball is hosted on Github and, as you would expect, uses git to manage its version history. Here's a few useful commands to help you get the code and navigate around.
- Get the code. By default it is saved to a
vpinballsubdirectory of your current directory. You'll need to havegititself installed on your machine first of all, of course.
git clone https://github.com/vpinball/vpinball.git
- Display the SHA1 hash of the current commit in the repository
$ git rev-parse HEAD
- Alternatively, retrieve a shortened version of the SHA1 hash (typically 7 characters by default)
$ git rev-parse --short HEAD
- Switch to a specific commit
$ git checkout <commithash>
- Switch back to
HEAD
$ git checkout master
- You've really screwed up somehow and want to nuke any changes you've made to the repo. WARNING: any changes you might have made that aren't committed will be blown away as well. Use with caution.
$ git reset --hard HEAD
This section shows by example how a problem with messed up graphics in Total Nuclear Annihilation was traced down to the exact commit that introduced the issue. It was done on Linux but the process would be similar on all platforms. If you are new to bisecting on git, the git-scm page on bisecting is an excellent resource.
First off, a build from about a month prior to the debug session was tested to see if that would show the problem or not. Git is basically a time machine, so we can step back to any point in time in the commit history with git checkout <sha>. The commits made to the VPinball repository show the SHA1 hash for every change. We are interested in turning back time to commit 0edea9521a842b8fac901cf2b7c0cf0c17a6a04f, like so...
$ git checkout 0edea9521a842b8fac901cf2b7c0cf0c17a6a04f
Tip
The short version of the SHA1 has can also be used when bisecting, so git checkout 0edea952 would also work.
Compile and run as usual. Note that we always want to run the platforms/linux-x64/external.sh script in case any of the third party libraries have changed along the way.
$ platforms/linux-x64/external.sh
$ cp make/CMakeLists_bgfx-linux-x64.txt CMakeLists.txt && cmake -DCMAKE_BUILD_TYPE=Release -B build && cmake --build build -- -j$(nproc)
$ ./build/VPinballX_BGFX -play <path to table>
That worked and gave good graphics in TNA. We'll prepare for bisecting by going to the HEAD of the repo where we know the graphics aren't rendering properly, double check that we're on the master branch, and make sure we're up to date before continuing.
[dk@ryzen vpinball]$ git checkout master
Previous HEAD position was 0edea9521 tiny leftovers
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
[dk@ryzen vpinball]$ git branch
* master
[dk@ryzen vpinball]$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
Let's start bisecting...
[dk@ryzen vpinball]$ git bisect start
status: waiting for both good and bad commits
[dk@ryzen vpinball]$ git bisect bad # Current version is bad
status: waiting for good commit(s), bad commit known
[dk@ryzen vpinball]$ git bisect good 0edea9521a842b8fac901cf2b7c0cf0c17a6a04f
Once you have specified at least one bad and one good commit, git bisect selects a commit in the middle of that range of history, checks it out, and outputs something similar to the following.
Bisecting: 66 revisions left to test after this (roughly 6 steps)
[1930ce1ad163721f6e7948e5941feed33f17abf8] Fix ImgUI font texture
It's working! git set its time machine in the middle of the range of changes. At each step, compile and run as usual,
$ platforms/linux-x64/external.sh
$ cp make/CMakeLists_bgfx-linux-x64.txt CMakeLists.txt && cmake -DCMAKE_BUILD_TYPE=Release -B build && cmake --build build -- -j$(nproc)
$ ./build/VPinballX_BGFX -play <path to table>
and let git bisect know if the results are bad or good. In this case, the problem was still there so we tell git it is bad. Rinse and repeat...
[dk@ryzen vpinball]$ git bisect bad
Bisecting: 32 revisions left to test after this (roughly 5 steps)
[6ab342b50334b632c43b33326b4df0049397b9a6] Fix backglass missing mipmaps
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect bad
Bisecting: 16 revisions left to test after this (roughly 4 steps)
[d363dd418887b096c0ed9fff92b651d35b97e8cf] remove CString def for standalone
Unfortunately, VPinball didn't build at this commit on Standalone. It bombed with...
/home/dk/build/vpinball/src/parts/pintable.cpp: In member function ‘virtual HRESULT ScriptGlobalTable::LoadValue(BSTR, BSTR, VARIANT*)’:
/home/dk/build/vpinball/src/parts/pintable.cpp:734:4: error: ‘else’ without a previous ‘if’
734 | else
| ^~~~
/home/dk/build/vpinball/src/parts/pintable.cpp: In member function ‘VPX::Sound* PinTable::ImportSound(const std::string&)’:
/home/dk/build/vpinball/src/parts/pintable.cpp:4252:1: warning: no return statement in function returning non-void [-Wreturn-type]
4252 | }
| ^
[ 30%] Building CXX object CMakeFiles/vpinball.dir/standalone/inc/b2s/collections/ZOrderCollection.cpp.o
<snip>
make[2]: *** [CMakeFiles/vpinball.dir/build.make:563: CMakeFiles/vpinball.dir/src/parts/pintable.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:123: CMakeFiles/vpinball.dir/all] Error 2
In a case like this you can do a git bisect visualize. This lets you select an adjacent commit where it is likely to start working again.
The above suggests that 63586d0bb28f04919d429661cb79a0c18fbd1629 should have the fix. So...
[dk@ryzen vpinball]$ git reset --hard 63586d0bb28f04919d429661cb79a0c18fbd1629
HEAD is now at 63586d0bb fix warnings and standalone compile error
<compile and run as usual>
That got us back in business
[dk@ryzen vpinball]$ git bisect good
Bisecting: 7 revisions left to test after this (roughly 3 steps)
[23d073be433284956c05f238dd7cbc0912a29d0c] d'oh
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect good
Bisecting: 4 revisions left to test after this (roughly 2 steps)
[11c0c18314d7e7e01aea3419521e5dd0d5be7848] B2S: add mech rotation support
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect good
Bisecting: 2 revisions left to test after this (roughly 1 step)
[2ea259436b3dde520dfb3c4cd52e94d6d28ef332] Merge pull request #2560 from teamsuperpanda/master
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect good
Bisecting: 0 revisions left to test after this (roughly 1 step)
[666c939e752eab18e6a66b1c58ef36ba8084508c] Share VPX scripted DMD (main & flashers) through plugin API
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect bad
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[47be039c11eb7693f85eb26088c9e6d7048a5921] BGFX: add preliminary DX12 mipmap generation using a compute shader
<compile and run as usual>
[dk@ryzen vpinball]$ git bisect bad
47be039c11eb7693f85eb26088c9e6d7048a5921 is the first bad commit
commit 47be039c11eb7693f85eb26088c9e6d7048a5921 (HEAD)
Author: Vincent Bousquet <[email protected]>
Date: Sun Jul 6 00:56:01 2025 +0200
BGFX: add preliminary DX12 mipmap generation using a compute shader
src/parts/flasher.cpp | 6 +
src/parts/light.cpp | 7 +
src/renderer/RenderDevice.cpp | 12 +-
src/renderer/RenderDevice.h | 4 +
src/renderer/RenderTarget.cpp | 12 +-
src/renderer/RenderTarget.h | 3 +-
src/renderer/Sampler.cpp | 181 ++++-
src/renderer/Sampler.h | 3 +-
src/renderer/Shader.cpp | 4 +-
src/renderer/VRDevice.cpp | 2 +-
src/shaders/bgfx/bgfx_compute.sh | 327 ++++++++
src/shaders/bgfx/cs_mipmap.sc | 97 +++
src/shaders/bgfx/shaders.ps1 | 11 +
src/shaders/bgfx_mipmap.h | 2386 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
third-party/include/bgfx-imgui/imgui_impl_bgfx.cpp | 2 +-
15 files changed, 3004 insertions(+), 53 deletions(-)
create mode 100644 src/shaders/bgfx/bgfx_compute.sh
create mode 100644 src/shaders/bgfx/cs_mipmap.sc
create mode 100644 src/shaders/bgfx_mipmap.h
The problem has been found! At this point git bisect log can be used to summarize the steps taken to find the problem.
[dk@ryzen vpinball]$ git bisect log
git bisect start
# status: waiting for both good and bad commits
# bad: [d5a42f01e9dc774a24be7050521b68975b22beed] fix standalone builds
git bisect bad d5a42f01e9dc774a24be7050521b68975b22beed
# status: waiting for good commit(s), bad commit known
# good: [0edea9521a842b8fac901cf2b7c0cf0c17a6a04f] tiny leftovers
git bisect good 0edea9521a842b8fac901cf2b7c0cf0c17a6a04f
# bad: [1930ce1ad163721f6e7948e5941feed33f17abf8] Fix ImgUI font texture
git bisect bad 1930ce1ad163721f6e7948e5941feed33f17abf8
# bad: [6ab342b50334b632c43b33326b4df0049397b9a6] Fix backglass missing mipmaps
git bisect bad 6ab342b50334b632c43b33326b4df0049397b9a6
# good: [63586d0bb28f04919d429661cb79a0c18fbd1629] fix warnings and standalone compile error
git bisect good 63586d0bb28f04919d429661cb79a0c18fbd1629
# good: [23d073be433284956c05f238dd7cbc0912a29d0c] d'oh
git bisect good 23d073be433284956c05f238dd7cbc0912a29d0c
# good: [11c0c18314d7e7e01aea3419521e5dd0d5be7848] B2S: add mech rotation support
git bisect good 11c0c18314d7e7e01aea3419521e5dd0d5be7848
# good: [2ea259436b3dde520dfb3c4cd52e94d6d28ef332] Merge pull request #2560 from teamsuperpanda/master
git bisect good 2ea259436b3dde520dfb3c4cd52e94d6d28ef332
# bad: [666c939e752eab18e6a66b1c58ef36ba8084508c] Share VPX scripted DMD (main & flashers) through plugin API
git bisect bad 666c939e752eab18e6a66b1c58ef36ba8084508c
# bad: [47be039c11eb7693f85eb26088c9e6d7048a5921] BGFX: add preliminary DX12 mipmap generation using a compute shader
git bisect bad 47be039c11eb7693f85eb26088c9e6d7048a5921
# first bad commit: [47be039c11eb7693f85eb26088c9e6d7048a5921] BGFX: add preliminary DX12 mipmap generation using a compute shader
Our work here is almost done. About all that's left is to file an issue on Github to guide the developers to a fix. Once that's done, we can tell git we're done with bisecting.
[dk@ryzen vpinball]$ git bisect reset
Previous HEAD position was 47be039c1 BGFX: add preliminary DX12 mipmap generation using a compute shader
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
The VPinball docs include a page on how to build the software. Once you have cloned the software and installed any required dependencies, it basically boils down to just a handful of steps to build the latest version of the software at any time. From within your vpinball directory and assuming Linux x64...
$ git pull
$ platforms/linux-x64/external.sh
$ cp make/CMakeLists_bgfx-linux-x64.txt CMakeLists.txt && cmake -DCMAKE_BUILD_TYPE=Release -B build && cmake --build build -- -j$(nproc)
If you want to compile the software in Debug mode to enable low level debugging with something like gdb, you'll need to specify a Debug version of the build for both the third-party libraries and VPinball itself.
$ git pull
$ BUILD_TYPE=Debug ./platforms/linux-x64/external.sh
$ cp make/CMakeLists_bgfx-linux-x64.txt CMakeLists.txt && cmake -DCMAKE_BUILD_TYPE=Debug -B build && cmake --build build -- -j$(nproc)
Once the Debug version is built, it can be run under a debugger like gdb e.g.
[dk@ryzen vpinball]$ gdb --args ./build/VPinballX_BGFX -play "/bigdisk/dk/games/vpinball/AC-DC LUCI Premium VR (Stern 2013)/AC-DC LUCI Premium VR (Stern 2013) v1.1.3.vpx"
Renderdoc is a powerful tool for analyzing the low level details of how VPinball constructs the images it displays to players. Because of the variety of platforms VPinball runs on, the various backends supports on those platforms, and the diversity of hardware out there, bugs can creep in that are seen on some machines but not others.
The installation of Renderdoc varies by platform. Prebuilt installers for Windows are available from the Renderdoc website. It is available pre-packaged for many Linux distributions so details will vary between one and the other. For example, Arch users can install the Renderdoc package via
$ sudo pacman -Sy renderdoc
and run the GUI version via the included qrenderdoc program.
Launch qrenderdoc and fill in the "Launch Application" tab as shown below, using directory locations appropriate to your setup. Usually Renderdoc is set up to capture frames with the F12 or Print Screen key. Unfortunately the former also brings up VPinball's LiveUI by default, and the latter is often set up to trigger screenshots in your Desktop Environment of choice. You can fiddle with that or you can do as shown below, which is to "Queue Capture" for a given frame number. That has the advantage that repeated runs will always capture the exact same frame so you can directly compare between them if a fix is made between one capture and another.
At time of writing and on Linux, running VPinball 10.8.1 and above under Renderdoc causes it to use OpenGL even if you try to force it to use Vulkan in the VPinballX.ini file. The fix is to add an Environment Variable SDL_VIDEODRIVER with the value x11. Select the ... button at the right of the Environment Variables field and fill in the details as shown. Click either "Add" if this variable is new or "Update" if it already exists, and then "OK" when done.
Selecting the "Launch" button at the bottom right of this tab will start up VPinball for the table specified in the "Command-line Arguments" and automatically trigger a capture at (in this case) Frame 465. VPinball will continue to run until you exit the program, after which you'll be greeted with a screen like this one showing the captured frame ready for subsequent analysis.
The frame can then be exported for later analysis via File → Export as → XML + ZIP capture. This creates two large files totaling tens of megabytes in size that can be reloaded later or shared with a developer for further analysis.
Note
A capture can be analyzed on different machines but only if the GPUs on those machines are from the same company. So a capture on a machine with an AMD card can be analyzed on another machine only as long as it also has an AMD card.