Performance Guide - Omni-guides/Wabbajack-Modlist-Linux GitHub Wiki
START HERE
by Ouruboros, with help from Arnold, Charlistic and Omni
This guide is here to help increase the performance of Wabbajack lists on the Steam Deck, but the same steps also work for any Windows or Linux machine. I do all my modifications on Windows, but the tools/steps used should in theory also work on linux with some modifications depending on your set-up.
Before you start any of the guides, there are a couple of points you need to be aware of:
-
This will void your official support of the list you are trying this on, no exceptions. Don’t post your issues during or after these guides in the support channels.
-
If you need help during any of the steps the best place to ask is in the modification channel of whatever list you are using, if there is any. When asking for help, be as precise as you can be with your question/issue.
-
Manage your expectations, this will help increase performance, but will not make a list like Ro or Vagabond work on a Steam Deck.
-
I won’t promise that this guide will always be up-to-date, things might change with any tools/mods covered. When in doubt, I recommend checking for any instructions on the nexus pages of the mod.
-
If you follow the steps and recommendations in here none of the changes you will make will be destructive and are perfectly reversible. Just by switching back to the default profile the list came with and maybe restoring a backed up stockgame folder.
-
Every list is different, that is why this guide can’t always be super detailed and you will need a bit of knowledge about modding and how MO2 works. Checkout this guide to learn some of the basics: https://github.com/LivelyDismay/Learn-To-Mod/wiki.
-
The impact on graphics will depend on how far you go with some settings and how sensitive you are to these changes. Personally I don’t find them super noticeable on the small screen of the Steam Deck.
-
Feel free to report in the unofficial-linux-help channel on the Wabbajack discord on what list you have gotten to work on the Steam Deck. Even better if you send some screenshots, so that other people will know how it will look.
-
Again this will void any official support for the list you are using.
-IMPORTANT- Before you make any change you will need to make a new profile in MO2, you can do this by clicking on the figure/id card in the top left. Then select the profile you want to change and select copy, then name it something recognizable like Deck or Performance. -IMPORTANT-
This and other steps might look a bit different based on the skin that is used in the list, but the buttons should be in the same place. The same is true for all steps in this guide.
Performance cheat sheet
You will want to start with a guide that helps resolve the biggest bottleneck you have on your system and then move on to the next bottleneck you can see, until you are happy with your fps. To find the bottleneck with your Wabbajack list you will need to use a performance monitoring tool. On the Steam Deck you can use the build in tool, click the button with the 3 dots under the trackpad-> lighting bold -> Performance Overlay 2. On linux you can use MangoHud: https://github.com/flightlessmango/MangoHud and for Windows there is MSI Afterburner: https://www.msi.com/Landing/afterburner/graphics-cards.
For the GPU you will want to look at the percentages, 95+ will likely mean you have a GPU bottleneck. For Vram you will want to check how most of the total is used, if you are near or at the max you have a bottleneck. On the Steam Deck Vram and ram share the same 16gb, so you will need to look at both. A CPU bottleneck is harder to see, because of the lack of proper multicore support. Resulting in low cpu usages, while still running into a bottleneck on modern CPUs. The best way to tell is to see if the CPU percentage stays locked, while your fps is going down. Good places to test are Riverwood, Falkreath, Whiterun stables and Solitude docks.
Guide | Game | Bottleneck | Impact |
---|---|---|---|
Replacing ENB with Community Shaders | Skyrim SE | GPU | A different look, but not necessarily better or worse. CS sticks closer to vanilla than most ENBs. |
Using Bethin Pie to make a performance ini | Skyrim SE or Fallout 4 | GPU and CPU | Depending on how low you go with the presets, the lower the presets the bigger the impact. |
Installing the Upscaler mod | Skyrim SE or Fallout 4 | GPU | A slight blur or artifacting on some things in motion. Depending on your settings and resolution. |
Optimization of textures with vRAMr | Skyrim SE | Vram | A lower texture resolution. The impact will depend on your resolution, screen size, and how close you get to the texture. Not really that noticeable on the Steam Deck. |
Running DynDOLOD with a lower preset | Skyrim SE | GPU and CPU | Will make distant stuff more closer in look to vanilia. Not really that noticeable on the Steam Deck. |
Removing mods from a list | Skyrim SE or Fallout 4 | - | Will depend on what you remove. |
Replacing ENB with Community Shaders (Skyrim SE)
Community Shaders is an alternative to ENB for Skyrim SE. It offers a couple of advantages over ENB, most important is better performance. It’s also relatively new and doesn’t have all of the same features from the ENB, but more is always being added. Also you want to make sure you have the Wine Components and dependencies installed from here to your prefix/compact data if you are on Linux/ Steam Deck: https://github.com/Omni-guides/Wabbajack-Modlist-Linux/wiki/ENB,-Reshade-and-Community-Shaders.
Step 1 - Removing the ENB
To start you will need to remove the ENB series files itself, how you need to do this will depend on the list. Some lists will have it as a mod within MO2, in others it will be a part of the stockgame folder. You can check by clicking on Data in MO2 and scroll down to the root folder, you are looking for the following files: d3d11.dll, d3dcompiler.dll, enblocal and enbseries. MO2 will show you what mods they came from, turn that mod off in the left panel. If they are not there, you will need to delete them from the stockgame folder, but make a back-up of the folder before you change anything. You can find the stockgame folder in the install folder of your mod list, it will likely have stock, game, root and/or skyrim in the name. You know you have the right folder when you see the SkyrimSE.exe.
Within MO2 there are likely going to be more mods with ENB in their names, these are fine to be kept in place. You only need to remove the ENB series files and the dll’s.
Step 2 - Installing Community Shaders
Community Shaders nexus page can be found here: https://www.nexusmods.com/skyrimspecialedition/mods/86492. You don’t have to do anything special to install, just do it as you would any other mod in MO2. I recommend making a new separator to place the mod in, named something like CS, you can do this by Right clicking on any separator in the modlist and going to _All mods _-> Create separate above. It’s a skse plugin, so you don’t need to worry about the load order. Do make sure that you have all the mods installed that CS needs, you can find them on the nexus page under Requirements. You can use the search function at the bottom of MO2 to check. If you are worried about Auto Parallax, it will not cause any issue if you install it on a list without parallax textures.
Step 3 - Adding Reshade
Some lists already come with their own Reshade set-up even if they don’t come with CS, if that is the case you can skip this part.
Reshade can be found here: https://reshade.me/ and helps fill in some of the gaps left by the ENB, you will want to download the one just named Reshade. To install it run the setup exe you downloaded from the website. Click on Browse and then you will have one of two options, depending on the list. You will either need to select the SkyrimSE.exe in a stockgame folder (a separate install of Skyrim, often with a lower version then Steam e.g. 1.5.97 or 1.6.640) or within your Skyrim SE install in Steam. You can check which of these two options you need by clicking on the spanner at the top of MO2, then click on Paths. Look at Managed Game at the bottom, if steamapps is a part of the file path there is no stockgame folder. If the file path looks something like the screenshot then you have a stockgame folder.
Click on next and select DirectX 10/11/12. Here you have two options, if you already have a preset in mind you can just select that with the Browse button. If not, just click on Uncheck all and then Check all, followed by Next. Now you have set-up a Reshade.
If you are lucky you have a stockgame folder and you can skip this. You will need to create a new mod in MO2, by right clicking All Mods -> Create empty mod inside. Name it some like Reshade. Right click on that mod and then Open in Explorer add a folder called Root. Add to that folder all the stuff that Reshade has added to your Skyrim SE install folder:_ reshade-shaders, dxgi.dll, Reshade.ini and ReShadePreset.ini_. See the screenshot for an example. I recommend you drag this new mod to the same separator as CS.
You will now need to find a reshade profile, you can select any from nexus that you find interesting or even make your own. I would recommend finding something that supports CS + the light and weather mods that are in the list. But reshade profiles don’t have hard requirements like other mods and you can try one that doesn’t support the mod in the list, the profile will only interact with reshade and nothing else. The only issue is that stuff might look off. The .ini from the profile needs to be placed in the root (the location of the SkyrimSE.exe), if you don’t have a stockgame you can add them to the root folder of the Reshade mod you made.
Step 4 - Reshade and CS addons/modifications
CS has a lot of addons you might want to try and are just as easy to install as the main mod, just make sure there are below CS in MO2. Light Limit Fix (https://www.nexusmods.com/skyrimspecialedition/mods/99548) and SSE ReShade Helper (https://www.nexusmods.com/skyrimspecialedition/mods/78961) are two that I strongly recommend installing. The first makes ENB lights work with CS and the latter prevents reshade from messing with the menus. Any other comes down to taste and what your reshade profile recommends to get the same look. It’s something I recommend you play around with, but you will want to install most of them if you want something similar to an ENB.
In game you can make changes to both your reshade setup and CS. The HOME key will open the reshade menu and the END key for CS, feel free to play around with the options if you want. This menu also is where you can switch between profiles for Reshade, by switching them at the top of the menu. Just click on the name and then select the .ini you add to the root. I also recommend turning on the Performance Mode if you are on a Steam Deck.
Using Bethin Pie to make a performance ini (Skyrim SE/Fallout 4)
Bethini Pie is a program designed for better editing of the ini (game settings) file than the Skyrim SE/Fallout 4 launcher, both to increase the performance or graphics of a modded install.
Step 1 - Setup Bethini Pie
Bethini Pie can be found here: https://www.nexusmods.com/site/mods/631, but there is a decent chance that it’s already part of the list you are trying. Check the tools folder within your list to be sure. Download the file and extract it to wherever you like.
Launch the Bethini exe and select your game. Then you will need to point it to the location of your game ini’s, they are located in the profiles folder within the install folder of your Wabbajack list. Select the skyrim.ini in the folder named the same as the profile you made at the start of these guides.
Step 2 - How to use Bethini
You should now see all the settings you can change. Changing them by hand is an option, but I would recommend sticking to the Bethini Presets, first try Medium and work your way to Poor if you need to. Be sure to check any lighting mods, weather mods, and reshade profile for any recommended setting changes they make. Double check they are still in place after selecting the presets, to preserve the look of the list, but you can skip this if you just want the extra performance. Don’t forget to click, file -> save before you quit Bethini.
Installing the Upscaler mod (Skyrim SE/Fallout 4)
The Upscaler mod will add FSR, DLSS or XeSS to your installed list, helping with any issue stemming from GPU performance. I recommend DLSS for Nvidia users (20XX and newer) and FSR for anybody else. This will not work with an ENB installed.
Step 1 - Install UpscalerBasePlugin
UpscalerBasePlugin can be found here: https://www.nexusmods.com/site/mods/502 and will work both with Skyrim and Fallout. Download and install it as you would any mod, MO2 will likely give you a warning, ignore those and make sure it’s installed as shown in the screenshot. If you don’t get a warning click on manual and again make sure it’s the same as the screenshot.
Now download the required file you need for DLSS or FSR, they are linked on the mod page. Back in MO2 right click on UpscalerBasePlugin and then Open in Explorer. Add the unzipped files you downloaded to the UpscalerBasePlugin folder.
Step 2 - Setting up the Upscaler
Install Skyrim Upscaler (https://www.nexusmods.com/skyrimspecialedition/mods/80343) or Fallout 4 Upscaler (https://www.nexusmods.com/fallout4/mods/68586) as you would any mod. It’s a skse plugin, so you don’t need to worry about the load order. Do make sure that you have all the mods installed that the Upscaler needs, you can find them on the nexus page under Requirements. You can use the search function within MO2 to check.
Now double click on the mod and go to the INI Files tab, make sure that either FSR or DLSS is selected based on your earlier choice. You don’t have to worry about the other settings, you can set those in game by opening the menu with END.
Optimization of textures with vRAMr (Skyrim SE)
vRAMr is a great tool that makes it easy to reduce the resolution of textures within a modded install of Skyrim SE. Especially helpful when your vram is quickly filling up, but I will not cover how to use it here. Instead use the great pdf guide made for the tool, that you can find in the same places as the tool: https://www.nexusmods.com/skyrimspecialedition/mods/90557. It will cover everything from setting it up in MO2 to using it.
I recommend using the layered option when you are in the first part of the guide. The profile folder can be found at the root of the install folder of the list you are using, select the folder named the same as the profile you made at the start. See the screenshot for an example. During the second part of the guide you will be asked to select the texture resolutions, I would recommend the performance option for a Steam Deck.
Lastly, two warnings. First this will take a couple of hours before you are done. Second it will take up a fairly large chuck of your ssd/hdd and increase the size of the list.
Running DynDOLOD with a lower preset (Skyrim SE)
DynDolod is a tool for the setup of custom and better looking Lods (distant stuff like buildings and trees) for Skyrim SE. The higher the settings that were used when generating the Lods, the higher the performance cost. That is why rerunning the tool with lower settings can help increase performance. I will not cover how to set it up, because every list I use already comes with its setup, but it’s often hidden in the ui. Click on the gears/arrow in MO2 to fix that, then go to the tool and uncheck hide in user interface. Check here if you do need to set up the tool: https://www.youtube.com/watch?v=encZYHEeQrQ.
Step 1 - Turn off the outputs in MO2
Somewhere near the bottom of the mod list is going to be two mods you will need to turn off. They will probably have something like DynDOLOD, Texgen, and/or output in their name. You will need to turn those off. After that check the top right of MO2 and click on the exclamation mark. Check if there is a warning in there about missing master in there.
If this is the case you will need to find these plugins in the load order and temporarily turn them off. The search function can help with finding them in the list.
Step 2 - Running TexGen
TexGen creates all the textures that are used by DynDOLOD. Open it in MO2 and check for the following settings. You will want to make sure Grass and HD Tree are turned off. For the Steam Deck you will also want to lower to resolutions, to something around 720p or 1080p. Click on start to begin, this will take a couple of minutes. When it’s done click on Zip and Exit. You will likely find this zip fill in (install folder)/tools/DynDOLOD/TexGen_Output or an alternative location that you choose in TexGen. Drag this zip file into MO2 and place it under the included TexGen output, then turn it on.
Step 3 - Running DynDOLOD
Open DynDOLOD in MO2 and when it’s done loading click on Advanced Mode if you don’t get the full menu. First click on Low or Medium in the top left, for the Steam Deck I would recommend sticking to Low. Then make sure that both Grass LOD and Upgrade NearGrid are both turned off. For the trees make sure only Tree LOD is selected. Lastly check the top left and make sure that at least Tamriel (Skyrim) is selected, these are all the worldspaces DynDOLOD will generate the Lods for. If you want to play around more with the settings, I would also highly recommend learning what they do here: https://dyndolod.info/Help/Advanced-Mode.
Now click on OK and you will need to wait for at least an hour and maybe more, when done click on Zip and Exit. You will likely find this zip file in (install folder)/tools/DynDOLOD/DynDOLOD_Output or an alternative location that you chose in DynDOLOD. Drag this zip file into MO2 and place it under the included DynDOLOD output, then turn it on. Make sure the .esp and .esm from DynDOLOD are in the same place in the load order as they were at the start and are toggled on. You can peek at the built in profile to learn where they were in the load order. Also don’t forget to turn on the plugins you might have disabled in the first part
Removing mods from a list (Skyrim SE/Fallout 4)
Removing mods from a list is your last resort and not something I would recommend you do. There is a lot you can break and YOU are responsible for any changes/removals you make. The steps you need to take are also something that is highly dependent on the list and mod you are trying to remove. But here is a list of some tips and things to be aware of.
- Most textures can be safely removed, because they don’t have any plugins. They are just a collection of texture files. You can check by double clicking the mod in MO2 and looking under the Optional ESPs tab, if there are no .esp, .esm, or .esl you are safe.
-
First you will want to look for mods that are both heavy on performance and relatively easy to remove. Like 4k/8k texture mods or Grass Cache, but you will need to use your own judgment for that.
-
The biggest problem you will run into when removing stuff from a list is missing masters from other mods or patches. MO2 will show you this in the top right with an exclamation mark and will also tell you which plugins are missing their master.
-
In some cases the missing master comes from a DynDOLOD output, Synthesis output or other patching program. In this case you can just rerun those tools.
-
In other cases it involves a patch between mods that you might no longer need, you can just turn those off. Depending on what you mod you removed
-
You can try the following guides to help resolve the issue. First start with Making an empty esp: https://www.canva.com/design/DAF10-6Maj0/dtRmf4QivMqNAV6q7yhCJQ/edit and follow that up with Basic SSE Editing: https://www.canva.com/design/DAF10pUrC0Y/KmR7s0xcYKjj_W_ZDPblsQ/edit. -Warning- this is a destructive process -Warning-. Thanks to Setlot in Jolly Co-OP for making these guides.
-
Alternatively you can use xedit to remove the master requirement from a plugging. This is the way I personally do it. Check out this guide: https://github.com/LivelyDismay/Learn-To-Mod/blob/main/lessons/Remove%20a%20Master.md. -Warning- this is a destructive process -Warning-.
Changelog
- removed CS v8 warning (05/04/2024)
- added warning about CS v8 on Linux (17/03/2024)