Lora Trainer Setup Guides - CS1o/Stable-Diffusion-Info GitHub Wiki

Lora Trainer Sezup Guides for Nvidia and AMD:

Nvidia Install Guides:

AMD Install Guides:

OneTrainer with ZLUDA
FluxGym with ZLUDA
Khoya_ss with Zluda WIP, testing needed.

[NVIDIA] OneTrainer

Install OneTrainer on Windows for NVIDIA GPUs: Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing OneTrainer:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/Nerogar/OneTrainer.git
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.
Launch the start-ui.bat and its ready to use.

(Source: https://github.com/Nerogar/OneTrainer)

[NVIDIA] Kohya_ss

Install Kohya_ss on Windows for NVIDIA GPUs: Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing Koya_ss:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
If only Python 3.10.11 64bit is installed, launch the setup.bat, and wait until its done. Then close the cmd.
If multiple Python Versions are installed, launch the setup-3.10.bat, and wait until its done. Then close the cmd.
Launch the gui.bat and its ready to use.

Optional: CUDNN 8.9.6.50 The following steps are optional but will improve the learning speed for owners of NVIDIA 30X0/40X0 GPUs. These steps enable larger training batch sizes and faster training speeds. Run the setup.bat and select 2. (Optional) Install cudnn files (if you want to use the latest supported cudnn version).

(Source: https://github.com/bmaltais/kohya_ss)

[NVIDIA] FluxGym

Getting FluxGym:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/cocktailpeanut/fluxgym && cd fluxgym && git clone -b sd3 https://github.com/kohya-ss/sd-scripts
Press enter and after its done you can close the cmd.

Installing/Launching FluxGym: Now we need to create a Launch File to easily install and start FluxGym:

To create the File first open the link below in a new tab: https://raw.githubusercontent.com/CS1o/Stable-Diffusion-Info/main/Resources/Start-FluxGym-Nvidia.bat
Then on the Site make a right click and select "Save Site as/under"
Before Saving make sure to set the Filetype to All Files (.) then press Save.
If it's still a .txt file, then you need to Enable the File Extensions in the File Explorer by clicking: in Win10 View > File Name Extensions in Win11 View > Show > File Name Extensions Then rename the File and change the .txt to .bat
Move the Start-FluxGym-Nvidia.bat into the fluxgym Folder.
Launch the Start-FluxGym-Nvidia.bat. It will now install everything needed and when it's done the Webui opens in your default Browser.
RTX 50XX series GPUs have to do some additional steps to make it work: Right click and edit the Start-FluyGym-Nvidia.bat Change this line: pip install torch==2.3.1 torchvision==0.14.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121 to this: pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 && pip install -U bitsandbytes Then save and close. Delete the venv folder before relaunching the Start-FluyGym-Nvidia.bat

(Source: https://github.com/cocktailpeanut/fluxgym)

[AMD] OneTrainer with ZLUDA

Install OneTrainer on AMD Windows with Zluda: Supported GPUs

Preparation: Make sure your Graphics Card Driver is updated. Requires AMD Adrenalin 24.1.1 Driver or newer. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.11.9 64bit setup.exe from here: https://www.python.org/downloads/release/python-3119/ In the first Python install screen check "Add Python to Path"

Getting the OneTrainer AMD Fork:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/lshqqytiger/OneTrainer.git && cd OneTrainer && py scripts/install_zluda.py
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.

Setting up Zluda Important: If you already installed anything else with Zluda from my Guides, you can jump directly to Step 7. of the Zluda setup. Important: If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is higher/newer than a RX6800 but is not a 9070/60, then:

Download and install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is below/older than a RX 6800 or is a 9070/60, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy all files to OneTrainer/.zluda and overwrite if asked.
Launch the start-ui.bat. It will open up in your default Browser. Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

Attention: In the Training tab set the Attention to Default or SDP instead of xformers as xformers won't work on AMD and you'll get an Error.

For training Loras with OneTrainer checkout this Guide on Civitai.com: https://civitai.com/articles/4789/lazy-lora-making-with-onetrainer-and-ai-generation

(Source: https://github.com/lshqqytiger/OneTrainer)

[AMD] FluxGym with ZLUDA

Install FluxGym on AMD Windows with Zluda: Supported GPUs

Preparation: Make sure your Graphics Card Driver is updated. Requires AMD Adrenalin 24.1.1 Driver or newer You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Getting FluxGym:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/cocktailpeanut/fluxgym && cd fluxgym && git clone -b sd3 https://github.com/kohya-ss/sd-scripts
Press enter and after its done you can close the cmd.

Installing/Launching FluxGym: Now we need to create a Launch File to easily install and start FluxGym:

To create the File first open the link below in a new tab: https://raw.githubusercontent.com/CS1o/Stable-Diffusion-Info/main/Resources/Start-FluxGym-Zluda.bat
Then on the Site make a right click and select "Save Site as/under"
Before Saving make sure to set the Filetype to All Files (.) then press Save.
If it's still a .txt file, then you need to Enable the File Extensions in the File Explorer by clicking: in Win10 View > File Name Extensions in Win11 View > Show > File Name Extensions Then rename the File and change the .txt to .bat
Move the Start-FluxGym-Zluda.bat into the fluxgym Folder.
Launch the Start-FluxGym-Zluda.bat. It will now install everything needed and when it's done the Webui opens in your default Browser.
Using FluxGym wont work as Zluda is needed, so close everything and proceed with the next steps.

Download and install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is below/older than a RX 6800 or is a 9070/60, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy the three renamed files to fluxgym\venv\Lib\site-packages\torch\lib and overwrite if asked.
Now go into the fluxgym\sd-scripts Folder. There right click the flux_train_network.py and edit it with Notepad or Notepad++.
In the Script at line 7 it looks like this:

import torch
from accelerate import Accelerator

change it to:

import torch
from accelerate import Accelerator
import torch
torch.backends.cudnn.enabled = False
torch.backends.cuda.enable_flash_sdp(False)
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(False)
if hasattr(torch.backends.cuda, "enable_cudnn_sdp"):
    torch.backends.cuda.enable_cudnn_sdp(False)

Save and close the File.

Launch the Start-FluxGym.bat. It will open up in your default Browser. Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

(Source: https://github.com/cocktailpeanut/fluxgym)

[AMD] Kohya_ss with Zluda

WIP: Kohya_ss is very complex and has many settings. I dont have the time to test as it always complains about my test Folder structures. So if you know how to use Kohya_ss and want to test it with Zluda feel free to try and please let me know if it works or not. Install Kohya_ss on AMD Windows with Zluda Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing Kohya_ss:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
If only Python 3.10.11 64bit is installed, launch the setup.bat, and wait until its done. Then close the cmd.
If multiple Python Versions are installed, launch the setup-3.10.bat, and wait until its done. Then close the cmd.
Launch the gui.bat and select 1 for installing kohya_ss. After its done close everything.
In the Kohya_ss folder, click in the explorer bar (not searchbar) There Type cmd and hit enter and then run these three commands on by one: venv\scripts\activate pip uninstall torch torchvision -y pip install torch==2.3.1 torchvision --extra-index-url https://download.pytorch.org/whl/cu118
Now right click and edit the gui.bat file. At the last line where it shows: python.exe kohya_gui.py %* change it to: python.exe kohya_gui.py --noverify --inbrowser %* Save and close the File.
Launch the gui.bat again. Let it install and close when its done. Proceed with the Zluda setup.

Download and install AMD HIP SDK 6.2.4 from here
Download the latest Zluda files for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
If you want to use the latest ROCm HIP SDK 6.2.4 and your GPU is below/older than a RX 6800 or is a 9070/60, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy the three renamed files to kohya_ss\venv\Lib\site-packages\torch\lib and overwrite if asked.
Launch the gui.bat, wait a bit and then open the local Kohya webui: http://127.0.0.1:7860 Important: Make sure under Parameters > Advanced > CrossAttention to disable xFormers, set it to None.

Maybe additional needed steps: (let me know)

Go into the kohya_ss\sd-scripts Folder. There right click the sdxl_train_network.py and edit it with Notepad or Notepad++.
In the Script at line 5 it looks like this:

import torch
from accelerate import Accelerator

overwrite it to:

import torch
from accelerate import Accelerator
import torch
torch.backends.cudnn.enabled = False
torch.backends.cuda.enable_flash_sdp(False)
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(False)
if hasattr(torch.backends.cuda, "enable_cudnn_sdp"):
    torch.backends.cuda.enable_cudnn_sdp(False)

Save and close the File. Relaunch the gui.bat, wait a bit and then open the local Kohya webui: http://127.0.0.1:7860