Lora Trainer Setup Guides - CS1o/Stable-Diffusion-Info GitHub Wiki

Lora Trainer Sezup Guides for Nvidia and AMD:

Nvidia Install Guides:

AMD Install Guides:

OneTrainer with ROCm/TheRock Currently broken caused by mismatching dependencies!
Khoya_ss with ROCm/TheRock Currently broken caused by mismatching dependencies!
[FluxGym with ROCm/TheRock]

[NVIDIA] OneTrainer

Install OneTrainer on Windows for NVIDIA GPUs: Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing OneTrainer:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/Nerogar/OneTrainer.git
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.
Launch the start-ui.bat and its ready to use.

(Source: https://github.com/Nerogar/OneTrainer)

[NVIDIA] Kohya_ss

Install Kohya_ss on Windows for NVIDIA GPUs: Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing Koya_ss:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
If only Python 3.10.11 64bit is installed, launch the setup.bat, and wait until its done. Then close the cmd.
If multiple Python Versions are installed, launch the setup-3.10.bat, and wait until its done. Then close the cmd.
Launch the gui.bat and its ready to use.

Optional: CUDNN 8.9.6.50 The following steps are optional but will improve the learning speed for owners of NVIDIA 30X0/40X0 GPUs. These steps enable larger training batch sizes and faster training speeds. Run the setup.bat and select 2. (Optional) Install cudnn files (if you want to use the latest supported cudnn version).

(Source: https://github.com/bmaltais/kohya_ss)

[NVIDIA] FluxGym

Getting FluxGym:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/cocktailpeanut/fluxgym && cd fluxgym && git clone -b sd3 https://github.com/kohya-ss/sd-scripts
Press enter and after its done you can close the cmd.

Installing/Launching FluxGym: Now we need to create a Launch File to easily install and start FluxGym:

To create the File first open the link below in a new tab: https://raw.githubusercontent.com/CS1o/Stable-Diffusion-Info/main/Resources/Start-FluxGym-Nvidia.bat
Then on the Site make a right click and select "Save Site as/under"
Before Saving make sure to set the Filetype to All Files (.) then press Save.
If it's still a .txt file, then you need to Enable the File Extensions in the File Explorer by clicking: in Win10 View > File Name Extensions in Win11 View > Show > File Name Extensions Then rename the File and change the .txt to .bat
Move the Start-FluxGym-Nvidia.bat into the fluxgym Folder.
Launch the Start-FluxGym-Nvidia.bat. It will now install everything needed and when it's done the Webui opens in your default Browser.
RTX 50XX series GPUs have to do some additional steps to make it work: Right click and edit the Start-FluyGym-Nvidia.bat Change this line: pip install torch==2.3.1 torchvision==0.14.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121 to this: pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 && pip install -U bitsandbytes Then save and close. Delete the venv folder before relaunching the Start-FluyGym-Nvidia.bat

(Source: https://github.com/cocktailpeanut/fluxgym)

[AMD] OneTrainer with ZLUDA

Install OneTrainer on AMD Windows with Zluda: Supported GPUs

Preparation: Make sure your Graphics Card Driver is updated. Requires AMD Adrenalin 24.1.1 Driver or newer. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.11.9 64bit setup.exe from here: https://www.python.org/downloads/release/python-3119/ In the first Python install screen check "Add Python to Path"

Getting the OneTrainer AMD Fork:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/lshqqytiger/OneTrainer.git && cd OneTrainer && py scripts/install_zluda.py
Press enter and after its done you can close the cmd.
Launch the install.bat, and wait until its done. Then close the cmd.
Launch the update.bat, and wait until its done. Then close the cmd.

Setting up Zluda Important: If you already had setup Zluda before from my other Guides, you can skip the Zluda setup. If you have an an RX 580 move here: Guide for ROCm 5.7

Important: To use ROCm HIP SDK 6.4.2 and your GPU is higher/newer than a RX6800 or is a 9070/60, do:

Download and install AMD HIP SDK 6.4.2 from here
Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
For GPUs that are below/older than a RX 6800, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy all files to OneTrainer/.zluda and overwrite if asked.
Launch the start-ui.bat. It will open up in your default Browser. Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

Attention: In the Training tab set the Attention to Default or SDP instead of xformers as xformers won't work on AMD and you'll get an Error.

For training Loras with OneTrainer checkout this Guide on Civitai.com: https://civitai.com/articles/4789/lazy-lora-making-with-onetrainer-and-ai-generation

(Source: https://github.com/lshqqytiger/OneTrainer)

[AMD] FluxGym with ZLUDA

Currently broken with HIP SDK 6.4 Install FluxGym on AMD Windows with Zluda: Supported GPUs

Preparation: Make sure your Graphics Card Driver is updated. Requires AMD Adrenalin 24.1.1 Driver or newer You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Getting FluxGym:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it SD-Zluda. Dont use the same if you had installed the DirectML webui before.
You go into the folder you created in this case SD-Zluda, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone https://github.com/cocktailpeanut/fluxgym && cd fluxgym && git clone -b sd3 https://github.com/kohya-ss/sd-scripts
Press enter and after its done you can close the cmd.

Installing/Launching FluxGym: Now we need to create a Launch File to easily install and start FluxGym:

To create the File first open the link below in a new tab: https://raw.githubusercontent.com/CS1o/Stable-Diffusion-Info/main/Resources/Start-FluxGym-Zluda.bat
Then on the Site make a right click and select "Save Site as/under"
Before Saving make sure to set the Filetype to All Files (.) then press Save.
If it's still a .txt file, then you need to Enable the File Extensions in the File Explorer by clicking: in Win10 View > File Name Extensions in Win11 View > Show > File Name Extensions Then rename the File and change the .txt to .bat
Move the Start-FluxGym-Zluda.bat into the fluxgym Folder.
Launch the Start-FluxGym-Zluda.bat. It will now install everything needed and when it's done the Webui opens in your default Browser.
Using FluxGym wont work as Zluda is needed, so close everything and proceed with the next steps.

Setting up Zluda Important: If you already had setup Zluda before from my other Guides, you can skip the Zluda setup. If you have an an RX 580 move here: Guide for ROCm 5.7

Important: To use ROCm HIP SDK 6.4.2 and your GPU is higher/newer than a RX6800 or is a 9070/60, do:

Download and install AMD HIP SDK 6.4.2 from here
Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
For GPUs that are below/older than a RX 6800, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy the three renamed files to fluxgym\venv\Lib\site-packages\torch\lib and overwrite if asked.
Now go into the fluxgym\sd-scripts Folder. There right click the flux_train_network.py and edit it with Notepad or Notepad++.
In the Script at line 7 it looks like this:

import torch
from accelerate import Accelerator

change it to:

import torch
from accelerate import Accelerator
import torch
torch.backends.cudnn.enabled = False
torch.backends.cuda.enable_flash_sdp(False)
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(False)
if hasattr(torch.backends.cuda, "enable_cudnn_sdp"):
    torch.backends.cuda.enable_cudnn_sdp(False)

Save and close the File.

Launch the Start-FluxGym.bat. It will open up in your default Browser. Noteworthy: The first Training can take longer as it will first initialize ZLUDA in the background. This can take up from 15 to 40 minutes. First time only.

(Source: https://github.com/cocktailpeanut/fluxgym)

[AMD] Kohya_ss with Zluda

WIP: Kohya_ss is very complex and has many settings. I dont have the time to test as it always complains about my test Folder structures. So if you know how to use Kohya_ss and want to test it with Zluda feel free to try and please let me know if it works or not. Install Kohya_ss on AMD Windows with Zluda Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.10.11 64bit setup.exe from here: https://www.python.org/downloads/release/python-31011/ In the first Python install screen check "Add Python to Path"

Installing Kohya_ss:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
If only Python 3.10.11 64bit is installed, launch the setup.bat, and wait until its done. Then close the cmd.
If multiple Python Versions are installed, launch the setup-3.10.bat, and wait until its done. Then close the cmd.
Launch the gui.bat and select 1 for installing kohya_ss. After its done close everything.
In the Kohya_ss folder, click in the explorer bar (not searchbar) There Type cmd and hit enter and then run these three commands on by one: venv\scripts\activate pip uninstall torch torchvision -y pip install torch==2.3.1 torchvision --extra-index-url https://download.pytorch.org/whl/cu118
Now right click and edit the gui.bat file. At the last line where it shows: python.exe kohya_gui.py %* change it to: python.exe kohya_gui.py --noverify --inbrowser %* Save and close the File.
Launch the gui.bat again. Let it install and close when its done. Proceed with the Zluda setup.

Setting up Zluda Important: If you already had setup Zluda before from my other Guides, you can skip the Zluda setup. If you have an an RX 580 move here: Guide for ROCm 5.7

Important: To use ROCm HIP SDK 6.4.2 and your GPU is higher/newer than a RX6800 or is a 9070/60, do:

Download and install AMD HIP SDK 6.4.2 from here
Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here
Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA
Then restart your PC and skip to Step 6. below and proceed there with the Guide.
For GPUs that are below/older than a RX 6800, you need to do the following steps:

Check if your GPU is supported by comparing the GPU GFX List with the List of GFX Library Files for 6.2.4

If the files are available for your GPU then you need to download them from here:

If they are not available then you need to use an older HIP SDK Version (5.7 or 6.1)

Download and install AMD HIP SDK 6.2.4 from here

Download the latest Zluda files (ZLUDA-windows-rocm6-amd64.zip) for ROCm 6 from here

Unzip the ZLUDA-windows-rocm6-amd64.zip and move the zluda folder onto the C: Drive and rename it to ZLUDA

Go into C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. There copy and rename the library folder to old_library

Open the gfx .zip file and drag and drop all files of the library folder into the library folder. Not into the old_library folder.

If the zip contains a rocblas.dll, copy that into the C:\Program Files\AMD\ROCm\6.2\bin\rocblas folder. Important: Restart your PC after this Step before proceeding with the steps below.

Add the C:\ZLUDA folder and %HIP_PATH%bin to your PATH in the System Variables. Like shown here: https://www.wikihow.com/Change-the-PATH-Environment-Variable-on-Windows It should look like this here
Go into the C:\ZLUDA\ Folder. There make copy of the cublas.dll and the cusparse.dll and nvrtc.dll inside the folder. Rename the copies to cublas64_11.dll and cusparse64_11.dll and nvrtc64_112_0.dll Copy the three renamed files to kohya_ss\venv\Lib\site-packages\torch\lib and overwrite if asked.
Launch the gui.bat, wait a bit and then open the local Kohya webui: http://127.0.0.1:7860 Important: Make sure under Parameters > Advanced > CrossAttention to disable xFormers, set it to None.

Maybe additional needed steps: (let me know)

Go into the kohya_ss\sd-scripts Folder. There right click the sdxl_train_network.py and edit it with Notepad or Notepad++.
In the Script at line 5 it looks like this:

import torch
from accelerate import Accelerator

overwrite it to:

import torch
from accelerate import Accelerator
import torch
torch.backends.cudnn.enabled = False
torch.backends.cuda.enable_flash_sdp(False)
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(False)
if hasattr(torch.backends.cuda, "enable_cudnn_sdp"):
    torch.backends.cuda.enable_cudnn_sdp(False)

Save and close the File. Relaunch the gui.bat, wait a bit and then open the local Kohya webui: http://127.0.0.1:7860

[AMD] OneTrainer with ROCm/TheRock

**Currently broken, caused by mismatching dependencies. Feel free to let me know some fixes. Install OneTrainer on AMD Windows with TheRock: [Supported GPUs] GFX11xx and GFX12xx

Preparation: Make sure your Graphics Card Driver is updated. You need to download and install Git 64bit setup.exe from here: https://git-scm.com/download/win You need to download and install Python 3.11.9 or 3.12.10 64bit Windows installer .exe In the first Python install screen check "Add Python to Path"

Setup OneTrainer:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: https://github.com/Nerogar/OneTrainer.git
Press enter and after its done you can close the cmd.
Launch the install.bat. It can take very long. After its done close the cmd.

Setup TheRock: 7. In the OneTrainer folder, click in the explorer bar (not searchbar) and type cmd then press enter. 8. Now run these commands one by one: venv\Scripts\activate.bat pip uninstall torch torchvision -y Now copy and paste only the one command that is for your GPU type: RX 9070/9060, XT, (GFX120x):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx120X-all/ torch torchvision RX 7900XTX (GFX110x):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx110X-dgpu/ torch torchvision AMD Strix Halo iGPU (GFX1151):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx1151/ torch torchvision 9. Now right click and edit the start-ui.bat file. In the second line add this: set TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 Save and close the File. 9. Launch the start-ui.bat to launch OneTrainer. 10. To update OneTrainer launch the update.bat sometimes.

[AMD] Khoya_ss with ROCm/TheRock

**Currently broken, caused by mismatching dependencies. Feel free to let me know some fixes. Install Khoya_ss on AMD Windows with TheRock: [Supported GPUs] GFX11xx and GFX12xx

Setup Kohya_ss:

Make a new Folder on your Drive (not on Desktop, Downloads, Documents, Programms, Onedrive) and name it Ai.
You go into the folder you created in this case C:\Ai, then click in the File Explorer bar (not searchbar) and type cmd then press enter.
Then you copy and paste this command: git clone --recursive https://github.com/bmaltais/kohya_ss.git
Press enter and after its done you can close the cmd.
Launch the setup.bat and select 1 for installing kohya_ss. It can take very long. After its done the Menu reappears, then close the cmd.

Setup TheRock: 7. In the Kohya_ss folder, click in the explorer bar (not searchbar) and type cmd then press enter. 8. Now run these commands one by one: venv\Scripts\activate.bat pip uninstall torch torchvision xformers -y Now copy and paste only the one command that is for your GPU type: RX 9070/9060, XT, (GFX120x):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx120X-all/ torch torchvision RX 7900XTX (GFX110x):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx110X-dgpu/ torch torchvision AMD Strix Halo iGPU (GFX1151):pip install --index-url https://d2awnip2yjpvqn.cloudfront.net/v2/gfx1151/ torch torchvision 9. Now right click and edit the gui.bat file. Change the Python Version in second line at set PYTHON_VER= to either 3.11.9 or 3.12.10 Below add this line: set TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 At the bottom part there are two lines where it shows: python.exe kohya_gui.py %* change boath to: python.exe kohya_gui.py --noverify --inbrowser %* Save and close the File. 9. Launch the gui.bat and Khoya_ss should open in the Browser. Important: Make sure under Parameters > Advanced > CrossAttention to disable xFormers, set it to None.