The GPU and the Torch Library

The GPU (Graphics Processing Unit) is a microprocessor that is specifically designed to process and display computer video. The GPU may be integrated into the computer motherboard or come on its own discrete circuit board. Typically the discrete GPU systems are more suitable for creating generative AI images, and NVIDIA had become the standard for this work.

Torch is a GPU support library written in C++ and PyTorch is the variant of Torch that has a Python interface. Python is the programming language used by many generative AI systems, including FooocusPlus. Because of its central role in the generative AI process, the Torch software system has a crucial role and it must be customized to support the installed GPU.

In FooocusPlus, the Torchruntime program library analyzes the GPU and the computer operating system and automatically installs the correct versions of Torch and its dependencies.

The GPU Type

While the built-in Torchruntime module generally makes an optimal setup for Torch and its related components, there may be times when a manual override is desireable or necessary. The GPU-Type command line argument gives FooocusPlus users this ability, as follows:

• "amd64" uses Torch version "2.2.2"
• "cu124" uses Torch 2.4.1+cu124
• "cu128" uses Torch 2.7.1+cu128
• "directml" uses Torch version "2.2.2" (the same as "amd64")
• "rocm5.2" uses Torch 1.13.1
• "rocm5.7" uses Torch 2.3.1

Command Line Arguments

FooocusPlus is typically started with a batch file similar to this one:

@echo off

python_embedded\python.exe -s FooocusPlusAI\entry_with_update.py --language en %*

pause

In this example, the command line argument, "--language en", configures FooocusPlus to run the user interface (UI) with English language text. But to control the GPU type, use this syntax:

@echo off

:: Please copy this file to the FooocusPlus directory to run it

.\python_embedded\python.exe -s FooocusPlusAI\entry_with_update.py --language en --gpu-type cu124 %*

pause

In this example, the gpu-type is set for "cu124", which uses Torch 2.4.1. This file is available as run_FooocusPlus_cu124.bat in the FooocusPlus/UserDir/batch_startups directory. It needs to be copied to the FooocusPlus directory to run it.