User manual for xpu - flagos-ai/FlagTree GitHub Wiki

💫 KLX xpu3.6

  • Based on Triton 3.6, x64
  • Available for P800

1. Quick start

1.1 Use the image (P800)

Tips: You can use apt install aria2; aria2c -c ... instead of wget ... to download large files with resume support.

IMAGE=harbor.baai.ac.cn/flagtree/flagtree-xpu3.6-py310-torch2.9.0-ubuntu22.04:202608-base
# Plan A: docker pull (59.9GB)
docker pull ${IMAGE}
# Plan B: docker load (32GB)
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/flagtree-xpu3.6-py310-torch2.9.0-ubuntu22.04.202608-base.tar.gz
docker load -i flagtree-xpu3.6-py310-torch2.9.0-ubuntu22.04.202608-base.tar.gz
# host
xpu-smi  # Driver Version: 5.0.21.47

CONTAINER=flagtree-dev-xxx
docker run -dit \
    --net=host --privileged \
    --ulimit stack=67108864 --ulimit memlock=-1 \
    --ulimit nofile=120000 --shm-size=256g \
    --group-add video --cap-add=SYS_PTRACE --cap-add=SYS_ADMIN \
    --security-opt seccomp=unconfined \
    --device=/dev/xpu0:/dev/xpu0 --device=/dev/xpu1:/dev/xpu1 \
    --device=/dev/xpu2:/dev/xpu2 --device=/dev/xpu3:/dev/xpu3 \
    --device=/dev/xpu4:/dev/xpu4 --device=/dev/xpu5:/dev/xpu5 \
    --device=/dev/xpu6:/dev/xpu6 --device=/dev/xpu7:/dev/xpu7 \
    --device=/dev/xpuctrl:/dev/xpuctrl --device /dev/fuse \
    -v /etc/localtime:/etc/localtime:ro \
    -v /data:/data -v /home:/home \
    -w /root --name ${CONTAINER} ${IMAGE} bash
docker exec -it ${CONTAINER} /bin/bash

# container
xpu-smi  # Driver Version: 515.58
python3 -m pip show pybind11  # Should be 3.0.1

1.2 Source-free Installation

# Note: First install PyTorch, then execute the following commands
python3 -m pip uninstall -y triton  # Repeat the cmd until fully uninstalled
RES="--index-url=https://resource.flagos.net/repository/flagos-pypi-hosted/simple"
python3.10 -m pip install flagtree===0.7.1a1+xpu3.6 $RES

2. Build from Source

2.1 Manually download the FlagTree dependencies

If your network connection is available, you do not need to download the dependencies which will be fetched automatically during the build.

mkdir -p ~/.flagtree/xpu; cd ~/.flagtree/xpu
# llvm-pub22
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/llvm-pub-22.1.8-ubuntu2004_x86_64-v2.tar.gz
mkdir llvm-pub22 && \
    tar zxvf llvm-pub-22.1.8-ubuntu2004_x86_64-v2.tar.gz -C llvm-pub22 --strip-components=1
# xpu-device-libs
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/xpu-device-libs-ubuntu-x64_v0.3.6.1.1.tar.gz
tar zxvf xpu-device-libs-ubuntu-x64_v0.3.6.1.1.tar.gz
# xpu-python-impl-so
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/xpu-python-impl-so_v0.3.6.9.0.tar.gz
tar zxvf xpu-python-impl-so_v0.3.6.9.0.tar.gz
# xpu-runtime-so
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/xpu-runtime-so_v0.3.6.2.0.tar.gz
tar zxvf xpu-runtime-so_v0.3.6.2.0.tar.gz
# xpu-sdnn-objects
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/xpu-sdnn-objects_v0.3.6.9.0.tar.gz
tar zxvf xpu-sdnn-objects_v0.3.6.9.0.tar.gz
# xre-Linux-x86_64
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/xre-Linux-x86_64_v0.3.0.tar.gz
tar zxvf xre-Linux-x86_64_v0.3.0.tar.gz
# xtdk-llvm19-printf
wget https://klx-sdk-release-public.su.bcebos.com/v1/triton/XTDK/20260403/xtdk-llvm19-ubuntu2004_x86_64.tar.gz \
    -O xtdk-llvm19-printf-ubuntu2004_x86_64.tar.gz
mkdir xtdk-llvm19-printf && \
    tar zxvf xtdk-llvm19-printf-ubuntu2004_x86_64.tar.gz -C xtdk-llvm19-printf --strip-components=1
# xtdk-llvm19
wget https://klx-sdk-release-public.su.bcebos.com/xtdk_llvm19/trust/latest/xtdk-llvm19-ubuntu2004_x86_64.tar.gz
mkdir xtdk-llvm19 && \
    tar zxvf xtdk-llvm19-ubuntu2004_x86_64.tar.gz -C xtdk-llvm19 --strip-components=1
# llvm_trust
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/llvm22/20260615/xtdk-llvm22-ubuntu2004_x86_64.tar.gz
mkdir llvm_trust && \
    tar zxvf xtdk-llvm22-ubuntu2004_x86_64.tar.gz -C llvm_trust --strip-components=1

2.2 Manually download the Triton dependencies

The Triton dependencies are already downloaded and installed in the image. If your network connection is available, you do not need to download the dependencies which will be fetched automatically during the build.

cd ${YOUR_CODE_DIR}/FlagTree
# For Triton 3.6 (x64)
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/build-deps-triton_3.6.x-linux-x64.tar.gz
sh python/scripts/unpack_triton_build_deps.sh ./build-deps-triton_3.6.x-linux-x64.tar.gz

After executing the above script, the original ~/.triton directory will be renamed, and a new ~/.triton directory will be created to store the pre-downloaded packages. Note that the script will prompt for manual confirmation during execution.

2.3 Build Commands

cd ${YOUR_CODE_DIR}/FlagTree
git checkout main
export FLAGTREE_BACKEND=xpu
MAX_JOBS=32 python3 -m pip install . --no-build-isolation -v

3. Testing and validation

After installing flagtree, you can check it with:

python3 -m pip show flagtree

Before testing, you need to execute export XPU_EVENT_KL3_ENABLE=1

Refer to Tests of xpu3.6 backend

💫 KLX xpu3.0

  • Based on Triton 3.0, x64
  • Available for P800

1. Quick start

1.1 Use the image (P800)

# Plan A: docker pull (44.9GB)
IMAGE=harbor.baai.ac.cn/flagtree/flagtree-xpu-py310-torch2.5.1-ubuntu20.04:202604-base
docker pull ${IMAGE}
# Plan B: docker load (21GB)
IMAGE=flagtree-xpu-py310-torch2.5.1-ubuntu20.04:202604-base
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/flagtree-xpu-py310-torch2.5.1-ubuntu20.04.202604-base.tar.gz
docker load -i flagtree-xpu-py310-torch2.5.1-ubuntu20.04.202604-base.tar.gz
CONTAINER=flagtree-dev-xxx
docker run -dit \
    --net=host --privileged \
    --ulimit stack=67108864 --ulimit memlock=-1 \
    --ulimit nofile=120000 --shm-size=256g \
    --group-add video --cap-add=SYS_PTRACE --cap-add=SYS_ADMIN \
    --security-opt seccomp=unconfined \
    --device=/dev/xpu0:/dev/xpu0 --device=/dev/xpu1:/dev/xpu1 \
    --device=/dev/xpu2:/dev/xpu2 --device=/dev/xpu3:/dev/xpu3 \
    --device=/dev/xpu4:/dev/xpu4 --device=/dev/xpu5:/dev/xpu5 \
    --device=/dev/xpu6:/dev/xpu6 --device=/dev/xpu7:/dev/xpu7 \
    --device=/dev/xpuctrl:/dev/xpuctrl --device /dev/fuse \
    -v /etc/localtime:/etc/localtime:ro \
    -v /data:/data -v /home:/home \
    -w /root --name ${CONTAINER} ${IMAGE} bash
docker exec -it ${CONTAINER} /bin/bash

1.2 Source-free Installation

# Note: First install PyTorch, then execute the following commands
python3 -m pip uninstall -y triton  # Repeat the cmd until fully uninstalled
RES="--index-url=https://resource.flagos.net/repository/flagos-pypi-hosted/simple"
python3.10 -m pip install flagtree===0.5.1+xpu3.0 $RES

2. Build from Source

2.1 Manually download the FlagTree dependencies

If your network connection is available, you do not need to download the dependencies which will be fetched automatically during the build.

mkdir -p ~/.flagtree/xpu; cd ~/.flagtree/xpu
wget https://klx-sdk-release-public.su.bcebos.com/v1/triton/flaggems/2025_4_season/llvm/20260304/XTDK-llvm19-ubuntu2004_x86_64.tar.gz
tar zxvf XTDK-llvm19-ubuntu2004_x86_64.tar.gz
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/xre-Linux-x86_64_v0.3.0.tar.gz
tar zxvf xre-Linux-x86_64_v0.3.0.tar.gz
wget https://klx-sdk-release-public.su.bcebos.com/XTriton/xpu-device-libs-ubuntu-x64_v0.3.6.1.1.tar.gz
tar zxvf xpu-device-libs-ubuntu-x64_v0.3.6.1.1.tar.gz

2.2 Manually download the Triton dependencies

The Triton dependencies are already downloaded and installed in the image. If your network connection is available, you do not need to download the dependencies which will be fetched automatically during the build.

cd ${YOUR_CODE_DIR}/FlagTree
# For Triton 3.1 (x64)
wget https://baai-cp-web.ks3-cn-beijing.ksyuncs.com/trans/build-deps-triton_3.1.x-linux-x64.tar.gz
sh python/scripts/unpack_triton_build_deps.sh ./build-deps-triton_3.1.x-linux-x64.tar.gz

After executing the above script, the original ~/.triton directory will be renamed, and a new ~/.triton directory will be created to store the pre-downloaded packages. Note that the script will prompt for manual confirmation during execution.

2.3 Build Commands

cd ${YOUR_CODE_DIR}/FlagTree/python
git checkout -b triton_v3.1.x origin/triton_v3.1.x
export FLAGTREE_BACKEND=xpu
MAX_JOBS=32 python3 -m pip install . --no-build-isolation -v

3. Testing and validation

After installing flagtree, you can check it with:

python3 -m pip show flagtree

Before testing, you need to execute export XPU_EVENT_KL3_ENABLE=1

Refer to Tests of xpu3.0 backend

⚠️ **GitHub.com Fallback** ⚠️