Compilation - gnomeCreative/HYBIRD GitHub Wiki

Compiling HYBIRD

The preferred way to compile HYBIRD is thorugh a terminal, either Windows or Linux. However, HYBIRD comes with CMake functionality that allows to create compilation packages for IDEs, for example for Visual Studio.

The CPU version can also compile and run via Cygwin, therefore with Unix compilers but on a Windows machine. For the GPU version this is not possible.

Compiling HYBIRD locally

CPU Version

For compiling the CPU version, we recommend using a Linux-based terminal. On Windows, Cygwin or WSL are both good options. Cygwin can be downloaded here.

Ensure that the following compilers (or their later versions) are installed:

  • gcc 12.4.0

  • g++ 12.4.0

  • gfortran 12.4.0

  • cmake 3.30.0

  • gdb 12.4.0

The inexperienced user is directed to the official Cywing webpage to install the required compilers.

Once the required dependencies are installed, follow these commands are require to compile the code: 

# Navigate to the target directory  
cd $(yourdirectory)  

# clone the repository
git clone https://github.com/gnomeCreative/HYBIRD.git

# move inside the newly created HYBIRD directory 
cd HYBIRD

# Switch to the last stable CPU branch (main-CPU)
git checkout main-CPU

# Create and enter the build directory  
mkdir -p build && cd build  

# Configure the build system (use Release mode for better performance)  
cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release  

# Compile the project using available processor cores
cmake --build . --parallel $(nproc)

GPU Version

The GPU version of the code requires the CUDA Toolkit. However, CUDA is not compatible with Cygwin, so the compilation must be performed using the Windows terminal. Once the CUDA Toolkit is installed, follow these steps to compile the code: 

# Navigate to the target directory  
cd $(yourdirectory)  

# Clone the repository
git clone https://github.com/gnomeCreative/HYBIRD.git

# Move into the newly created HYBIRD directory
cd HYBIRD

# Switch to stable GPU branch
git checkout main-GPU

# Create and enter the build directory  
mkdir build
cd build

# Configure the build system - this depends on the GPU architecture, the line below is an example
cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"

# if CMake does not find CUDA compile, force CMake to detect CUDA by setting:
cmake .. -DCMAKE_CUDA_COMPILER="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\<version>\bin\nvcc.exe"

# Compile the project using available processor cores  
cmake --build . --parallel $(nproc)
Important: the line 
cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"
needs to be adaped to the specific GPU architecture of the system. This can be easily checked online.

GPU version on Windows via WSL (Linux subsystem)

If you are using Windows 10 (version 2004 and higher, Build 19041+) or Windows 11, you can compile and run HYBRID using the Windows Subsystem for Linux (WSL). WSL is the recommended method for running HYBRID on a GPU. To set up WSL, open Windows Terminal as an administrator and install WSL by using the command wsl --install. A complete description of WSL can be found at the official website.

Now WSL has been installed. However, the system needs to be prepared to install CUDA. To do so, first enable sudo on the local machine (navigate to Settings → System → For Developers, then activate sudo.). Once sudo is enabled, follow these steps to install CUDA:

# Update package lists and upgrade installed packages  
sudo apt update && sudo apt upgrade -y

# Check the Ubuntu version currently running  
lsb_release -a

# Install CUDA (For Ubuntu 20.04, replace "ubuntu2204" with "ubuntu2004" in the URL) 
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt update
sudo apt install -y cuda

# Install CUDA toolkit
sudo apt install nvidia-cuda-toolkit

# Verify the installed CUDA version  
nvcc --version
nvidia-smi

# Test if CUDA is working correctly in WSL2  
# If the output is "CUDA is working in WSL2!", the installation was successful 
cat <<EOF > test.cu
#include <iostream>
int main() {
    std::cout << "CUDA is working in WSL2!\n";
    return 0;
}
EOF
nvcc test.cu -o test
./test

# Install CMake
sudo apt update
sudo apt install -y cmake

# Verify CMake installation (Version 3.22 or higher is required)  
cmake --version

Once these preliminary steps are completed, you can compile the code:

# Navigate to the target directory  
# note that WSL is being used and the directory syntax change 
cd /mnt/c/$(yourdirectory)  

# clone the repository and navigate to HYBIRD directory
git clone https://github.com/gnomeCreative/HYBIRD.git
cd HYBIRD

# Switch to stable GPU branch
git checkout main-GPU

# Create and enter the build directory  
mkdir build
cd build

# Configure the build system
cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"

# Compile the project using available processor cores  
cmake --build . --parallel $(nproc)

Important: the line 

cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"
needs to be adaped to the specific GPU architecture of the system. This can be easily checked online.

Compiling HYBRID an HPC system (e.g. Stanage @ Sheffield University)

Alternatively, HYBRID can be run on High-Performance Computing (HPC) systems, if available. The following section explains the procedure for compiling and running HYBRID on Stanage, the University of Sheffield’s HPC. For other HPC systems similar steps are likely needed.

The interface of WinSCP

The interface of WinSCP

To make things easier for those who are not used to working remotely with a terminal, it is recommended to also download the software WinSCP. This software creates a graphical interface to manage files and folders on the cluster. When downloaded, open it and select "New Session". Fill in the form with your login data, selecting "stanage.shef.ac.uk" as the Host name, and fill in your cluster username and password. You can also save these credentials so that you do not have to type them again.

The login interface necessary to create a new connection

The login interface necessary to create a new connection

Once logged in, you will be able to see the files in your folder on the cluster and prepare the simulation environment there (both for compiling and running). Compilation needs to be operated inside the HPC system. This is very important! The cluster will be able to run an executable only if it has been compiled there. An executable compiled on a local PC and copied to the cluster will not run.

Compilation requires logging to the HPC system via terminal. For example, using ssh on Cygwin, one can simply open the terminal and execute the command below. If ssh is not recognized, this means that the Cygwin installation did not include it. This can be easily fixed by running the Cygwin installer, and choosing to include openssh in the installation (or any equivalent if Cygwin is not used).

# Login to your account
ssh $(yourusername).$(hpcname).$(address)

One inside the HPC system, the compilation follows a similar procedure as for a local machine, but the correct modules need to be loaded.

CPU version

# Navigate to the target directory  
cd $(yourdirectory) 

# load compilation modules
module load CMake
module load GCC/12.3.0

# clone the repository and move to HYBIRD directory 
git clone https://github.com/gnomeCreative/HYBIRD.git
cd HYBIRD

# Switch to the last stable CPU branch
git checkout $(cpu_stable_branch)

# Create the build directory and change into it
mkdir -p build && cd build

# Configure CMake from the command line passing configure-time options.
cmake .. -DCMAKE_BUILD_TYPE=Release

# Build the required targets, in this case all of them, using 8 threads.
cmake --build . --parallel 8

GPU version

HYBRID can also be run in the GPU version on an HPC system. In this case, the Cygwin terminal is sufficient, as the software is neither compiled nor executed locally. To compile the code, and once logged in the HPC system (see above) follow these steps:

# check if CUDA nodes are available
sinfo --partition=gpu

# switch from login node to interactive GPU node (must for compilation)
# Get an A100 interactive job (if available request 1/4 node, otherwise it waits for one to become available)
srun --pty --partition=gpu --qos=gpu --gpus-per-node=1 --mem=80G --cpus-per-task=12 /bin/bash -i
# Alternatively, get an H100 interactive job (if available request 1/4 node, otherwise it waits for one to become available)
srun --pty --partition=gpu-h100 --qos=gpu --gpus-per-node=1 --mem=80G --cpus-per-task=12 /bin/bash -i

# load compilation modules
module load CMake
module load GCC/12.3.0
module load CUDA/12.4.0

# clone the repository and navigate to the software directory
git clone https://github.com/gnomeCreative/HYBIRD.git
cd HYBIRD

# Switch to stable GPU branch
git checkout cuda_freesurface

# Setup CMake build (80 denotes A100 support, 90 denotes H100 support)
mkdir build
cd build
cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"

# Compile
cmake --build . --parallel 12
Important: the line 
cmake .. -DCMAKE_CUDA_ARCHITECTURES="80;90"
needs to be adaped to the specific GPU architecture of the system. This can be easily checked online.
⚠️ **GitHub.com Fallback** ⚠️