Remote Judge - houyhlouis/LLMOJ GitHub Wiki

Independent judge installation

This guide installs only judge, without web/database/Redis/MinIO services. Each judge has its own key and initiates the web-server connection. Use Ubuntu 24.04 / 26.04 amd64 (fresh 26.04 deployment remains unverified), systemd 254+, unified cgroup v2, at least 2 effective logical CPUs and 256 MiB effective RAM for one execution slot, and 30 GiB free disk. Building on the judge needs additional RAM; 8 GiB is recommended for compilation. Restricted namespace/mount/cgroup containers are unsupported.

1. Download the same source version

Read sourceCommit in the web node's config/install-complete.json, or select the same release tag for both nodes. Replace the repository and version:

sudo apt-get update
sudo apt-get install -y ca-certificates curl git xz-utils build-essential cmake \
  libfmt-dev python3 python3-yaml util-linux pkg-config
sudo git clone https://github.com/houyhlouis/LLMOJ.git /opt/LLMOJ
cd /opt/LLMOJ
sudo git checkout WEB_SERVER_COMMIT_OR_TAG
# Source archives include testlib; initialize only if testlib.h is absent:
if [ ! -f apps/judge/vendor/testlib/testlib.h ]; then
  sudo git submodule update --init --recursive
fi

2. Install pinned Node/pnpm and build judge

Versions and checksum match the installer. Stop on a checksum failure; do not bypass validation.

OJ_NODE_STAGE=$(mktemp -d)
curl --proto '=https' --tlsv1.2 -fL --retry 3 \
  https://nodejs.org/dist/v24.21.0/node-v24.21.0-linux-x64.tar.xz \
  -o "$OJ_NODE_STAGE/node.tar.xz"
printf '%s  %s\n' \
  fd8e59d5a511510f6a298afb548f18c7d2b1be404d8b4a27d94fbe49f56cb2d6 \
  "$OJ_NODE_STAGE/node.tar.xz" | sha256sum --check
# Continue only after the checksum succeeds:
sudo install -d -m 0755 /opt/LLMOJ/runtime
sudo tar -xJf "$OJ_NODE_STAGE/node.tar.xz" -C /opt/LLMOJ/runtime
sudo ln -sfn node-v24.21.0-linux-x64 /opt/LLMOJ/runtime/node
rm -r -- "$OJ_NODE_STAGE"

sudo env PATH=/opt/LLMOJ/runtime/node/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  npm install --prefix /opt/LLMOJ/runtime/tooling --ignore-scripts --no-audit --no-fund [email protected]

sudo env PATH=/opt/LLMOJ/runtime/node/bin:/opt/LLMOJ/runtime/tooling/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  npm_config_nodedir=/opt/LLMOJ/runtime/node CMAKE_BUILD_PARALLEL_LEVEL=2 \
  NODE_OPTIONS=--max-old-space-size=2300 pnpm install --frozen-lockfile --store-dir runtime/pnpm-store

Continue building with the same tool path:

sudo env PATH=/opt/LLMOJ/runtime/node/bin:/opt/LLMOJ/runtime/tooling/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
  npm_config_nodedir=/opt/LLMOJ/runtime/node CMAKE_BUILD_PARALLEL_LEVEL=2 \
  NODE_OPTIONS=--max-old-space-size=2300 pnpm_config_verify_deps_before_run=false bash -e -c '
    pnpm build:artifacts
    pnpm --filter simple-sandbox build
    pnpm build:native
    pnpm --filter @libreoj/judge build
  '

Workspace dependencies are installed. build:artifacts also builds shared frontend/bootstrap artifacts; it does not install or start a web service on this judge node.

3. Prepare the original rootfs

Follow Docker's Ubuntu guide to install Engine and buildx. For custom sources, see Docker-Sources. Then build:

sudo docker info
sudo docker buildx version
sudo env BUILDX_BUILDER=default bash deploy/sandbox/build.sh build
bash deploy/sandbox/build.sh plan

Use the actual ROOTFS_ID and archive path printed by the plan/build. The following matches the installer's host profile: x32 is compiled but not executed; amd64/i386 and language checks still run.

sudo env LIBREOJ_X32_SUPPORTED=0 unshare --net --mount --propagation private -- \
  bash deploy/sandbox/build.sh stage /opt/LLMOJ/runtime/sandbox-archives/rootfs-ACTUAL_ROOTFS_ID.tar.gz

Alternatively, build the identical recipe on a trusted build machine and copy rootfs-<ID>.tar.gz plus its .sha256 file to the judge for staging and language verification. This avoids Docker on the judge itself. Do not mix versions. Docker is for building; submissions execute in native simple-sandbox.

4. Save the private key and generate config

Register the node using Distributed-Judging. Put its key on a single line using an editor, keeping it out of command history:

Create the key file below only on first setup; if /root/judge-01.key already exists, retain it instead of overwriting it.

sudo install -m 0600 /dev/null /root/judge-01.key
sudoedit /root/judge-01.key
sudo install -d -m 0751 /opt/LLMOJ/data
sudo install -d -m 0700 /opt/LLMOJ/data/judge /opt/LLMOJ/data/judge/testdata /opt/LLMOJ/data/judge/cache
sudo env HYHOJ_ROOT=/opt/LLMOJ NODE_BINARY=/opt/LLMOJ/runtime/node/bin/node \
  OJ_JUDGE_REMOTE=1 OJ_JUDGE_SERVER=https://oj.example.com \
  OJ_JUDGE_KEY_FILE=/root/judge-01.key OJ_JUDGE_SLOTS=2 \
  python3 deploy/configure-judge.py

The server URL must be an origin without /api. The key file must be regular, root-owned and 0600; its contents are not printed. Initial explicit slots must fit effective CPU/RAM capacity and the 511 libuv limit. Omitting OJ_JUDGE_SLOTS uses min(effective CPUs-1, effective RAM MiB//256, 511) for both default and maximum, with no extra memory reservation. One effective CPU or less than 256 MiB RAM gives zero capacity and is rejected. This variable does not resize existing YAML; use the maintained resize script. See Judge configuration for every field, thread-pool sizing and existing-node resizing.

The generated config/judge.yaml is also root-only. Remote mode disables the local AI socket and creates libreoj-judge.target. Reruns retain config and reject a different server/key; rotate credentials explicitly after stopping the judge.

5. Register systemd and verify isolation

Install only judge units/workspace mounts, avoiding the web service templates:

sudo chmod -R a+rX apps packages node_modules runtime/node-v24.21.0-linux-x64 runtime/tooling runtime/pnpm-store
shopt -s nullglob
OJ_JUDGE_MOUNTS=(deploy/systemd/*.mount)
sudo systemd-analyze verify deploy/systemd/libreoj-judge.service \
  deploy/systemd/libreoj-judge.target "${OJ_JUDGE_MOUNTS[@]}"
sudo install -m 0644 deploy/systemd/libreoj-judge.service \
  deploy/systemd/libreoj-judge.target "${OJ_JUDGE_MOUNTS[@]}" /etc/systemd/system/
sudo systemctl daemon-reload
for OJ_JUDGE_MOUNT in "${OJ_JUDGE_MOUNTS[@]}"; do
  sudo systemctl enable --now "$(basename "$OJ_JUDGE_MOUNT")"
done
sudo systemd-run --unit=libreoj-install-sandbox-check --wait --pipe --collect \
  --property='Delegate=cpu memory pids' --property=DelegateSubgroup=supervisor \
  --property=LimitCORE=0 --property=RuntimeMaxSec=120 --property=TimeoutStopSec=15 \
  --property=WorkingDirectory=/opt/LLMOJ/apps/judge \
  /opt/LLMOJ/runtime/node/bin/node /opt/LLMOJ/deploy/sandbox/verify-installed.mjs --root /opt/LLMOJ
# Start judging only after the sandbox check succeeds:
sudo systemctl enable --now libreoj-judge.target
sudo systemctl status libreoj-judge.service
sudo journalctl -u libreoj-judge.service -n 50 --no-pager

The check compiles and runs a real program and verifies UID, read-only rootfs, network isolation, the syscall filter, tmpfs and cgroup limits. Afterwards, sudo bash /opt/LLMOJ/deploy/sandbox/self-test.sh checks every language (self-test). Never recursively change rootfs ownership; it contains the sandbox UID.

Confirm the judge is online and submit programs on the web site. No inbound judge web port is required; allow outbound access to the web origin/download sources. Stop with sudo systemctl stop libreoj-judge.target; disable startup with sudo systemctl disable libreoj-judge.target.

Generated configuration/units have isolated checks; full independent-machine installation and real multi-node scheduling still require on-site verification.