Valeria - norlab-ulaval/Norlab_wiki GitHub Wiki
Ressources:
8xA100 80 Gb
1xA100 40 Gb
When you need exclusive access to these ressources, you can reserve them for a predetermined time slot by asking [email protected]
.
See public/exemples
on Valeria for examples.
Interactive session: salloc --account=ul-val-prj-def-phgig4 --cpus-per-task=8 --time=1:00:00 --gres=gpu:a100:1 --partition=gpu
Start by signing in using your IDUL at Créer un compte VALERIA.
You might need to email Valeria for collaborator status if you get errors.
SSH accessible via ULaval's VPN: ssh <user>@login.valeria.science
To add ssh keys, you must contact Valeria's support.
You can read Tutorial: Upload a dataset to VALERIA for a more up to date guide on uploading data to VALERIA.
Read Résumé des caractéristiques du stockage Lustre et Objet (s3) de VALERIA to find what storage is the most adapted for your use case.
We recommend to use Globus, and to store datasets in projects/ul-val-prj-def-phgig4/
.
Clone willGuimont/docker-globus and follow the instructions to run Globus in a Docker container, be sure to modify the volume to point to what you want to transfer.
Go to Mes outils, and start the Globus web client. You can then follow Valeria's documentation: Procédure pour transférer vos fichiers avec Globus.
Store datasets in projects/ul-val-prj-def-phgig4/
.
Install rclone
, then add the following to .config/rclone/rclone.conf
[valeria]
type = s3
provider = Ceph
env_auth = false
access_key_id = <access key from valeria>
secret_access_key = <secret key from valeria>
endpoint = s3.valeria.science
acl = private
Validate it can access Valeria using rclone size valeria:
# Single file
rclone copy my/local/file valeria:my/remote/folder
# Complete folder
rclone sync my/local/dataset valeria:my/remote/dataset
SSH to Valerian ssh <user>@login.valeria.science
, run val-generate-s3-config
, to have access to rclone
from Valeria.