Conda in offline envs - core-unit-bioinformatics/knowledge-base GitHub Wiki

Creating conda environments on offline infrastructure

Problem:

  • You want to create a conda environment on a compute infrastructure without internet access
    • typical scenario: HPCs / compute clusters

Solution 1:

  • Create the Conda environment on a different system (e.g., your laptop), pack it, transfer it to the offline system and unpack it.

Solution 2:

  • If the offline infrastructure offers local mirrors to popular Conda channels such as conda-forge or bioconda, making use of those channel mirrors typically requires configuring your .condarc file in a specific way.

.condarc file

More info in the official docs: docs.conda.io

Important Several .condarc files may exist in your environment. Locate all active ones as follows:

$ conda config --show-sources

# prints something like the following on your screen

==> /home/YOURUSER/.condarc <==  # standard .condarc location in $HOME
add_pip_as_python_dependency: True
allow_softlinks: False
auto_update_conda: False
env_prompt: ({name})
default_threads: 2
envs_dirs:
  - /your/system/path/to/conda/installation/envs  # default location for conda envs
pkgs_dirs:
  - /your/system/path/to/conda/installation/pkgs  # default location to store downloaded package data
channel_priority: strict
channels:
  - https://conda.anaconda.org/conda-forge  # example: default channel URL that requires internet access
  - https://conda.anaconda.org/bioconda
  - nodefaults
allowlist_channels:
  - https://conda.anaconda.org/conda-forge
  - https://conda.anaconda.org/bioconda
  - nodefaults
denylist_channels:
  - defaults
always_yes: False

If several .condarc files are found, delete or rename all but one (preferably, the one in your $HOME) to avoid configuration conflicts because all .condarc files will be parsed and evaluated (precedence depends on parsing order). Note that the infrastructure admins may set a system-wide .condarc file, which would normally be protected from any changes by unprivileged users.

HHU-specific: HPC HILBERT

Only for HHU members

  • HILBERT offers Conda channel mirrors; please find the official documentation in the HHU Wiki.

On HILBERT, a valid .condarc file may look like this:

HILBERT $ conda config --show-sources

==> /home/YOURUSER/.condarc <==
allow_softlinks: False
auto_update_conda: False
auto_activate_base: False
env_prompt: ({name})
envs_dirs:
  - /your/system/path/to/conda/installation/envs
pkgs_dirs:
  - /your/system/path/to/conda/installation/pkgs
channel_priority: strict
channels:
  - http://conda.repo.test.hhu.de/conda-forge  # example: URL of channel mirror that is reachable from HILBERT
  - http://conda.repo.test.hhu.de/bioconda
  - http://conda.repo.test.hhu.de/main
show_channel_urls: True
⚠️ **GitHub.com Fallback** ⚠️