conda pack use - core-unit-bioinformatics/knowledge-base GitHub Wiki

How to use conda-pack

Deploying environments on a cluster without internet access is problematic if either no local mirrors for the relevant conda channels exist of if packages dynamically download additional data during setup; for example, this is the case for many genomic feature or annotation packages from the R Bioconductor ecosystem. The only option left to deploy environments on offline systems HPC is to use conda-pack, a tool for packaging conda environments created on system A (online) and porting them to system B (offline).

The last update for the conda-pack documentation was in 2017, making some commands outdated.

It is strongly recommended not to install any additional packages in your conda base environment. The only exception to that rule may be conda-pack, and the following instructions assume that.

  1. install conda-pack on system A (online)
$ conda install conda-pack
  1. create the conda environment to be packaged on system A and pack it with conda-pack
# for example
$ conda env create -f env_spec.yaml -p ./env_prefix

# hint: read through conda-pack --help
$ conda pack -p ./env_prefix -o my_env.tar.gz -j 6
  1. transfer the .tar.gz file to system B (offline) using the appropriate means (rsync, scp, Globus...)

  2. Untar the .tar.gz file; optionally, untar into a specific directory

# unpack 'here'
$ tar -xzf my_env.tar.gz

# unpack into specific folder
$ mkdir new_env
$ tar -xzf my_env.tar.gz -C new_env
  1. Activate the environment
# if unpacked into specific folder
$ conda activate -p ./new_nev
  1. After activating the environment, the script conda-unpack is available in your $PATH. This script rewrites path prefixes and must be executed to properly unpack / deploy the conda environment on system B.
(new_env) $ conda-unpack
  1. Deactivate and reactivate the environment - it should now be working as on system A
(new_env) $ conda deactivate
$ conda activate ./new_env
# environment should be working now
⚠️ **GitHub.com Fallback** ⚠️