Tips and tricks for using JupyterLab on Hulk - Hansen-Group/HansenWiki GitHub Wiki

Jupyter Lab

This line adds the $HOME/bin folder to the PATH, so every script, binary with execution permissions will be able to be executable everywhere the PATH is available. You don't need to use the full path, as long as you have linked the executable to the PATH folder. The syntax

cat <<EOF >> myfile.txt
    something 
    something else    
EOF

is for cat a block of lines.

mkdir -p $HOME/bin

cat <<EOF >> $HOME/.bash_profile
if [ -d "$HOME/bin" ] ; then
  export PATH=$PATH:$HOME/bin
fi
EOF

Conda installation, just in case you don't like the current system version you can override and specific version as long as you have been using only and only one version.

mkdir -p $HOME/bin && cd "$_" 
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p $HOME/bin/miniconda3
rm Miniconda3-latest-Linux-x86_64.sh

Add miniconda bin path to $PATH

echo -e 'export PATH=$HOME/bin/miniconda3/bin:$PATH' >>$HOME/.bash_profile

reload path to apply changes to the PATH

source $HOME/.bash_profile

Add extra channels to Conda

cat <<EOF >>$HOME/.condarc
channels:
  - conda-forge
  - bioconda
  - defaults
EOF

REMOTE

1a. Create a conda environment for R and Julia to avoid conflict with a local installation.

When you have a lot of dependencies I suggest mamba, a multithread version of conda.

conda install -yc conda-forge mamba

mamba create -n jupyteR -yc r -c conda-forge \
    r-essentials r-base ipykernel jupyterlab r-irkernel

Install Julia and the Jupyter-Julia kernel

mkdir $HOME/bin && cd "$_"
wget https://julialang-s3.julialang.org/bin/linux/x64/1.6/julia-1.6.4-linux-x86_64.tar.gz
tar zxvf julia-1.6.4-linux-x86_64.tar.gz && rm julia-1.6.4-linux-x86_64.tar.gz

echo 'export PATH=$PATH:$HOME/bin/julia-1.6.4/bin' >> ~/.bash_profile
source $HOME/.bash_profile

julia -e 'import Pkg; Pkg.update(); Pkg.add("IJulia")'

2. Configure a password to avoid using Token

You only need to do this the first time

source activate jupyteR

jupyter lab --generate-config
jupyter lab password

3. Launch jupyter

Non-available ports in Hulk's local address

netstat -lntu | awk 'NR>2 {print $4}' | grep -E '0.0.0.0:' | sed 's/.*://' | sort -n | uniq

Select an available port in a range of 8800-8899, just because Jupyter notebooks convention ports. Now, using a terminal multiplexer.

tmux -u new -s julia

srun -J notebook -N 1 \
    --ntasks-per-node=1 --pty $(which bash) -c \
    "jupyter lab --ip 0.0.0.0 \
        --port 8887 --no-browser \
        --notebook-dir=/emc/cbmr/users/$USER/notebooks"

# Tmux dettach
# CTRL + B + D

4. Get the local IP from the network using VPN

hostname -I

For me, this is the local address of the server to use through ssh: 10.128.7.72. You can use 'hulk' as well.

LOCAL

5. Open a browser tab with the address, and introduce your password.

10.128.7.72:8888/lab

Or:

hulk:8888/lab

At the moment the previous step only works with Pulse Secure. However, you can use the Malte's solution, tunneling through Porus

6. Connect to the server using SSH via Porus

ssh -o TCPKeepAlive=no \
    -o ServerAliveCountMax=20 \
    -o ServerAliveInterval=15 \
    -N -L 8000:10.128.7.72:8888 [email protected]

Or

ssh -o TCPKeepAlive=no \
    -o ServerAliveCountMax=20 \
    -o ServerAliveInterval=15 \
    -N -L 8000:hulk:8888 [email protected]
⚠️ **GitHub.com Fallback** ⚠️