Table of Contents How to run Caffe in EENet 1. Install protobuf 2. Install Caffe 3. Testing the installation

EENet has most of the prerequisites already installed, only protobuf is an old version. To install recent version of protobuf:

wget https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.gz
tar xzf protobuf-2.6.1.tar.gz
cd protobuf-2.6.1
./configure --prefix=$HOME
make
make install

I’m happy to have all stuff installed to $HOME/lib and $HOME/bin. If you want something else, use ./configure --prefix=$HOME/protobuf or similar.

Add those directories to corresponding paths in your .bash_profile or .bashrc before standard path:

export PATH=$HOME/bin:$PATH
export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH:/opt/opencv-2.4.10/lib

While we are at it, I added OpenCV as well. Log out-in so they take effect.

git clone https://github.com/tambetm/caffe.git
cd caffe
cp Makefile.config.example Makefile.config

Now change following lines in Makefile.config:

1. Comment out lines 26-27:

   #             -gencode arch=compute_50,code=sm_50 \
   #             -gencode arch=compute_50,code=compute_50
   

2. To make use of Intel Math Library for faster matrix operations change lines 33, 37 and 38:

   BLAS := mkl
   ...
   BLAS_INCLUDE := /opt/intel/mkl/include
   BLAS_LIB := /opt/intel/mkl/lib

3. Change line 48 to point to Numpy installation in your virtualenv sandbox:

                 $(HOME)/sandbox/lib64/python2.7/site-packages/numpy/core/include

4. Add protobuf and OpenCV directories to library and include paths at lines 62-63:

   INCLUDE_DIRS := $(PYTHON_INCLUDE) $(HOME)/include /usr/local/include /opt/opencv-2.4.10/include
   LIBRARY_DIRS := $(PYTHON_LIB) $(HOME)/lib /usr/local/lib /usr/lib /opt/opencv-2.4.10/lib 

   NB! Protobuf includes and libraries must go before /usr/local!!!

In addition you need to change the line 228 in Makefile to:

        LIBRARIES += boost_thread-mt stdc++

Then it is useful to push compilation to cluster nodes:

srun --cpus-per-task=8 make all -j8
srun --cpus-per-task=8 make test -j8
srun --gres=gpu:1 --constraint=K20 make runtest
srun make pycaffe

Activate Python virtualenv and install missing dependencies:

source ~/sandbox/bin/activate
pip install jinja2
pip install tornado
pip install pyzmq
pip install jsonschema
pip install ipython
pip install matplotlib
pip install protobuf
pip install pyyaml
pip install scikit-image

Then run IPython notebook on GPU server, without browser and listening all interfaces:

cd examples
srun --gres=gpu:1 --constraint=K20 ipython notebook --no-browser --ip='*'

Now you have to figure out, on which node your code runs. Use squeue command in other window for that. Once you have figured that out, open in your browser something like http://idu41.grid.eenet.ee:8888.

NB! Leaving that IPython notebook open without any security for long time is potentially dangerous!