Installation_Linux - dividiti/ck-caffe GitHub Wiki

[ Home ]

Installing CK-Caffe on Linux

Before installing CK-Caffe on the target system, several libraries and programs should be installed. So far, instructions for the following Linux flavours are available:

Conventions

In this guide, shell commands prefixed with '$' should be run as user, whereas commands prefixed with '#' should be run as root (or as user with 'sudo').

For example, to install the 'pip' package manager and then Jupyter on Ubuntu, run as root:

# apt install python-pip
# pip install jupyter

or as user:

$ sudo apt install python-pip
$ sudo -H pip install jupyter

[Ubuntu] Installing CK-Caffe dependencies

Installing the dependencies is recommended via 'apt install' (for standard Ubuntu packages), or 'pip install' (for standard Python packages, typically of more recent versions than those available via 'apt install'). This can be simply done by opening a Linux shell and copying-and-pasting commands from cells below.

[Ubuntu] Installing core CK dependencies

Collective Knowledge has only two dependencies: Python (2.x and 3.x) and Git, which can be installed as follows:

# apt install  \
    python-dev \
    git

[Ubuntu] Installing common dependencies

Some CK packages and Caffe require common Linux utilities (e.g. make, cmake, wget), which can be installed as follows:

# apt install \
    coreutils \
    build-essential \
    make \
    cmake \
    wget \
    python-pip

[Ubuntu] Installing Caffe dependencies

The BVLC Caffe framework has quite a few dependencies. If you've already run Caffe on your machine, it's likely that you've already satisfied all of them. If not, however, you can easily install them in one gollop as follows:

# apt install \
    libboost-all-dev \
    libgflags-dev \
    libgoogle-glog-dev \
    libhdf5-serial-dev \
    liblmdb-dev \
    libleveldb-dev \
    libprotobuf-dev \
    protobuf-compiler \
    libsnappy-dev \
    libopencv-dev
# pip install \
    protobuf

NB: On NVIDIA's Tegra platform, install OpenCV either via JetPack or something like:

# apt install \
    libopencv4tegra-dev

[Ubuntu] Installing optional dependencies

# apt install \
    libatlas-base-dev
# pip install \
    jupyter \
    pandas numpy scipy matplotlib \
    scikit-image scikit-learn \
    pyyaml

[Ubuntu] Checking all dependencies

You can check all the dependencies on an Ubuntu system by running this notebook. (View the output of this notebook on an Odroid XU3 board here.)

[Ubuntu] Installing CK

Please proceed to installing CK.

[Gentoo] Installing CK-Caffe dependencies

Installing the dependencies is recommended via 'emerge' (for standard Gentoo packages), or 'pip install' (for standard Python packages, typically of more recent versions than those available via 'emerge'). This can be simply done by opening a Linux shell and copying-and-pasting commands from cells below.

[Gentoo] Installing core CK dependencies

Collective Knowledge has only two dependencies: Python (2.x and 3.x) and Git, which can be installed as follows:

# emerge  \
    dev-lang/python \
    dev-vcs/git

[Gentoo] Installing common dependencies

Some CK packages and Caffe require common Linux utilities (e.g. make, cmake, wget), which can be installed as follows:

# emerge \
    sys-devel/gcc \
    sys-devel/make \
    dev-util/cmake \
    net-misc/wget \
    dev-python/pip

[Gentoo] Installing Caffe dependencies

The BVLC Caffe framework has quite a few dependencies. If you've already run Caffe on your machine, it's likely that you've already satisfied all of them. If not, however, you can easily install them in one gollop as follows:

# emerge \
    dev-libs/boost \
    dev-util/boost-build \
    dev-cpp/gflags \
    dev-cpp/glog \
    sci-libs/hdf5 \
    dev-db/lmdb \
    dev-libs/leveldb \
    dev-libs/protobuf \
    app-arch/snappy \
    media-libs/opencv
# pip install \
    protobuf

[Gentoo] Installing optional dependencies

# emerge \
    sci-libs/atlas
# pip install \
    jupyter \
    pandas numpy scipy matplotlib \
    scikit-image scikit-learn \
    pyyaml

[Gentoo] Installing CK

Please proceed to installing CK.

[Yocto] Installing CK-Caffe dependencies

NB: This section is work-in-progress.

NB: Not all CK-Caffe dependencies can be automatically installed on Yocto.

Add the following Bitbake layers to your bblayers.conf (e.g. build/build/conf/bblayers.conf):

BBLAYERS ?= " \
  ${TOPDIR}/../poky/meta \
  ${TOPDIR}/../poky/meta-yocto \
  ${TOPDIR}/../poky/meta-yocto-bsp \
  ...
  ${TOPDIR}/../meta-linaro/meta-linaro-toolchain \
  ${TOPDIR}/../meta-openembedded/meta-oe \
  ${TOPDIR}/../meta-openembedded/meta-networking \
  ${TOPDIR}/../meta-openembedded/meta-multimedia \
  ${TOPDIR}/../meta-openembedded/meta-python \
  "

[Yocto] Installing core CK dependencies

Collective Knowledge has only two dependencies: Python (2.x and 3.x) and Git.

Add the following to your image recipes:

# Core CK dependencies.
IMAGE_INSTALL_append... = " \
    python \
    git \
"

[Yocto] Installing common dependencies

Some CK packages and Caffe require common Linux utilities (e.g. make, cmake, wget).

Add the following to your image recipes:

# Common CK-Caffe dependencies.
IMAGE_INSTALL_append... = " \
    gcc \
    make \
    cmake \
    wget \
    zlib \
    python-setuptools \
    python-pip \
"

[Yocto] Installing Caffe dependencies

The BVLC Caffe framework has quite a few dependencies. If you've already run Caffe on your machine, it's likely that you've already satisfied all of them. If not, however, you may need to install some of them manually.

[Yocto] Installing Caffe dependencies automatically

You can install some of the Caffe dependencies automatically by adding the following to your image recipes:

# Caffe dependencies.
IMAGE_INSTALL_append... = " \
    boost \
    libunwind \
    glog \
    protobuf \
    leveldb \
    opencv \
    opencv-samples \
    libopencv-core \
    libopencv-highgui \
    libopencv-imgproc \
    libopencv-features2d \
    libopencv-calib3d \
    libopencv-flann \
    libopencv-ocl \
"

NB: This list probably overapproximates the real dependencies.

Also, run:

# pip install \
    protobuf

[Yocto] Installing Caffe dependencies manually

[Yocto] hdf5

TODO

[Yocto] lmdb
$ cd tmp/
$ wget https://github.com/LMDB/lmdb/archive/LMDB_0.9.18.tar.gz
$ tar xvzf LMDB_0.9.18.tar.gz
$ cd lmdf-LMDB-0.9.18/libraries/liblmdb
$ make && make install

NB: Installs to /usr/local/ by default.

[Yocto] gflags
$ cd /tmp
$ wget https://github.com/gflags/gflags/archive/v2.1.2.tar.gz
$ cd gflags-2.1.2
$ mkdir -p build && cd build
$ cmake .. -DCMAKE_CXX_FLAGS=-fPIC
[Yocto] snappy
$ cd /tmp
$ wget https://github.com/google/snappy/releases/download/1.1.3/snappy-1.1.3.tar.gz
$ tar xvzf snappy-1.1.3.tar.gz
$ cd snappy-1.1.3
$ ./configure
$ make -j4 && make install

NB: Installs to /usr/local/ by default.

[Yocto] Installing CK

Please proceed to installing CK.

Installing CK

Clone CK from GitHub into e.g. '$HOME/CK':

$ git clone https://github.com/ctuning/ck.git $HOME/CK

Add the following to your '$HOME/.bashrc' and run 'source ~/.bashrc' after that:

# Collective Knowledge.
export CK_ROOT=${HOME}/CK
export CK_REPOS=${HOME}/CK_REPOS
export CK_TOOLS=${HOME}/CK_TOOLS
export PATH=${HOME}/CK/bin:$PATH

Install the Python interface to CK:

$ cd $HOME/CK && sudo python setup.py install

Test that both the command line and Python interfaces work:

$ ck version
V1.8.2dev
$ python -c "import ck.kernel as ck; print (ck.__version__)"
V1.8.2dev

Feedback

Please send your comments and feedback to http://groups.google.com/group/collective-knowledge .