03 Installation - netassist-ua/netgraphz2-gpl GitHub Wiki

Obtaining NetGraphz2

To obtain latest development version of NetGraphz2 clone our git repository. By default we suppose installing NetGraphz2 into /opt/netgraphz2-gpl directory. For future purposes we call this directory as installation root directory or installation directory, path provided in documentation specified relative to this directory.

cd /opt
git clone "https://github.com/netassist-ua/netgraphz2-gpl.git"

Unpacking

  • Clone or unpack release distribution archive into /opt/netgraphz2-gpl. You can setup NetGraphz2 into the custom path modifying configuration directives right way.
  • Every file and executable relative path is specified in respect of /opt/netgraphz2 in the instruction below.

Prerequirements installation

Users and folders

Add netgraphz2 user to your system in order to run NetGraphz2 applications

adduser netgraphz2 --system --disabled-password --no-create-home
usermod netgraphz2 -G daemon

Create log directory for the application

mkdir /var/log/netgraphz2
chown netgraphz2 /var/log/netgraphz2

Basic packages

  • Install required applications and components: Neo4j, MongoDB, PHP >= 5.6 + FPM, PHP phalcon extension, Apache HTTPd or NGINX web-server with FastCGI support (we recommend to use Apache 2.4 with proxy_fcgi extension), Python2 >= 2.7.5, Icinga 2

In Debian Jessie you can install it all by following commands.

Add repositories for thrird-party applications:

wget -O - https://debian.neo4j.org/neotechnology.gpg.key | sudo apt-key add - 
echo 'deb http://debian.neo4j.org/repo stable/' >/tmp/neo4j.list
mv /tmp/neo4j.list /etc/apt/sources.list.d
echo 'deb http://http.debian.net/debian jessie-backports main'  > /etc/apt/sources.list.d/backports.list
wget -O - http://packages.icinga.org/icinga.key | apt-key add - 
echo 'deb http://packages.icinga.org/debian icinga-jessie main' > /etc/apt/sources.list.d/icinga.list
echo 'deb-src http://packages.icinga.org/debian icinga-jessie main' >> /etc/apt/sources.list.d/icinga.list
apt-get update

Install required packages

apt-get install neo4j icinga2 mongodb python python-dev python-pip apache2 php5-dev gcc libpcre3-dev git php5 php5-fpm php5-mongo php5-json php5-mysql php5-imagick php5-intl php5-gd php5-mongo php5-memcache php5-memcached php5-rrd php-pear php5-mcrypt make

  • Install py2neo Python2 extension. Required to run Icinga2 configuration generator script, importer scripts for our customer. If you set up graph from the stratch, probably you don't need this extensions so you can skip this step.

      pip install py2neo

  • Compile and install PHP Phalcon extension. Required to run NetGraphz2 web interface

      cd /usr/src
  git clone --depth=1 git://github.com/phalcon/cphalcon.git
  cd cphalcon/build
  ./install

  • Enable Phalcon PHP extension. Append line: extension=phalcon.so into your php.ini . In Debian Jessie better use following commands:

      echo 'extension=phalcon.so' > /etc/php5/mods-available/phalcon.ini
  php5enmod phalcon `

  • Install PECL Spl_Types extension

      pecl install SPL_Types

  • Enable PECL Spl_Types extension. Append line: extension=spl_types.so into your php.ini. In Debian Jessie:

      echo 'extension=spl_types.so' > /etc/php5/mods-available/spl_types.ini
  php5enmod spl_types

  • Configure PHP FPM:

  • Set listening TCP port to 9000 for pool (/etc/php5/fpm/pool.d/www.conf in Debian):

      listen = 127.0.0.1:9000

  • ...comment or delete line

       listen = /var/run/php5-fpm.sock

    Change automatic restart parameters (PHP sometimes falls down with SEGV) for daemon (/etc/php5/fpm/php-fpm.conf in Debian):

       emergency_restart_threshold = 2
   emergency_restart_interval = 5s

  • Configure your web-server to enable URL rewrite, PHP FastCGI processing. For Debian Jessie:

      a2enmod proxy_fcgi
  a2enmod rewrite

  • Restart PHP-FPM and Apache2

For Debian Jessie use service php5-fpm restart and service apache2 restart commands.

gRPC installation

For Debian: libgprc-dev in repository is doesn't work well. Follow instructions below.

  • Download and install protobuf 3

    • Clone git repository

        git clone "https://github.com/netassist-ua/protobuf.git"

    • Install prequirements: curl, autoconf, automake, libtool g++, gcc, make.

      For Debian Jessie:

       apt-get install curl autoconf automake libtool g"++"

    • Compile and install:

         cd protobuf
   ./autogen.sh
   ./configure
   make
   make install
   ldconfig

  • Install dependencies: zlib, openssl development files. For Debian execute:

      apt-get install zlib1g-dev libssl-dev autoconf build-essential libtool

  • Pull gRPC source from git

      git clone -b php_ext_argtype_fix https://github.com/netassist-ua/grpc.git
  cd grpc
  git submodule update --init

  • Compile gRPC from git

      make 
  make install

  • Compile and install PHP gRPC extension

      cd src/php/ext/grpc
  phpize
  ./configure
  make
  make install

  • Enable gRPC extension for PHP. Add line extension=grpc.so into your php.ini.

    For Debian Jessie:

      echo 'extension=grpc.so' > /etc/php5/mods-available/grpc.ini
  php5enmod grpc

Notifications server

First of all you should install Node.js and NPM (Node.js package manager). You need Node.js version 0.10.2 or later to run notification server. Usual steps for Debian Jessie +:

apt-get install nodejs nodejs-dev npm
ln -s /usr/bin/nodejs /usr/bin/node

After installing packages, follow notifications subfolder and download dependencies for notification server application.

cd notifications
npm install

Systemd startup unit

We have a systemd startup file for Notification server. It has path /opt/netgraphz2-gpl default path to the notification server application. If you have other installation location, you can change path by modifying this startup definition file. To enable this startup definition:

cp systemd/netgraphz2.notifications.service /lib/systemd/system`
systemctl enable netgraphz2.notifications`

Backend

First you should have Golang runtime >= 1.3.3 installed. In Debian Jessie we have usual boring steps:

apt-get install golang

We recommend also to install backported version of Golang runtime (some not very clever Debian maintainer broken compiler 08.02.2016)

apt-get install golang-go/jessie-backports

In order to obtain dependencies you should have Bazaar and Git client installed. For Debian Jessie:

apt-get install bzr git

Obtain dependencies

Backend package use some external dependencies easy to obtain by Go utilities. Change directory to your installation location and execute following commands:

cd backend
source ./set_env
pushd .
cd src/ng_backend
go get
popd
pushd .
cd src/ng_cli
go get
popd

Build and install backend:

The final step is to build and install binaries. Change directory to your installation location and execute following commands:

cd backend
source ./set_env
go build ng_backend
go build ng_cli
go install ng_backend
go install ng_cli

Backend binary will be placed into backend/bin folder of the installation directory. There is a sample configuration file named config.json available at the same location.

Systemd startup unit

We have a systemd startup file for backend. It has path /opt/netgraphz2-gpl default path to the notification server application. If you have other installation location, you can change path by modifying this startup definition file. To enable this startup definition:

cp systemd/netgraphz2.backend.service /lib/systemd/system
systemctl enable netgraphz2.backend

Web frontend

After installation of required packages, installing web frontend is easy.

  • Install vendor packages by Composer:

      cd web-phalcon
  curl -sS https://getcomposer.org/installer | php
  php composer.phar install

  • Point web-site root directory to web-phalcon/public of your installation directory

  • Make directory with web-phalcon/app/cache and chmod it to 777

If you want to use Apache 2.4 on Debian Jessie, you can follow next steps:

  • Execute from installation directory:

      mkdir web-phalcon/app/cache
  chown 777 web-phalcon/app/cache

  • Copy basic site definition (001-netgraphz2.conf) from apache2 subfolder of the installation directory to your apache2 site-available directory (/etc/apache2/sites-available).

  • Edit copied site definition file in the apache2 directory. Change ServerName, DocumentRoot, ServerAdmin, Directory section path and ProxyPassMatch to your actual values.

  • Enable NetGraphz2 site by executing command

      a2ensite 001-netgraphz2

  • Restart PHP-FPM and Apache2

      service apache2 restart
  service php5-fpm restart

MongoDB user settings database

To get NetGraphz2 user accounts working, you should install MongoDB database dump into your MongoDB server. To install working database dump run mongorestore command from mongo directory relative to the installation location. It will import installation database to database name 'netgraphz2'.

cd mongo
mongorestore

Access to the MongoDB is available though the mongo CLI application.

Linking Icinga2

After installation of Icinga2 monitoring system, enable Icinga2 MKLiveStatus extension (https://github.com/Icinga/icinga2/blob/master/doc/15-livestatus.md). Place LiveStatus configuration file into /etc/icinga2/conf.d/livestatus.conf with following contents:

library "livestatus"
object LivestatusListener "livestatus-tcp" {
  socket_type = "tcp"
  bind_host = "127.0.0.1"
  bind_port = "6558"
}

object LivestatusListener "livestatus-unix" {
  socket_type = "unix"
  socket_path = "/var/run/icinga2/cmd/livestatus"
}

The following configuration will assign listening address to 127.0.0.1 and listening port to 6558 (TCP) and unix socket to /var/run/icinga2/cmd/livestatus. It's possible to choose other ports if you configure web-application correctly.

  • Configure Icinga2 scripts (icinga_scripts/config.json). Default port of notifier API is set to be 3434 TCP.

  • Link Icinga2 configuration directories:

     cd /etc/icinga2
 ln -s /opt/netgraphz2/icinga/scripts/ scripts.netgraphz2
 ln -s /opt/netgraphz2/icinga/conf.d/ netgraphz2.d

  • Edit icinga2.conf, add line:

     include_recursive "netgraphz2.d"

  • OPTIONAL: Generate Icinga2 configuration stubs from existing graph database using utility located in icinga/config_generator directory (look in "Icinga2 host configuration stubs generator" chapter)

  • Add your hosts configuration into Icinga2 and restart Icinga2

     service icinga2 restart

Putting all together

By default NetGraphz2 is set up to run on following ports:

  • Notification server - 3433 for socket.io, 3433 for scripts API
  • Backend server - 8088 for RPC, 27015 for collectd listening port

To get information about configuration file sections and location read Configuration page of this wiki.

Web site is configured to access gRPC server on localhost:8088. It should work fine with default configurations.

After configuring NetGraphz2, start services follwing Running up page of this wiki.

Neo4j

Backend server Neo4j configuration is pointed to localhost:7474 with username neo4j and password changed. Notification server pointed to Neo4j database the same way.

To make NetGraphz2 work you should change these values in notification and backend server configurations.

Notifications

Correct URL to the socket.io server should be configurated in client JS configuration to make NetGraphz2 notifications work!

Icinga2 MKLiveStatus

Default configuration for LiveStatus client on backend server - TCP connection to localhost on port 6558. You have to set up Icinga2 MKLiveStatus plugin on this port or change notifications server configuration.

MongoDB

MongoDB configuration is specified for PHP web site. If you have non-default configuration (localhost:27015, no auth) on your MongoDB instance, configure web-phalcon right way following wiki page instructions.