Multi Node Setup Guide - DNS-OARC/dsp GitHub Wiki

This setup guide will instruct you on how to setup the DNS Statistics Collector (DSC) on multiple collector nodes and the DNS Statistics Presenter (DSP) on a central server to generate graphs for all the nodes. The instructions works on either a Debian or Ubuntu distribution.

If you are looking for a single node setup please read All in One Setup Guide.

Collector Node

The collector node runs DSC to collect DNS statistics that is written down into XML data files. These files are then sent to the central server for graphing.

Install Software

Install the required DSC software, see Packages for information about which distribution releases are supported.

Debian

DISTRIBUTION=`lsb_release -sc` # or set to your distribution of Debian
wget -O - https://pkg.dns-oarc.net/dns-oarc.distribution.key.gpg | sudo apt-key add -
echo "deb http://pkg.dns-oarc.net/debian $DISTRIBUTION main" | sudo tee /etc/apt/sources.list.d/dns-oarc.list
sudo apt-get update
sudo apt-get install dsc

Ubuntu

sudo add-apt-repository ppa:dns-oarc/dsc
sudo apt-get update
sudo apt-get install dsc

Setup Collector

cd /etc/dsc
sudo cp dsc.conf.sample dsc.conf
sudo vi dsc.conf

When editing dsc.conf do:

  • Uncomment any of the bpf_program to limit the packet capturing (optional)
  • Uncomment one interface setting, change the interface name if it is not correct

Create the data files directory:

sudo mkdir -p /var/lib/dsc

Start DSC

DSC is currently not installed as a service so we need to start it.

sudo dsc /etc/dsc/dsc.conf

DSC writes it's data files every minute (default settings) but writing the first data file after start up can take up to 2 minutes, wait and check that you get data files. Consult log files if there are none after a while.

ls /var/lib/dsc/

Central Server

The Central Server runs two things, the Extractor to generate DAT files from the XML files and the Presenter, a CGI application, to generate and display graphs.

Install Software

Install the required DSC software, see Packages for information about which distribution releases are supported.

Debian

DISTRIBUTION=`lsb_release -sc` # or set to your distribution of Debian
wget -O - https://pkg.dns-oarc.net/dns-oarc.distribution.key.gpg | sudo apt-key add -
echo "deb http://pkg.dns-oarc.net/debian $DISTRIBUTION main" | sudo tee /etc/apt/sources.list.d/dns-oarc.list
sudo apt-get update
sudo apt-get install dsp

Ubuntu

sudo add-apt-repository ppa:dns-oarc/dsc
sudo apt-get update
sudo apt-get install dsp

Setup Extractor

The Extractor is the part of DSP that prepares the DSC data into DAT files for the Grapher to read, schedule refile-and-grok.sh to run each minute.

sudo crontab -e

Example crontab:

* * * * * nice /usr/lib/dsp/refile-and-grok.sh

Setup Presenter

We create a minimal configuration for the Presenter.

cd /etc/dsp
sudo vi dsc-grapher.cfg

Add the following to dsc-grapher.cfg:

trace_windows 1hour 4hour 1day 1week
accum_windows 1day 2days 3days 1week

Setup Nginx and Fastcgi

DSP is a Perl script along with a Perl library that needs to run as a CGI application, for this guide we use Nginx and Fastcgi but other solutions are also possible.

sudo apt-get install nginx fcgiwrap
sudo vi /etc/nginx/sites-available/default

When editing default add the following to the server section:

    location = /dsc-grapher.pl {
        gzip            off;
        root            /usr/lib/cgi-bin/dsp;
        include         fastcgi_params;
        fastcgi_param   DOCUMENT_ROOT $document_root;
        fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param   SCRIPT_NAME $fastcgi_script_name;
        fastcgi_param   AUTH_USER $remote_user;
        fastcgi_param   REMOTE_USER $remote_user;
        fastcgi_pass    unix:/var/run/fcgiwrap.socket;
    }

    location ~ ^/dsc/(.+\.png)$ {
        alias   /usr/share/dsp/html/$1;
    }

Make sure both services are started and configuration refreshed:

sudo service fcgiwrap restart
sudo service nginx restart

Setup Presenter Cache

The Presenter puts it's generated graphs in a cache and we need to create the directory and set the permissions to be the same as Nginx uses, this can be different depending on what web server and CGI you use.

sudo mkdir /var/cache/dsp
sudo chown www-data:www-data /var/cache/dsp/

Final Touches

For the final step we need to configure DSP for each of the nodes we have and setup a transfer of files from the nodes to the central server.

The Presenter is structured to have multiple nodes under multiple servers but the $SERVER can represent other things like companies, customers or locations.

For each node a $SERVER and $NODE need to be defined and will be used in the following steps, the $SERVER can be the same for all nodes but $NODE needs to be unique.

Central Server

Configure DSP

echo "server $SERVER $NODE" | sudo tee -a /etc/dsp/dsc-grapher.cfg

Create Data Files Directory

sudo mkdir -p /var/lib/dsc/$SERVER/$NODE

Node

Transfer Files

This step is really up to you and your infrastructure, there are many ways to transfer files from one server to another or share a file area. What ever way you choice see that it is automated and monitored.

This example uses scp and expects the necessary authentication and access to be already set up.

scp /var/lib/dsc/*.xml central-server:/var/lib/dsc/$SERVER/$NODE/

All Done!

You should now be able to access DSP at http://central-server/dsc-grapher.pl .