SecBuzzerESM Installation Manual - secbuzzer/DetectionEdge GitHub Wiki

SecBuzzerESM Installation Manual

1. Hardware requirements

Depends on different purposes, if only test basic functions, kindly use and install personal version; if test in a larger scale field, hardware criteria or network traffic would change accordingly, basic requirements kindly suggest below: Personal version/ test：

  • OS：Ubuntu 18.04.4 UP Server English version
  • CPU：2.0 GHz 64-bit processor, dual core (or two virtual CPUs)
  • MEM：10G UP
  • HDD：80G UP

Field version/ test：

  • OS：Ubuntu 18.04.4 UP Server English version
  • CPU：2.2 GHz 64-bit processor, 8 core
  • MEM：32G UP
  • HDD：2T ~ 4T

Items need to confirm：

  • Make sure if OS support SSH, if no, kindly follow the instruction bellow,
sudo apt-get install openssh-server
  • Confirm time/ time zone if correct( (kindly set up in GMT+8).
  • Confirm OS system is 'clear' without intalling any other program, ex.：Docker, Elasticsearch, Fluentd, Grafana etc. ESM would intall those programs automatically.

2. Instructions for System Installation (Offline)

Kindly follow bellow instructions for offline installation:(no require any internet connection)

2.1 Preparation

Kindly download offline installation package and use a way that you used to save and transfer files like Linux SCP、FTP to Ubuntu OS.

In the case of command scp:(refer the diagram bellow and adjust accordingly)

images/install_10_off.png

Login acount before entering Ubuntu, because of root in the process of installation.

sudo su

images/install_01.png

2.2 Unzip the offline installation package

Unzip the package and follow instructions below

tar zxvf SecBuzzerESM_XXX.tgz (XXX stands for version number)

images/install_11_off.png

2.3 Move under the opt directory

Move the unziped package "SecBuzzerESM" under opt .

mv SecBuzzerESM /opt/

images/install_12_off.png

2.4 Check the name of the NIC(network interface)

Kindly execute the following command to check the name of the NIC, and memorize it, which needs to be set in the configuration file in advance.

ifconfig

images/install_04.png

2.5 Set up SecBuzzerESM.env

The section explains system setups, first of all, enter SecBuzzerESM.env configuration file and set up. As seem a case of nano editor below(can use the editor by preference),

cd /opt/SecBuzzerESM
nano SecBuzzerESM.env

SecBuzzerESM.env included setting parameters are as follows, and their setting methods and meanings will be explained later.

# === Elasticsearch === 
ES_VOLUME_PATH=/opt/Logs/ES/volume
ES_MEM=1024m
# === Fluentd ===
FLUENTD_LOG_PATH=/opt/Logs/Fluentd/Logs
FLUENTD_BUFFER_PATH=/opt/Logs/Buffers
# === Suricata ===
SURICATA_LOG_PATH=/opt/Logs/Suricata
IF_NAME=
HOME_NET=
# === Grafana ===
GF_SMTP=false
GF_SMTP_HOST=smtp.gmail.com:465
GF_SMTP_FROM_ADDRESS=
GF_SMTP_USER=
GF_SMTP_PASSWORD=
# === WEB ===
WEB_IF_NAME=
API_KEY_VALUE=
ORG_3_CODE=
# === Flow Scan & NMAP Scan ===
FLOWSCAN_SOURCEIP=
FLOWSCAN_DESTIP=
FLOWSCAN_TOTALCOUNT=1000
NMAPSCAN_IP=
# === DEV ===
DEV_MODE=True (the parameter only appears in beta version, kindly remove it if it shows in official version.

After edit completely, enter Ctrl+X and save(in a case of nano).

2.5.1 Elasticsearch

  • ES_VOLUME_PATH

The location of Elasticsearch Volume file is used for a data storage, recommend to remain the default.

  • ES_MEM

Setting the limitation memory of Elasticsearch, default value is 1024m (personal version).The basic configuration principle is half of the total memory, but not more than 32G, ex. If the total system memory is 32G, the memory can be set to 16g, if the total memory is 64G, it can be set to 31g (not more than 32G).

Note: The memory unit is recommended to use lowercase, for example：m、g

2.5.2 Fluentd

  • FLUENTD_LOG_PATH

Fluentd is a location which can collect data, recoomeded to remain default value.

  • FLUENTD_BUFFER_PATH

Fluentd Buffer is a storage location, recoomeded to remain default value.

2.5.3 Suricata

  • SURICATA_LOG_PATH

Suricata When an alarm is detected, the alarm data will be output to eve.json,，This parameter is as a storage path of eve.json ，recoomeded to remain default value.

  • IF_NAME

Suricata, a monitered network interface card, which is monitoring network traffic to detect abnormal behaviors. Depends on different circumstances, the NIC will be different. the field must be adjusted.

Note: Please refer the description in section 2.4 for the method to obtain the interface of the network card.

  • HOME_NET

Set the monitored IPs or network segments. Each IP and network segment can be separated by a comma. The supported formats and examples as follows：

HOME_NET=10.0.0.1,192.168.0.1/24,172.16.1.1/24

Note: No spaces, double quotation marks or single quotation marks are required between each IP or network segment.

2.5.4 Grafana (Stopped)

Grafana visual dashboard has been built-in by ESM, the following setting parameters are used to set up an email box to send those notifications.

Note: This function has not been activated yet, the setting can be skipped first.

  • GF_SMTP

SMTP if function is on, kindly fill in true

  • GF_SMTP_HOST

SMTP Server set up, default is Google

  • GF_SMTP_FROM_ADDRESS

Address used when sending e-mails

  • GF_SMTP_USER

SMTP account

  • GF_SMTP_PASSWORD

SMTP password

2.5.5 WEB

  • WEB_IF_NAME

After rule triggered, ESM would proceed the notification to Cloud so it's transfered by a NIC as the same as Suricata and IF_NAME in personal version/test but in field test, usually set two different ones, IF_NAME for inputting Port Mirror, and WEB_IF_NAME for outputting data transmission.Required field .

  • API_KEY_VALUE

Receive API KEP from ESM, kindly follow below, Required field.

  • ORG_3_CODE

Receive "unit code" from ESM, kindly follow below, Required field.

Enter ESM and find obtain API Key to get API KEY on the menu in upper right corner.

images/install_05.png

2.5.6 Flow Scan & NMAP Scan (Stopped)

  • FLOWSCAN_SOURCEIP

By Network flow scanning, if it wants receive IP data, such as 70 network segments from IP: 192.168.70.24, it would save in 192.168.70.0/24.

Set a default code 192.168.70.0/24, if IPs and those network segments can not match each, it would not be able to receive data from those IPs.

  • FLOWSCAN_DESTIP

By Network flow scanning, if it wants receive IP data, such as 70 network segments from IP: 192.168.70.24, it would save in 192.168.70.0/24.

Set a default code 192.168.70.0/24, if IPs and those network segments can not match each, it would not be able to receive data from those IPs.

  • FLOWSCAN_TOTALCOUNT

By Network flow scanning, default total number is 1,000.

  • NMAPSCAN_IP

By NMAP scanning, if it wants to receive IP data, such as 70 network segments totally from IP: 192.168.70.24 would save in 192.168.70.0/24.

Set a default code 192.168.70.2, if IP and those network segments can not match each, it would not be able to receive data from those IPs.

2.5.7 DEV

  • DEV_MODE=True

The parameter which is used to define beta version from official version, only appears in beta version, kindly remove it in offical one once users start to connect to the network.

2.6 Set up SecBuzzerESM

2.6.1 install SecBuzzerESM Detection edge

After the configuration file set, the operation can be officially installed, kindly execute the following commands：

cd /opt/SecBuzzerESM
source Offline_Install.sh

Note：add a time command to calculate time automatically：time source Offline_Install.sh

images/install_13_off.png

2.6.2 Install MonitorAgent (Options)

/bin/bash MonitorAgent.sh

image/install_15.png

kindly send us back the values of the red chart

Hostname=
TLSPSKIdentity=
TLSPSKKey=

2.7 Active SecBuzzerESM

After installation, kindly start the system.

cd /opt/SecBuzzerESM
./compose.sh up # start ESM

If encounter temporary unsolvable problems during the execution of ESM, users can take the "Close ESM" or "Restart ESM" command to fix it.

./compose.sh down # off ESM (with restart command )
./compose.sh down && ./compose.sh up # restart ESM

images/install_07.png

2.7.1 Verify ESM if it's normal

docker ps -a

If start abnormal, it appears restartor exit in a field of STATUS. In a normal situation, it shows how much time to start in the Container.

images/install_08.png

2.7.2 Verrify Elasticsearch if it's normal

Users can find a way to connect to Elasticsearch(ES) and make sure if ES operate normally.

  • Connect to ES Head(check status tools of ES )：http://your_ip:19100
  • Then conect from ES Head to ES：http://your_ip:19200

As the chart shows below：

images/install_09.png

2.8 Troubleshooting

2.8.1 Check the version of system

Users can find cat or more command to view ESM version in HISTORY.md.

cd /opt/SecBuzzerESM
cat HISTORY.md # or more HISTORY.md

images/install_14.png

2.8.2 Set up firewall

If it's requested to upload those notifications to Cloud or update new rules of operation and detection in ESM monitoring environment, kindly allows those websites can be accessed ：(HTTPS，Port is 443)

  1. api.esm.secbuzzer.co port 80 and 443
  2. api.hub.secbuzzer.co port 80 and 443
  3. api.esm.secbuzzer.ai poer 80 and 443
  4. host-monitor.secbuzzer.co port 30051
  5. host-monitor.secbuzzer.ai port 30051

2.8.3 Refreshing pages too much times

If it's happened as「refreshing pages too many times」, it may be a Proxy in the field. Kindly prepare an external IP then contact ESM engineers to set up a whitelist.