Discord Alerts - CPNV-ES-MAS3-X/Pandora-Containerization GitHub Wiki

Config Discord Alerts


Sources


Web Hook Generated

On any discord text-channel you can go into the settings Edit Channel -> Integrations -> New Webhook

You get a link : [https://discord.com/api/webhooks/1186285641269530624/WD_O2H6qJN0tthpusXhRYpndMxyMOo5rOw8JD9ppV3zd4Lt7bcO9iN3ubkxKPBz8V6pS]

image-20231219140330849


Setup on SRV-PandoraFMS

sudo wget https://pandorafms.com/library/wp-content/uploads/2021/09/pandora_discord_cli.zip

sudo unzip pandora_discord_cli.zip
[Input]
pip3 install -r requirements.txt
[requirements.txt : Content ]

certifi==2020.11.8
chardet==3.0.4
discord-webhook==0.11.0
idna==2.10
requests==2.24.0
urllib3==1.25.11
[Output]
cpnv@pandorafms:/tmp/discord$ sudo pip3 install -r requirements.txt
Collecting certifi==2020.11.8
  Downloading certifi-2020.11.8-py2.py3-none-any.whl (155 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.4/155.4 KB 2.9 MB/s eta 0:00:00
Collecting chardet==3.0.4
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.4/133.4 KB 2.2 MB/s eta 0:00:00
Collecting discord-webhook==0.11.0
  Downloading discord_webhook-0.11.0-py3-none-any.whl (8.4 kB)
Collecting idna==2.10
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 KB 3.4 MB/s eta 0:00:00
Collecting requests==2.24.0
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.8/61.8 KB 1.6 MB/s eta 0:00:00
Collecting urllib3==1.25.11
  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.0/128.0 KB 3.6 MB/s eta 0:00:00
Installing collected packages: chardet, certifi, urllib3, idna, requests, discord-webhook
  Attempting uninstall: chardet
    Found existing installation: chardet 4.0.0
    Not uninstalling chardet at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'chardet'. No files were found to uninstall.
  Attempting uninstall: certifi
    Found existing installation: certifi 2020.6.20
    Not uninstalling certifi at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'certifi'. No files were found to uninstall.
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.26.5
    Not uninstalling urllib3 at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'urllib3'. No files were found to uninstall.
  Attempting uninstall: idna
    Found existing installation: idna 3.3
    Not uninstalling idna at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'idna'. No files were found to uninstall.
  Attempting uninstall: requests
    Found existing installation: requests 2.25.1
    Not uninstalling requests at /usr/lib/python3/dist-packages, outside environment /usr
    Can't uninstall 'requests'. No files were found to uninstall.
Successfully installed certifi-2020.11.8 chardet-3.0.4 discord-webhook-0.11.0 idna-2.10 requests-2.24.0 urllib3-1.25.11
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

Then import pandora_discord_cli.py in /usr/share/pandora_server/util/pandora_discord_cli.py

Add rights

sudo cp pandora_discord_cli.py /usr/share/pandora_server/util/pandora_discord_cli.py

cd /usr/share/pandora_server/util/pandora_discord_cli.py
sudo chmod 777 pandora_discord_cli.py
[pandora_discord_cli.py : Content ]

import requests, argparse, sys, os
from datetime import datetime
from re import search
from base64 import b64decode
from discord_webhook import DiscordWebhook, DiscordEmbed


### Variables and arg parser ###
parser = argparse.ArgumentParser(description='Test parser dic')
parser.add_argument('-d', '--data', help='Data in coma separate keypairs. Ex: test=5,house=2', required=True)
parser.add_argument('-u', '--url', help='Discord webhook URL', required=True)
parser.add_argument('-t', '--alert_tittle', help='Alert tittle', default='PandoraFMS alert fired')
parser.add_argument('-D', '--alert_desc', help='Alert description', default='alert')
parser.add_argument('-m', '--message', help='Discord message', default='')
parser.add_argument('-T','--tittle_color', help='Alert tittle descripcion in HEX EX: 53e514', default="53e514")
parser.add_argument('-A','--author', help='Alert custom author', default='PandoraFMS')
parser.add_argument('-F','--footer', help='Custom footer', default='')
parser.add_argument('--avatar_url', help='Custom avatar URL for the user which send the alert', default='')
parser.add_argument('--author_url', help='Alert custom url author', default='')
parser.add_argument('--author_icon_url', help='Alert custom author icon url ', default='')
parser.add_argument('--thumb', help='Custom thumbnail url', default='')
parser.add_argument('--api_conf', help='Api configuration parameters in coma separate keypairs. EX "user=admin,pass=pandora,api_pass=1234,api_url=http://test.artica.es/pandora_console/include/api.php"')
parser.add_argument('--module_graph', help='Uses pandora API to generate a module graph and attach it to the alert needs module_id and interval parameters in coma separate keypairs. EX "module_id=55,interval=3600"')
parser.add_argument('--tmp_dir', help='Temporary path to store graph images', default='/tmp')


args = parser.parse_args()

### Functions:
def parse_dic(cValues):
    """convert coma separate keypairs into a dic. EX "test=5,house=8,market=2" wil return "{'test': '5', 'casa': '8', 'mercado': '2'}" """
    data={}
    try :
        for kv in cValues.split(","):
            k,v = kv.strip().split("=")
            data[k.strip()]=v.strip()
    except Exception as e :
        print(f"Warning, error parsing keypairs values: {e}")
    return data

def add_embed_itmes(data):
    """iterate dictionary and set webhook fields, one for eacj keypair"""
    for k, v in data.items() :
        embed.add_embed_field(name=k, value=v)

def parse_api_conf(cConf):
    if args.api_conf :
    # Parse Api config
        print ("Api config enable")
        apid = parse_dic(cConf)
        
        if apid.get("user") is None:
            print ("Error no user defined in api_conf keypairs, skipping graph generation.")
            return
        
        if apid.get("pass") is None:
            print ("Error no password defined in api_conf keypairs, skipping graph generation.")
            return

        if apid.get("api_pass") is None:
            print ("Error no Api pass defined in api_conf keypairs, skipping graph generation.")
            return

        if apid.get("api_url") is None:
            apid['api_url'] = "http://127.0.0.1/pandora_console/include/api.php" 
            #print(f"api_url: {apid['api_url']}")

        return apid
    else:
        return None

def parse_graph_conf(cGraph):
    if not args.api_conf:
        print ("To get graph data api conf shoul be provided please set an api config")
        return
    
    if cGraph :
    # Parse Api config
        graphd = parse_dic(cGraph)
        if graphd.get("module_id") is None:
            print ("error no module_id defined in module_graph keypairs, skipping graph generation.")
            return
        
        if graphd.get("interval") is None:
            graphd["interval"] = 3600
        
        return graphd
    else:
        return None

def get_graph_by_moduleid (baseUrl,pUser, pPass, apiPass, moduleId, graphInterval) : 
    sep="url_encode_separator_%7C"
    try:
        url = f"{baseUrl}?op=get&op2=module_graph&id={moduleId}&other={graphInterval}%7C1&other_mode={sep}&apipass={apiPass}&api=1&user={pUser}&pass={pPass}"
        graph = requests.get(url)
        if graph.status_code != 200:
            print (f"Error requested api url, status code: {graph.status_code}. skipping graph generation")
            return None
        if graph.text == "auth error":
            print (f"Error requested Pandora api url, status code: {graph.text}. skipping graph generation")
            return None
        if graph.text == "Id does not exist in database.":
            print (f"Error requested Pandora api url, status code: {graph.text}. skipping graph generation")
            return None
    except:
        print("Error requested api url. skipping graph generation")
        return None

    return graph

## Main
## Basic message
webhook = DiscordWebhook(url=args.url, content=args.message, avatar_url=args.avatar_url)
# create embed object for webhook
embed = DiscordEmbed(title=args.alert_tittle, description=args.alert_desc, color=int(args.tittle_color, 16))
# set author
embed.set_author(name=args.author, url=args.author_url, icon_url=args.author_icon_url)
# set thumbnail
if args.thumb: embed.set_thumbnail(url=args.thumb)
# set footer
if args.footer : embed.set_footer(text=args.footer)
# set timestamp (default is now)
embed.set_timestamp()
# Parse data keys
data = parse_dic(args.data)
# add fields to embed
add_embed_itmes(data)

# Parse api config
api = parse_api_conf(args.api_conf)
# Parse graph config
graph_cfg = parse_graph_conf(args.module_graph)
  
## Generate module graph

if graph_cfg is not None and api is not None:
    graph = get_graph_by_moduleid (api["api_url"],api["user"], api["pass"], api["api_pass"], graph_cfg["module_id"], graph_cfg["interval"])
    
    if graph is not None:
        try:
            namef =  f"graph_{graph_cfg['module_id']}.{datetime.now().strftime('%s')}.png"
            filename = f"{args.tmp_dir}/{namef}"
            with open(filename, "wb") as f:
                f.write(b64decode(graph.text))
                f.close
            print (f"Graph generated on temporary file {filename}")
        except Exception as e :
            print(f"Error, cant generate graph file: {e}")
            filename = None

        try:
            with open(filename, "rb") as F:
                webhook.add_file(file=F.read(), filename=namef)
                f.close
            embed.set_image(url=f'attachment://{namef}')
        except Exception as e :
            print(f"Error, cant add graph file: {e}")
            filename = None

# add embed object to webhook
webhook.add_embed(embed)

# Execute webhook send
response = webhook.execute()

# clean temp file if exist
try:
    os.remove(filename)
except:
    pass

# print response
print (f"Message sent. status code: {response[0].status_code}") if response[0].status_code == 200 else print (f"Error status code: {response[0].status_code}")

We can do a first test to see if the script works using the following command

[Input]
python3 pandora_discord_cli.py -u <webhook-url> -d "Data=5, Agent=Test, Module=Ping"

[Input with WebHook]
python3 pandora_discord_cli.py -u https://discord.com/api/webhooks/1186285641269530624/WD_O2H6qJN0tthpusXhRYpndMxyMOo5rOw8JD9ppV3zd4Lt7bcO9iN3ubkxKPBz8V6pS -d "Data=5, Agent=Test, Module=Ping"
[Output in Shell]
To get graph data api conf shoul be provided please set an api config
Message sent. status code: 200
[Output in Discord]

image-20231218163513352

Another test we can do with more complex alert

[Input]
python3 pandora_discord_cli.py -d "Agent=Server22,Module=test_module,Group=Servers,State=Critical,Data=22,Timestamp=2020-11-04 11:14:00" \
-u https://discord.com/api/webhooks/1186285641269530624/WD_O2H6qJN0tthpusXhRYpndMxyMOo5rOw8JD9ppV3zd4Lt7bcO9iN3ubkxKPBz8V6pS \
--tittle_color ed2512 \
--footer "PandoraFMS Alert" \
-A "Sauron Systems" \
--author_icon_url "https://pandorafms.com/wp-content/uploads/2019/04/software-de-monitorizacion-pandorafms-logo.png" \
-m "We have bad news for you. Something is on CRITICAL status 2" \
--author_url https://pandorafms.com/ \
-D "Module test is going to critical" \
--thumb https://pandorafms.com/images/alerta_roja.png \
--avatar_url https://pandorafms.com/images/alerta_roja.png \
--api_conf "user=admin,pass=pandora,api_pass=pandora,api_url=http://10.0.0.124/pandora_console/include/api.php" \
--module_graph "module_id=6266, interval=3600" \
--tmp_dir /tmp
[Output]

image-20231218165426796

image-20231219141048712

Create Alert

Now for the PandoraFMS config we go to Alerts -> Commands -> Create.

image-20231218164826928

image-20231219141551885

Name : Discord Alert

Command : python3 /usr/share/pandora_server/util/pandora_discord_cli.py -d "_field1_" -u "_field2_" -D "_field3_" --tittle_color _field4_ --thumb _field5_ -A "Pandora FMS Alert system" --footer "PandoraFMS" --author_icon_url "https://pandorafms.com/wp-content/uploads/2019/04/software-de-monitorizacion-pandorafms-logo.png" --author_url https://pandorafms.com/ --tmp_dir /tmp

!!! Attention au chemin du script !!!

image-20231219141608061

1 field description : Data
1 field values : AgentName=_agentalias_,ModuleName=_module_,AgentGroup=_agentgroup_,Status"_modulestatus_,Dat a=_data_,Timestamp=_timestamp_

2 field description : Webhook
2 field values : https://discord.com/api/webhooks/1186285641269530624/WD_O2H6qJN0tthpusXhRYpndMxyMOo5rOw8JD9ppV3zd4Lt7bcO9iN3ubkxKPBz8V6pS

3 field description : Description
3 field values : Agent_agentalias_is in status _modulestatus_

4 field description : Color
4 field values : ed2512

5 field description : IMG Alert
5 field values : https://pandorafms.com/images/alerta_roja.png

6 field description : N/A (API not implemented)
6 field values : N/A

Create Action

Next we go to Alerts -> Action -> Create.

image-20231219142102483

Name : Discord Trigger
Command : Discord Alert 
Group : All
Threshold : 0 secondes

Under Recovery change the field 4 Color for : 00b343

Modify Critical Condition Template

Next we go to Alerts -> Templates.

Here we are going to change the settings of the existing template "Critical Condition" so that when it triggers, it use the Discord Trigger Action.

image-20231219142253022

Step 2 
Time Threshoold : 1min
Default Action : Discord Trigger

Create Alert

Add alert to agent as follows

image-20240109092858780

Tests

To stress the CPU, we're using the following, install on the client

sudo apt update
sudo apt install python3-pip

sudo pip install s-tui
sudo apt install stress

Start the UI

[Input]
s-tui
[Ouput]

image-20231219143151253

Click or select ( )Stress when you want to start the test.

POC - Tests

[Test results]
- Stress started at 15:41 - 
Without force checks, meaning we don't use it and let the server contact the client, interval of 5 min ~

- Stress stoped at 15:44 -
After the alert got triggered, stress stoped, waiting the recovery alert

- Recovery alert triggered at 15:48 -
Recovery alert triggered after contact confirmed status was back to normal

Configuration approved by the names and colors we see on alerts.

image-20231219143356471