Developer Guides ‐ Streaming Logs - MarechJ/hll_rcon_tool GitHub Wiki
🧭 You are here : Wiki home / Developer Guides / Streaming Logs
This is only available in version v9.7.0 or higher.
Overview
The log_stream is a Redis stream that stores logs from the game server in sequential order on a transient basis (they are not persisted to the database and are cleared on service startup) to support pushing new logs to external tools through a websocket endpoint.
Configuration
{
"enabled": false,
"stream_size": 1000,
"startup_since_mins": 2,
"refresh_frequency_sec": 1,
"refresh_since_mins": 2
}
stream_size: The number of logs the stream will retain before discarding the oldest logs.
startup_since_mins: The number of minutes of logs to request from the game service when the service starts up
refresh_frequency_sec: The poll rate for asking for new logs from the game server
refresh_since_mins The number of minutes of logs to request from the game service each loop
Permissions
For new installs these are added to the owner and admin group.
Existing installs can add these permissions to users or groups:
api.can_view_log_stream_configapi.can_change_log_stream_config
Service
The log_stream service will start by default but will only actually do anything if enabled is set to true in its config.
Each start up the redis stream is purged and logs (if any are available) are pulled from the game server.
Stream IDs
In a Redis stream the IDs must be unique and sequentially increasing.
We use the timestamp of a log entry and a 0 indexed incrementing suffix because more than one log can occur at the same timestamp, e.g. 1711657986-0 and 1711657986-1.
Connecting / Authentication
To connect you must make a web socket connection to your CRCON at your normal RCONWEB_PORT or RCONWEB_PORT_HTTPS with the /ws/logs endpoint.
For example: ws://localhost:8010/ws/logs
You must include a valid API key for an account with the appropriate permissions, there is no username/password based authentication.
Request Formats
After successfully connecting you must send a JSON message with your criteria before you will receive any logs.
You can specify a past stream ID (last_seen_id) and/or filter by action type (actions) (any valid type as defined in rcon.types.AllLogTypes).
If you don't care about past logs, simply pass null for last_seen_id, or if you don't care about either you can pass {}.
No filter and only new logs:
{}
Older logs:
{ "last_seen_id": null }
Filtered logs:
{ "actions": ["KILL", "TEAM KILL"] }
Response Formats
Data is return as JSON and is in the following formats:
Service disabled:
{
"error": "Log stream is not enabled in your config",
"last_seen_id": null,
"logs": []
}
If more than 25 logs would be returned, they're batched 25 logs per response and multiple responses are sent to avoid any potential issues with payload size.
Logs:
{
"last_seen_id": "1711657986-1",
"logs": [
{
"id": "1711657986-0",
"log": {
"version": 1,
"timestamp_ms": 1711657986000,
"relative_time_ms": -188.916,
"raw": "[449 ms (1711657986)] KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
"line_without_time": "KILL: maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
"action": "KILL",
"player": "maxintexas13",
"steam_id_64_1": "76561199195150101",
"player2": "Twisted",
"steam_id_64_2": "76561199096984835",
"weapon": "KARABINER 98K",
"message": "maxintexas13(Axis/76561199195150101) -> Twisted(Allies/76561199096984835) with KARABINER 98K",
"sub_content": null
}
},
{
"id": "1711657986-1",
"log": {
"version": 1,
"timestamp_ms": 1711657986000,
"relative_time_ms": -188.916,
"raw": "[340 ms (1711657986)] TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
"line_without_time": "TEAM KILL: Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
"action": "TEAM KILL",
"player": "Untreated HSV",
"steam_id_64_1": "76561198039482575",
"player2": "zeto61",
"steam_id_64_2": "76561198178304393",
"weapon": "MK2 GRENADE",
"message": "Untreated HSV(Allies/76561198039482575) -> zeto61(Allies/76561198178304393) with MK2 GRENADE",
"sub_content": null
}
}
],
"error": null
}
Sample Python Code
Courtesy of @excuseme on the Discord, this is an example of using the service
import asyncio
import json
from datetime import datetime
import logging
import websockets
from websockets.exceptions import ConnectionClosed
ALL_ACTIONS_TO_MONITOR = ['KILL', 'TEAM KILL', 'MATCH ENDED']
class CustomDecoder(json.JSONDecoder):
def __init__(self, *args, **kwargs):
super().__init__(object_hook=self.try_datetime, *args, **kwargs)
@staticmethod
def try_datetime(d):
ret = {}
for key, value in d.items():
try:
ret[key] = datetime.fromisoformat(value)
except (ValueError, TypeError):
ret[key] = value
return ret
class CRCONWebSocketClient:
def __init__(self, server, ):
self.server = server
self.stop_event = None
async def start_socket(self, stop_event):
self.stop_event = stop_event
headers = {"Authorization": f"Bearer {self.server.rcon_api_key}"}
if self.server.rcon_login_headers:
headers.update(self.server.rcon_login_headers)
websocket_url = self.server.rcon_web_socket + "/ws/logs"
while not self.stop_event.is_set():
try:
async with websockets.connect(websocket_url, extra_headers=headers,
max_size=1_000_000_000) as websocket:
logging.info(f"Connecting to {websocket_url}")
try:
await websocket.send(json.dumps({"last_seen_id": None, "actions": ALL_ACTIONS_TO_MONITOR}))
except ConnectionClosed:
logging.warning(
f"ConnectionClosed exception",
exc_info=cc)
break
logging.info(f"Connected to CRCON websocket {websocket_url}")
while not self.stop_event.is_set():
try:
message = await asyncio.wait_for(websocket.recv(), timeout=1.0)
await self.handle_incoming_message(websocket, message)
except asyncio.TimeoutError:
logging.debug("timeout error")
except asyncio.exceptions.CancelledError:
logging.debug("cancelled error")
except ConnectionClosed as cc:
logging.warning(
f"ConnectionClosed exception",
exc_info=cc)
break
except Exception as e:
logging.error(f"Error during handling of message by {message}", exc_info=e)
except Exception as e:
logging.error(f"Error connecting to {websocket_url}", exc_info=e)
await asyncio.sleep(5)
async def handle_incoming_message(self, websocket, message):
json_object = json.loads(message, cls=CustomDecoder)
if json_object:
logs_bundle = json_object.get('logs', [])
# do stuff with the logs