Ingesting log Data - JCSC-JE/Graylog-IR GitHub Wiki
There are a number of methods to ingest data to our Graylog analysis system. The system uses the follow ports and protocols to ingest data via Graylog inputs.
| Port | Protocol | Log Type | Graylog Input Type |
|---|---|---|---|
| 5044 | TCP | Windows EVTX | Beats |
| 12201 | UDP | CSV files | Raw/Plaintext UDP |
| 12201 | UDP | JSON files | Raw/Plaintext UDP |
| 12201 | UDP | IIS files | GELF UDP |
The method of ingestion depends on the format of log source you are ingesting, each type is detailed below
Windows EVTX logs
To ingest EVTX windows logs, there is a PowerShell script to ingest logs. Before executing the script make sure you change the IP address of your Graylog analysis system in file in the directory ingestion/evtx/winlogbeat-7.10.2/winlogbeat-evtx.yml
hosts: ["192.168.183.138:5044"]
Once changed, open up an new PowerShell terminal and execute the PowerShell script with an execution bypass in place
powershell.exe -exec bypass
.\evtx-upload.ps1
The script supports having multiple EVTX files in a single directory or a directory with sub directories for each system to be analysed. If you have both workstation and server files, use two separate high levels directories.
Once execute the script will ask for some information such as Client, Case Number,Log Type and the path to the logs. The Log Type must be either "Server" or "Workstation". There are a following caveats when using the script.
- If you don't not specify either "Workstation " or "Server", the data will not be routed correctly within the Analysis system.
- The path to the logs must not contain any spaces or ingestion will fail.
CSV Files
For csv ingestion, you can use either Filebeat or you can use any programming language to read the csv files and then send to the analysis system. If you use the python code example in the ingestion > csv directory then you will have to change the input type to GELF UDP.
JSON Files
You can use Filebeat to send JSON files into the analysis system, simply modify the file path and destination IP address in the filebeat.yml file in the Filebeat directory in this repository.
JSON files sent into the system use an Graylog extractor to extract the fields. If uses the standard of a colon for separating key value pair and a comma between fields.
Screenshot
Below is an example of EVTX logs having been ingested.
