Zeek Random Notes - C0ntr07/Firewalla-Information GitHub Wiki
cd /usr/local/zeek/bin
Some of the folders are symbolic links.
pi@firewalla:/usr/local/zeek/bin$ ls -l
total 251756
-rwxr-xr-x 1 root root 454008 Mar 12 05:10 bifcl
-rwxr-xr-x 1 root root 5557496 Mar 12 05:10 binpac
lrwxrwxrwx 1 root root 32 Mar 12 06:10 bro -> /usr/local/zeek/bin/zeek-wrapper
lrwxrwxrwx 1 root root 32 Mar 12 06:10 bro-config -> /usr/local/zeek/bin/zeek-wrapper
lrwxrwxrwx 1 root root 32 Mar 12 06:10 broctl -> /usr/local/zeek/bin/zeek-wrapper
lrwxrwxrwx 1 root root 32 Mar 12 06:10 bro-cut -> /usr/local/zeek/bin/zeek-wrapper
-rwxr-xr-x 1 root root 64632 Mar 12 06:01 capstats
-rwxr-xr-x 1 root root 1109168 Mar 12 05:11 paraglob-test
-rwxr-xr-x 1 root root 39050 Mar 12 05:10 trace-summary
-rwxr-xr-x 1 root root 250486648 Mar 12 06:01 zeek
-rwxr-xr-x 1 root root 2005 Mar 12 05:10 zeek-config
-rwxr-xr-x 1 root root 28532 Mar 12 05:10 zeekctl
-rwxr-xr-x 1 root root 38712 Mar 12 06:01 zeek-cut
-rwxr-xr-x 1 root root 826 Mar 12 05:10 zeek-wrapper
sudo ./zeekctl # Start Zeek and bring up a command prompt
pi@firewalla:/usr/local/zeek/bin$ sudo ./zeekctl
Welcome to ZeekControl 2.0.0
Type "help" for help.
[ZeekControl] >
Show all available commands.
[ZeekControl] > help
ZeekControl Version 2.0.0
capstats [<nodes>] [<secs>] - Report interface statistics with capstats
check [<nodes>] - Check configuration before installing it
cleanup [--all] [<nodes>] - Delete working dirs (flush state) on nodes
config - Print zeekctl configuration
cron [--no-watch] - Perform jobs intended to run from cron
cron enable|disable|? - Enable/disable "cron" jobs
deploy - Check, install, and restart
df [<nodes>] - Print nodes' current disk usage
diag [<nodes>] - Output diagnostics for nodes
exec <shell cmd> - Execute shell command on all hosts
exit - Exit shell
install - Update zeekctl installation/configuration
netstats [<nodes>] - Print nodes' current packet counters
nodes - Print node configuration
peerstatus [<nodes>] - Print status of nodes' remote connections
print <id> [<nodes>] - Print values of script variable at nodes
process <trace> [<op>] [-- <sc>] - Run Zeek with options and scripts on trace
quit - Exit shell
restart [--clean] [<nodes>] - Stop and then restart processing
scripts [-c] [<nodes>] - List the Zeek scripts the nodes will load
start [<nodes>] - Start processing
status [<nodes>] - Summarize node status
stop [<nodes>] - Stop processing
top [<nodes>] - Show Zeek processes ala top
Commands provided by plugins:
ps.bro [<nodes>] - Show Zeek processes on nodes' systems (deprecated)
ps.zeek [<nodes>] - Show Zeek processes on nodes' systems
What zeek processes are running.
[ZeekControl] > status
Getting process status ...
Getting peer status ...
Name Type Host Status Pid Peers Started
logger logger localhost running 12695 5 29 Aug 14:35:44
manager manager localhost running 12910 5 29 Aug 14:35:47
proxy-1 proxy localhost running 13158 5 29 Aug 14:35:50
worker-1 worker localhost running 13498 4 29 Aug 14:35:54
worker-2 worker localhost running 13500 4 29 Aug 14:35:54
[ZeekControl] >
Logs are maintained in the /bspool/logger directory.
Zeek’s generated log files can be summarized as follows:
- conn.log: A file containing information pertaining to all TCP/UDP/ICMP connections, this file contains most of the information gathered from the packet capture.
- files.log: A file consisting of analytic results of packets’ counts and sessions’ durations.
- packet_filter.log: A file listing the active filters applied to Zeek upon reading the packet capture file.
- x509.log: A file containing public key certificates used by protocols.
- weird.log: A file containing packet data non-conformant with standard protocols. It also contains packets with possibly corrupted or damaged packet header fields.
- (protocol).log (dns.log, dhcp.log, http.log, snmp.log): These are files containing information for packets found in each respective protocol. For instance, dns.log will only contain information generated by Domain Name Service (DNS) packets.
cd /bspool/logger
pi@firewalla:/bspool/logger$ ls -la
total 332
drwxr-xr-x 2 root root 780 Aug 29 17:36 .
drwxrwxrwt 9 root root 220 Aug 29 17:35 ..
-rw-r--r-- 1 root root 112 Aug 29 14:35 .cmdline
-rw-r--r-- 1 root root 183537 Aug 29 17:36 conn.log
-rw-r--r-- 1 root root 8273 Aug 29 17:36 dns.log
-rw-r--r-- 1 root root 335 Aug 29 14:35 .env_vars
-rw-r--r-- 1 root root 4518 Aug 29 17:36 files.log
-rw-r--r-- 1 root root 3634 Aug 29 17:36 http.log
-rw-r--r-- 1 root root 6 Aug 29 14:35 .pid
-rw-r--r-- 1 root root 18 Aug 29 17:24 .rotated.broker
-rw-r--r-- 1 root root 18 Aug 29 14:36 .rotated.cluster
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.conn
-rw-r--r-- 1 root root 18 Aug 29 17:15 .rotated.dhcp
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.dns
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.files
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.http
-rw-r--r-- 1 root root 18 Aug 29 16:45 .rotated.known_certs
-rw-r--r-- 1 root root 18 Aug 29 16:45 .rotated.known_hosts
-rw-r--r-- 1 root root 18 Aug 29 16:48 .rotated.known_services
-rw-r--r-- 1 root root 18 Aug 29 14:36 .rotated.loaded_scripts
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.notice
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.ntp
-rw-r--r-- 1 root root 18 Aug 29 14:36 .rotated.packet_filter
-rw-r--r-- 1 root root 18 Aug 29 15:33 .rotated.pe
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.sip
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.snmp
-rw-r--r-- 1 root root 18 Aug 29 16:54 .rotated.software
-rw-r--r-- 1 root root 18 Aug 29 17:12 .rotated.ssh
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.ssl
-rw-r--r-- 1 root root 18 Aug 29 16:30 .rotated.traceroute
-rw-r--r-- 1 root root 18 Aug 29 14:39 .rotated.tunnel
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.weird
-rw-r--r-- 1 root root 18 Aug 29 17:36 .rotated.x509
-rw-r--r-- 1 root root 991 Aug 29 17:36 sip.log
-rw-r--r-- 1 root root 561 Aug 29 17:36 ssl.log
-rw-r--r-- 1 root root 58 Aug 29 14:35 .startup
-rwx------ 1 root root 18 Aug 29 14:35 .status
-rw-r--r-- 1 root root 0 Aug 29 14:35 stderr.log
-rw-r--r-- 1 root root 182 Aug 29 14:35 stdout.log
Log File Description Field Descriptions
conn.log TCP/UDP/ICMP connections Conn::Info
dce_rpc.log Distributed Computing Environment/RPC DCE_RPC::Info
dhcp.log DHCP leases DHCP::Info
dnp3.log DNP3 requests and replies DNP3::Info
dns.log DNS activity DNS::Info
ftp.log FTP activity FTP::Info
http.log HTTP requests and replies HTTP::Info
irc.log IRC commands and responses IRC::Info
kerberos.log Kerberos KRB::Info
modbus.log Modbus commands and responses Modbus::Info
modbus_register_change.log Tracks changes to Modbus holding registers Modbus::MemmapInfo
mysql.log MySQL MySQL::Info
ntlm.log NT LAN Manager (NTLM) NTLM::Info
ntp.log Network Time Protocol NTP::Info
radius.log RADIUS authentication attempts RADIUS::Info
rdp.log RDP RDP::Info
rfb.log Remote Framebuffer (RFB) RFB::Info
sip.log SIP SIP::Info
smb_cmd.log SMB commands SMB::CmdInfo
smb_files.log SMB files SMB::FileInfo
smb_mapping.log SMB trees SMB::TreeInfo
smtp.log SMTP transactions SMTP::Info
snmp.log SNMP messages SNMP::Info
socks.log SOCKS proxy requests SOCKS::Info
ssh.log SSH connections SSH::Info
ssl.log SSL/TLS handshake info SSL::Info
syslog.log Syslog messages Syslog::Info
tunnel.log Tunneling protocol events Tunnel::Info
zeek-cut wont work because the data are saved in JSON. jq should be used instead.
zeek-cut is a flexible tool that can be called to format Zeek log files depending on needs. The zeek-cut utility can be utilized with more advanced commands to further increase customization.
Use head to see the fields in the log file.
pi@firewalla:/usr/local/zeek/bin$ head -n 1 /bspool/logger/conn.log
{"ts":1598742712.217716,"uid":"CWb5H11Ve47ebi0QK4","id.orig_h":"192.168.1.199","id.orig_p":57788,"id.resp_h":"8.8.8.8","id.resp_p":53,"proto":"udp","service":"dns","duration":0.1626579761505127,"orig_bytes":36,"resp_bytes":153,"conn_state":"SF","local_orig":true,"local_resp":false,"missed_bytes":0,"history":"Dd","orig_pkts":1,"orig_ip_bytes":64,"resp_pkts":1,"resp_ip_bytes":181,"orig_l2_addr":"80:b0:3d:53:b4:bf","resp_l2_addr":"20:6d:31:01:1a:04"}
pi@firewalla:/usr/local/zeek/bin$
Generally, the zeek-cut utility is typically coupled with cat using the pipe | command.
cat /bspool/logger/conn.log | ./zeek-cut ts id.orig_h proto
cat /bspool/logger/conn.log | head -n 10 | jq '.duration'
The -j argument to jq causes the output to be joined together without adding a newline. This example also adds a delimiter of ", " and a newline at the end of the query.
cat /bspool/logger/conn.log | head -n 10 | jq -j '.duration, ", ", .proto, "\n"'
In order to reference JSON object fields that include a ".", in order the familiar leading-dot syntax, use square brackets and quotation marks. (For example, accessing id.orig_h shown above is denoted as .["id.orig_h"].
cat /bspool/logger/conn.log | head -n 10 | jq -j '.duration, ", ", .proto, ", ", .["id.orig_h"], ":", .["id.orig_p"], ", ", .["id.resp_h"], ":", .["id.resp_p"], "\n"'
The JQ select function performs a Boolean operation on an identified field, returning the record if the operation returns true. For example, this selects all of the records where the number of response bytes (resp_bytes) is greater or less then the specified value.
cat /bspool/logger/conn.log | jq 'select(.resp_bytes > 300000)'
The Boolean expression accepts and and or modifiers to add additional query elements. This example adds to the prior command and limits the results to TCP streams.
cat /bspool/logger/conn.log | jq 'select(.resp_bytes > 100000 and .proto == "tcp")'