20250320 ‐ CyberSkyline Log Analysis (2) - rundas-r00t/blog GitHub Wiki
we are given an nginx log file access.log and are tasked to answer these questions:
- How many different IP addresses reached the server?
- How many requests yielded a 200 status?
- How many requests yielded a 400 status?
- What IP address rang at the doorbell?
- What version of the Googlebot visited the website?
- Which IP address attempted to exploit the shellshock vulnerability?
- What was the most popular version of Firefox used for browsing the website?
- What is the most common HTTP method used?
- What is the second most common HTTP method used?
- How many requests were for \x04\x01\x00P\xC6\xCE\x0Eu0\x00?
first, let's download the given file and do some prelim investigation:
Click to reveal
NOTE: i renamed the file to nginxaccess.log so that it's more specific so i don't accidentally use a different file in the log-analysis folder.
head nginxaccess.log
wc -l nginxaccess.log
ok so it's not too big. the first question asks about the number of different IP addresses. let's use awk and sort -u to see if we can get a unique list of IP's.
Click to reveal
awk '{print $1}' nginxaccess.log | sort -u > IPs.txt; cat IPs.txt
how may unique's do we have?
Nice.
next we need to look for Status 200's. sounds simple enough. let's try to grep those into a new file:
Click to reveal
grep "200" nginxaccess.log > 200s.txt; wc -l 200s.txt
Nice again!
the next question is basically the same; this time looking for Status 400:
Click to reveal
we need to adjust the prior command a bit to ensure we're not looking at time zone information, namely the -0400 as seen here:
in order to avoid this, we can use awk instead of grep. grep alone will include some Status 200s and Status 502s, etc. the status codes exist in the 9th column, so let's try:
awk '{print $9}' nginxaccess.log| grep "400" > 400s.txt; cat 400s.txt
and the total is:
wc -l 400s.txt
unfortunately, because of how awk interprets "", we are missing some of the Status 400's. One example being the 5th line of the file:
because it has "", from awk's point of view, it should return 0 instead of 400. so let's turn to ChatGPT and see if it can help.
this looks like what we need. let's try it.
awk -F'"' '{split($3, a, " "); if (a[1] == 400) print $0}' nginxaccess.log
well, it's not very clean, but upon visual inspection, it did grab the "" 400-type lines, so let's drop it in a file and do a line-count on it:
awk -F'"' '{split($3, a, " "); if (a[1] == 400) print $0}' nginxaccess.log > 400s.txt; wc -l 400s.txt
there we go! we finally got it!
the next question is "What IP address rang at the doorbell?". i have no idea what this means, but thankfully we have the power of grep!
Click to reveal
grep -i "door" nginxaccess.log
nothing. ok let's try bell.
grep -i "bell" nginxaccess.log
ok! we got it! awesome!
let's continue our awesome grepping skills to try to answer the next question:
Click to reveal
grep -i "googlebot" nginxaccess.logs
Nice!
for this next question, i must admit i am not familiar with the "shellshock" vulnerability, but i'm guessing it's a similar payload to nc -l 8080 -e /bin/sh, so let's look for /bin and see what we can see:
Click to reveal
cat nginxaccess.log | grep -i "bin"
ok looks like we found something!
let's refine it to cat nginxaccess.log | grep -i "bin/bash"
and then let's drop this into a file and awk the first column to see how many IP addresses we have that are sending this payload.
cat nginxaccess.log | grep -i "bin/bash" | awk '{print $1}' | sort -u > evilIP.txt; cat evilIP.txt
perfect! exactly what we were hoping to see.
the next question asks about firefox, so let's grep again and see what we get
Click to reveal
grep -i "firefox" nginxaccess.log
We have quite a few options to choose from. so let's awk again and see if we can isolate that column:
Click to reveal
grep -i "firefox" nginxaccess.log | awk '{print $18,$19,$20,$21}'
again, it's messy but now we can easily see that v31.0 appears most often
the next question asks for the most common HTTP method, aka "HTTP request type". ok so we're looking for GET, POST, etc.
so now, (you guessed it!) let's grep:
Click to reveal
this time, let's use a bash loop and just drop each request type into it's own file so we can easily count each type.
for type in GET POST HEAD PUT DELETE OPTIONS CONNECT TRACE; do grep "$type" nginxaccess.log > "req${type}.txt"; done
now we can wc -l each file to see which has the most requests (with a bash loop of course):
for file in reqCONNECT.txt reqDELETE.txt reqGET.txt reqHEAD.txt reqOPTIONS.txt reqPOST.txt reqPUT.txt reqTRACE.txt; do wc -l $file; done
so now we can easily see which type of request occurs the most, and which is in 2nd place, answering quetions 8 and 9.
and finally, for the last request, we're just going to grep the syntax of characters in the question, being sure to double-backslash b/c \ is a shell meta character that needs to be escaped:
Click to reveal
** NOTE: ** up until this point, we have been using "" with grep to find specific strings. best practice is to use single quotes ' '. Use double quotes (") when you want Bash to evaluate variables. Use single quotes (') when you want exact text, no variable substitution.
keep this in mind if you get no results for a phrase that you know exists; like this:
we get no results even though the question tells us that the string exists. so let's try again with single quotes:
grep '\\x04\\x01\\x00P\\xC6\\xCE\\x0Eu0\\x00' nginxaccess.log
there we go. now since it's asking for a total count again, let's drop it into a file and count the lines:
grep '\\x04\\x01\\x00P\\xC6\\xCE\\x0Eu0\\x00' nginxaccess.log > oddrequests.txt; wc -l oddrequests.txt
And there we have it. NginX logs? DONE