Automated Threat Mitigation with fail2ban and WAF - QualitativeDataRepository/TechnicalTeam GitHub Wiki

Proposal

We will configure fail2ban to automatically manage WAF ip set blocks.

Fail2ban

  • All interactions with the IDP are recorded in Apache access log.
  • Fail2ban can monitor this file and detect anomalous behavior from specific IP addresses.
  • Flagged IP addresses are blocked from accessing QDR by WAF.

Implementation

Fail2ban filter

  • Regex used to identify offending actions in IDP logs.
failregex = .*\s<HOST>.*POST.*idp/profile/SAML2/Redirect/SSO.*302.*

Fail2ban jail

  • 50 login failures in 10 minutes results in 1 hour ban.
logpath  = /var/log/apache2/idp.log
action   = waf
maxretry = 50
findtime = 10m
bantime  = 60m

Fail2ban action

These actions are executed to add and remove blocked IPs.

actionban = /bin/bash /etc/fail2ban/action.d/waf_add.sh <ip>/32
actionunban = /bin/bash /etc/fail2ban/action.d/waf_del.sh <ip>/32

waf_add.sh and waf_del.sh

  • Use aws CLI to manage waf resources.
#!/bin/bash
ip_set_id=$(aws wafv2 list-ip-sets --scope REGIONAL --region us-east-1  | jq -r .IPSets[].Id)
ip_set_token=$(aws wafv2 list-ip-sets --scope REGIONAL --region us-east-1  | jq -r .IPSets[].LockToken)
block_cidr=$1
aws wafv2 get-ip-set --name block --scope REGIONAL --region us-east-1 --id $ip_set_id > /tmp/ip_set_output
# Get token from the JSON
LOCK_TOKEN=$(jq -r '.LockToken' /tmp/ip_set_output)
# Get IP list from the JSON
arr=( $(jq -r '.IPSet.Addresses[]' /tmp/ip_set_output) )
# Add our ip to the list
arr+=( "${block_cidr}" )
echo "Adding to block ip set: $1"
next_token=$(aws wafv2 update-ip-set --name block --scope REGIONAL --region us-east-1 --lock-token $ip_set_token  --id $ip_set_id --addresses "${arr[@]}" | jq -r .NextLockToken)
echo $next_token > /tmp/waf_next_token

Breakglass and Fault Tolerance

  • Monitoring of fail2ban block list can be done with fail2ban-client status idp
  • Individual or all IPs can be removed from the command line, e.g. fail2ban-client unban --all
  • We will configure a nightly pruning of the WAF ip set block list to guarantee that no legitimate users are permanently blocked.
  • Blocking logic must be tuned so that multiple users connecting from a shared IP will not be blocked too quickly.
⚠️ **GitHub.com Fallback** ⚠️