Firewalld in Azure - hpaluch/hpaluch.github.io GitHub Wiki

Using Firewalld in Azure

[!CAUTION] Here is very risky setup that can easily make your VM unusable - and you will have to revert to snapshot (if having any) or drop it and create again (!)

Firewalld is normally disabled (and even not installed), because Microsoft wants to use Azure firewall rules. However sometimes we want to ensure that only expected traffic is here.

Tested OS: AlmaLinux 9, Image URN: almalinux:almalinux-x86_64:9-gen2:latest

Mysterious 168.63.129.16

Every Azure VM should run waagent - that provides basic communication with Cloud. You can find more information on https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/agent-linux

As described on https://learn.microsoft.com/en-us/troubleshoot/azure/virtual-machines/linux/linux-azure-guest-agent firewall may NOT block access to 168.63.129.16 which is basically Azure gate. You can find more info on https://learn.microsoft.com/en-us/azure/virtual-network/what-is-ip-address-168-63-129-16

If you have NFT tables installed (default) agent will automatically insert following rules:

$ nft list ruleset
...
# Warning: table ip security is managed by iptables-nft, do not touch!
table ip security {
	chain OUTPUT {
		type filter hook output priority 150; policy accept;
		ip daddr 168.63.129.16 tcp dport 53 counter packets 0 bytes 0 accept
		ip daddr 168.63.129.16 ip protocol tcp skuid 0 counter packets 3734 bytes 962201 accept
		ip daddr 168.63.129.16 ip protocol tcp ct state invalid,new counter packets 0 bytes 0 drop
	}
}

It means:

waagent is actually using iptables, not nftables (Warning: ...)
accept output connection to 168.63.129.16 to tcp port 53 - weird - DNS uses mostly UDP)
accept TCP output connection to 168.63.129.16 to any TCP port for client UID = 0 (root)
last rule says to discard any other new or invalid connections to 168.63.129.16 - this is clue that these Azure services are critical and should not be exposed(!)

Enabling firewalld

[!WARNING] You may easily make your VM unusable with steps below! Create Snapshot of your OS disk before proceeding!

Simply install firewalld with dnf install firewalld
enable old iptables and logging rejected unicast (single IP address target) packets by setting in /etc/firewalld/firewalld.conf
```
LogDenied=unicast
# waagent still uses iptables so we have to:
FirewallBackend=iptables
```
WARNING! Setting FirewallBackend=iptables will erase rules set by waagent (see above table security) Because default output policy is accept it have no effect (yet)... We have to add those rules later...
now reboot

After reboot firewall should be running - firewall-cmd --state

We should manually restore waagent rules using script restore-waagent-rules.sh with contents:

#!/bin/bash
set -xeuo pipefail
# OUTPUT_direct is reserved chain already created by firewalld
firewall-cmd --permanent --direct --add-rule \
   ipv4 filter OUTPUT_direct 0 -d 168.63.129.16/32 -p tcp -m tcp --dport 53 -j ACCEPT
firewall-cmd --permanent --direct --add-rule \
   ipv4 filter OUTPUT_direct 0 -d 168.63.129.16/32 -p tcp -m owner --uid-owner 0 -j ACCEPT
firewall-cmd --permanent --direct --add-rule \
   ipv4 filter OUTPUT_direct 0 -d 168.63.129.16/32 -p tcp -m conntrack --ctstate INVALID,NEW -j LOG \
   --log-prefix "waagent_DROP: "
firewall-cmd --permanent --direct --add-rule \
   ipv4 filter OUTPUT_direct 0  -d 168.63.129.16/32 -p tcp -m conntrack --ctstate INVALID,NEW -j DROP
firwall-cmd --reload
exit 0

and run it
you can find how often these rules are used with this command:

$ iptables -L OUTPUT_direct -n -v

Chain OUTPUT_direct (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            168.63.129.16        tcp dpt:53
   70  4200 ACCEPT     6    --  *      *       0.0.0.0/0            168.63.129.16        owner UID match 0
    0     0 LOG        6    --  *      *       0.0.0.0/0            168.63.129.16        ctstate INVALID,NEW LOG flags 0 level 4 prefix "waagent_DROP: "
    0     0 DROP       6    --  *      *       0.0.0.0/0            168.63.129.16        ctstate INVALID,NEW

Notice that "owner UID match 0" (owner is root) rule was already used many times (pkts and bytes are non-zero).

We can safely remove cockpit (tcp/9090) service:

firewall-cmd --permanent --zone=public --remove-service=cockpit
firewall-cmd --reload

You can same way remove dhcpv6-client from public zone if you don't use it...

If we try dmesg to see logged dropped packets there are interesting martian entries:

.. SRC=40.80.20.0 DST=10.X.X.X LEN=72 TOS=0x00 PREC=0x00 TTL=108 ID=0 DF
     PROTO=TCP SPT=23456 DPT=29110 WINDOW=17280 RES=0x00 ACK SYN URGP=0

Similar for source IP 40.119.96.0, 40.119.84.0, ... - port is always different. The only information I found was: https://ipinfo.io/ips/40.119.96.0/24

ASN	AS8075 Microsoft Corporation
BGP	40.112.0.0/13

So it is mystery for me...

Restricting SSH access

[!CAUTION]

If your client (trusted IP) change you can easily lockout yourself from VM!

The only clean way how to restrict access to service from IP is to create custom zone and assign it to IP range.

Here is sequence of command to use

replace X.X.X.X/32 with your IP address or set of IP addresses!!!

 Zone creation must be always "permanent":
firewall-cmd --new-zone=trusted-in --permanent
firewall-cmd --permanent --zone=trusted-in --permanent --add-source=X.X.X.X/32
firewall-cmd --permanent --zone=trusted-in --permanent --add-service=ssh
# must remove ssh service from public zone:
firewall-cmd --permanent --zone=public --permanent --remove-service ssh
# double-check above rules and apply them:
firewall-cmd --reload

We can see zone assignment using:

$ firewall-cmd --get-active-zones 

public
  interfaces: eth0
trusted-in
  sources: X.X.X.X/32

Restrict output connections

[!CAUTION] It is again dangerous and may easily render your VM unusable. Ensure that you created Snapshot before applying!

The only way how to restrict output connections in firewalld is to use so called "policies". We will follow https://access.redhat.com/solutions/7013886

Prepare script restrict-trust-out.sh wtiht contents:

#!/bin/bash
set -xeuo pipefail
# https://access.redhat.com/solutions/7013886
firewall-cmd --new-policy trust-out --permanent
firewall-cmd --set-target REJECT --policy trust-out --permanent
firewall-cmd --policy trust-out --add-egress-zone ANY --permanent
firewall-cmd --policy trust-out --add-ingress-zone HOST --permanent
# allow non-recursive trust-outgoing DNS connection
firewall-cmd --policy=trust-out \
        --add-rich-rule='rule family="ipv4" destination address="168.63.129.16" service name="dns" accept' --permanent
# if you are running DHCPv4 client you should allow also this rule for Renewal (non-broadcast IP):
#firewall-cmd --policy=trust-out \
#        --add-rich-rule='rule family="ipv4" destination address="168.63.129.16" service name="dhcp" accept' --permanent
# metadata server
firewall-cmd --policy=trust-out \
        --add-rich-rule='rule family="ipv4" destination address="169.254.169.254" service name="http" accept' --permanent
# beware - this will apply pending changes
firewall-cmd --reload
exit 0

Now run dmesg -Tw and watch if there are any unexpected entries regarding IP address 169.254.169.254 (less critical) and/or 168.63.129.16 (critical).

Please note that no output connection will work, for example:

$ curl www.google.com

To temporarily enable http(s) access create script enable-http.sh with contents:

#!/bin/bash
set -xeuo pipefail
sudo firewall-cmd --policy=trust-out --add-service=https --add-service=http
exit 0

After running it curl FQDN should work. You can again disable it with firewall-cmd --reload or similar script disable-http.sh

#!/bin/bash
set -xeuo pipefail
sudo firewall-cmd --policy=trust-out --remove-service=https --remove-service=http
exit 0

Sooner or later you will have to perform final test: Shutdown (from Azure portal). and Start of VM...

And again run dmesg to see if there are any unexpected denied packets...