LibVirt is universal virtualization layer - it supports several hypervisors (QEMU+KVM, Xen, ...) and even containers (called LXC however it uses different toolset for these).

Note: because libvirt started as wrapper on Xen it uses (sometimes confusing) Xen terms - for example 'destroy' normally means "stop" or "shutdown" but Not delete... Similarly create means start (fortunately there is now also start command)

Installation under Fedora 40:

$ sudo dnf install libvirt-daemon-kvm libvirt-daemon-config-network libvirt-client-qemu

$ sudo dnf install virt-manager  # pouplar GUI for LibVirt

# ensure that creepyware geoclue2 is masked:
$ sudo systemctl mask --now geoclue
$ sudo rm -f /etc/xdg/autostart/geoclue-demo-agent.desktop

If you plan to use virt-manager remotely via SSH X11 Forwarding you have to install these packages on Server (with virt-manager):

# required
$ sudo dnf install xorg-x11-xauth

# optional - for testing
$ sudo dnf install xclock xterm

Also on your SSHd server enable X11 forwarding, for example, create /etc/ssh/sshd_config.d/99-local.conf with contents:

X11Forwarding yes

And restart SSHd with systemctl restart sshd

Also ensure that your SSH client requests X11 forwarding, here is excerpt from my ~/.ssh/config

Host f40lvm-s350
        HostName 192.168.0.X
        User USERNAME
        IdentityFile ~/.ssh/MY_SSH_KEY
        IdentitiesOnly yes
        HostKeyAlias f40lvm-s350
        ForwardX11 yes

Now try ssh connection from your client using alias, in my example: ssh f40lvm-s350 And try some X11 application, for example xclock (use Ctrl-C in terminal to quit)

But before running virt-manager please read following chapter:

By default when you run any libvirt command it will use your private User connection (called "session"). So all networks and disks and VMs will be available to you only (and data stored under $HOME/.local)

To use always shared global (called system) connection you have to:

1. Add yourself to libvirt group using:

   sudo /usr/sbin/usermod -G libvirt -a $USER

2. Create file ~/.config/libvirt/libvirt.conf with following content:

   # see https://listman.redhat.com/archives/libvirt-users/2018-October/msg00067.html
   uri_default = "qemu:///system"

On Fedora I have to enable and start "libvirtd legacy service" with:

$ sudo systemctl enable --now libvirtd

To avoid virsh client error:

Failed to connect socket to '/var/run/libvirt/virtqemud-sock': No such file or directory

Remember that you should logout and login to ensure that you are member of group libvirt. Then you can for example try listing networks or pools:

$ virsh net-list

 Name      State    Autostart   Persistent
--------------------------------------------
 default   active   yes         yes

$ virsh pool-list

 Name   State   Autostart
---------------------------

If there is no pool you can follow guide from Ubuntu MAAS KVM. Run as you (non-privileged user)

virsh pool-define-as default dir - - - - "/var/lib/libvirt/images"
virsh pool-autostart default
virsh pool-start default
virsh pool-list

   Name      State    Autostart
  -------------------------------
   default   active   yes

virsh pool-dumpxml default

   ...

At least when using NAT network we can pass option log-queries to dnsmasq that is used to provide both DHCP and DNS server for NAT network.

I followed various sources including: https://serverfault.com/a/1017645

Here is diff of default network - using virsh net-edit default or using virsh net-dumpxml default:

diff -u default.xml default-log-queries.xml
--- default.xml	2024-07-01 19:15:27.965334382 +0200
+++ default-log-queries.xml	2024-07-01 19:28:14.214143882 +0200
@@ -1,4 +1,4 @@
-<network connections='1'>
+<network xmlns:dnsmasq='http://libvirt.org/schemas/network/dnsmasq/1.0'>
   <name>default</name>
   <uuid>2c9b8477-9b7c-4ca5-8c20-f1fbcc7df3c3</uuid>
   <forward mode='nat'>
@@ -14,5 +14,8 @@
       <range start='192.168.100.128' end='192.168.100.254'/>
     </dhcp>
   </ip>
+  <dnsmasq:options>
+    <dnsmasq:option value='log-queries'/>
+  </dnsmasq:options>
 </network>

And then you have to restart network using (scary) destroy and start:

virsh net-destroy default
virsh net-start default

You can also peek content of /var/lib/libvirt/dnsmasq/default.conf if there is your option (log-queries in our case).

After restart you can try on Host:

journalctl -u libvirtd -f

And boot any VM that uses NAT network under LibVirt.

I often use private isolated network in LibVirt to avoid mixing copy of production workload (in test VM) with real production (for example, GitLab CE with active push mirrors would screw mirror targets!). To create private IPv4 network we can mostly follow: https://libvirt.org/formatnetwork.html#isolated-network-config

Examples below was tested on:

• Host: openSUSE LEAP 15.6
• Guest (VM): Ubuntu 24.04 LTS

Here is full definition of LibVirt network named private stored in net-private.xml with contents:

<network xmlns:dnsmasq='http://libvirt.org/schemas/network/dnsmasq/1.0'>
  <name>private</name>
  <bridge name="virbr9"/>
  <domain name='example.com' localOnly='yes'/>
  <dns>
    <host ip='10.99.99.20'>
      <hostname>vm1</hostname>
    </host>
  </dns>
  <ip address="10.99.99.1" netmask="255.255.255.0">
    <dhcp>
      <range start="10.99.99.100" end="10.99.99.200"/>
      <host mac='10:20:30:40:50:60' name='vm1.example.com' ip='10.99.99.20'/>
    </dhcp>
  </ip>
  <ip family="ipv6" address="fd00:0609:c0a8:0a00::1" prefix="64"/>
  <dnsmasq:options>
    <dnsmasq:option value='log-queries'/>
    <dnsmasq:option value='log-facility=/var/log/dnsmasq-private-net.log'/>
  </dnsmasq:options>
</network>

WARNING!

• if you will use above log-facility to log to file you have to prepare log file that is compatible with AppArmor policy - in openSUSE it is in /etc/apparmor.d/usr.sbin.dnsmasq

  /var/log/dnsmasq*.log w,
  

• so we must use log name matching /var/log/dnsmasq*.log

• and set proper permissions:

  sudo chown dnsmasq:nogroup /var/log/dnsmasq-private-net.log
  sudo touch /var/log/dnsmasq-private-net.log

• WARNING! AppArmor will NOT report denied access!

To create, start and autostart this private network at 10.99.99.1/24 you have to run:

xmllint --noout --relaxng /usr/share/libvirt/schemas/network.rng net-private.xml
virsh net-define net-private.xml
virsh net-start private
virsh net-autostart private

To be able to install packages I will also install Proxy under Host (openSUSE LEAP):

sudo zypper in tinyproxy

Next I changed its configuration /etc/tinyproxy/tinyproxy.conf - diff:

23c23
< Port 8888
---
> Port 3128
30c30
< #Listen 192.168.0.1
---
> Listen 10.99.99.1
96c96
< #LogFile "/var/log/tinyproxy/tinyproxy.log"
---
> LogFile "/var/log/tinyproxy/tinyproxy.log"
120c120,121
< LogLevel Info
---
> #LogLevel Info
> LogLevel Connect
190c191
< MaxClients 100
---
> MaxClients 20
201a203
> Allow 10.99.99.0/24
285c287
< #ConnectPort 443
---
> ConnectPort 443

Enable and start tinyproxy with:

sudo systemctl enable --now tinyproxy
# test
netstat -tln | fgrep 3128

  tcp        0      0 10.99.99.1:3128         0.0.0.0:*               LISTEN

# test proxy access - we must use Private network IP, because tinyproxy listens only there
curl -fsS -D - -o /dev/null -x 10.99.99.1:3128 http://www.linux.cz

  HTTP/1.1 200 OK
  Via: 1.1 tinyproxy (tinyproxy/1.11.2)
  Date: Wed, 04 Dec 2024 09:32:28 GMT
  Server: Apache

Now we can create sample VM (I will use Ubuntu 24.04 LTS) - remember to assign it network with name "private".

Once VM boots verify its IP address - must start with 10.99.99. otherwise you are using wrong network:

$ ip -br -4 a | fgrep -w UP

eth0             UP             10.99.99.193/24 metric 100

Step specific for openSUSE LEAP 15.6:

• I have to allow access to port 3128/tcp to zone libvirt using:

firewall-cmd --zone=libvirt --add-port=3128/tcp --permanent
firewall-cmd --reload

• WARNING! Even when I have set firewall-cmd --set-log-denied=unicast there is nothing logged by firewall!

Transparent Proxy access for APT:

• WARNING! Do not set globally proxy variables - there is risk that some system component will misuse it.

• from https://askubuntu.com/a/257296, to enable just APT proxy access, create file /etc/apt/apt.conf.d/99zz_proxy.conf with contents:

  Acquire::http::Proxy "http://10.99.99.1:3128";

Now inside Ubuntu guest you can run sudo apt-get update and ensure that there is NO single line with Ign prefix (Ignored). All should be Get: or Hit:

Now you can try to install some program, for example sudo apt-get install ncdu (neat tool to show directory usages interactively).

To use proxy (temporarily) with other commands I created script /usr/local/bin/http_proxy.sh with contents:

#!/bin/bash
set -euo pipefail
p=10.99.99.1:3128

[ $# -gt 0 ] || {
	echo "Usage: $0 command args ..." >&2
	exit 1
}
set -x
http_proxy=$p https_proxy=$p "$@"
exit 0

Example usage:

http_proxy.sh curl -fsS -D - -o /dev/null https://www.google.com

To enable Virtio GL 3D graphics there is one hidden requirement - service virtnodedevd.service used to enumerate hardware (including GPU cards) on Host. You have to install it with:

zypper in libvirt-daemon-driver-nodedev

And reboot. After reboot verify that following command returns non-empty list and no error:

virsh nodedev-list

Only then you will be able to select your GPU backend (if fulfills additional requirements).

There exist libivrt-nss plugin that allows automatic resolving of running VM names through /etc/nsswitch.conf so you can ping and/or ssh to VM without any trickery in /etc/hosts (or in HostName IP trick in ~/.ssh/config).

To install under Fedora you can just follow: https://libvirt.org/nss.html

dnf install libvirt-nss
authselect enable-feature with-libvirt # this will add "libvirt libvirt_guest" to /etc/nsswitch.conf

But under openSUSE LEAP 15.6 it is more tricky. Installation is similar:

zypper in libvirt-nss

But you have to manually modify /etc/nsswitch.conf because authselect package is incomplete in openSUSE case. So you final line in /etc/nsswitch.conf should look like:

hosts:  	files libvirt libvirt_guest dns

But it is still NOT all. If you now try ssh VM_NAME or ping VM_NAME it will not work:

ssh: Could not resolve hostname VM_NAME: Name or service not known

Audit search will reveal problem:

ausearch -ts boot -m avc -sv no -i -x /usr/sbin/nsc

...
type=PROCTITLE msg=audit(05/19/25 16:31:02.784:3523) : proctitle=/usr/sbin/nscd
type=PATH msg=audit(05/19/25 16:31:02.784:3523) : item=0 \
   name=/var/lib/libvirt/dnsmasq//virbr9.macs inode=3409148 \
   dev=103:02 mode=file,644 ouid=root ogid=root rdev=00:00 \
   nametype=NORMAL cap_fp=none cap_fi=none cap_fe=0 cap_fver=0 \
   cap_frootid=0
type=CWD msg=audit(05/19/25 16:31:02.784:3523) : cwd=/
type=SYSCALL msg=audit(05/19/25 16:31:02.784:3523) : \
  arch=x86_64 syscall=openat success=no \
  exit=EACCES(Permission denied) a0=0xffffff9c \
  a1=0x7fc9ac000f60 a2=O_RDONLY a3=0x0 items=1 ppid=1 \
  pid=5970 auid=unset uid=nscd gid=nscd euid=nscd \
  suid=nscd fsuid=nscd egid=nscd sgid=nscd fsgid=nscd \
  tty=(none) ses=unset comm=nscd exe=/usr/sbin/nscd \
  subj=nscd key=(null)
type=AVC msg=audit(05/19/25 16:31:02.784:3523) : \
  apparmor=DENIED operation=open class="file" profile=nscd \
  name=/var/lib/libvirt/dnsmasq/virbr9.macs pid=5970 comm=nscd \
  requested_mask=r denied_mask=r fsuid=nscd ouid=root

To fix this problem:

• append to /etc/apparmor.d/local/usr.sbin.nscd

  # Site-specific additions and overrides for 'usr.sbin.nscd'
  /var/lib/libvirt/dnsmasq/*.macs r,
  

• reload NSCD profile for AppArmor:

  apparmor_parser -r /etc/apparmor.d/usr.sbin.nscd

Now same command ssh VM_NAME or ping VM_NAME should work without issues.