Performance tuning - Qrivi/KVM GitHub Wiki

These are some extra steps to optimize your host and QEMU configuration to get the most out of your KVMs. I recommend to do these only after a guest machine was set up successfully or the soonest after following the previous steps regarding the host.

Isolating CPU cores

Allow the kernel to isolate specific CPU cores so these can be dedicated to the guest, using the isolcpus parameter. We'll also isolate these from kernel timer ticks by setting nohz_full and remove them as candidates for RCU callbacks by specifying rcu_nocbs. This way only core 0 is preserved for the host and housekeeping. We'll also set transparent_hugepage to turn off transparent huge pages allowing for faster memory access for the VM.

You can use lscpu --extended to learn about your CPU architecture. In my case I am using a i7 7700 which has 4 cores and runs 2 threads on each core, making for a total of 8 CPUs. The CPUs on core 0 are CPU 0 and CPU 4.

CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ   MINMHZ
  0    0      0    0 0:0:0:0          yes 4200.0000 800.0000
  1    0      0    1 1:1:1:0          yes 4200.0000 800.0000
  2    0      0    2 2:2:2:0          yes 4200.0000 800.0000
  3    0      0    3 3:3:3:0          yes 4200.0000 800.0000
  4    0      0    0 0:0:0:0          yes 4200.0000 800.0000
  5    0      0    1 1:1:1:0          yes 4200.0000 800.0000
  6    0      0    2 2:2:2:0          yes 4200.0000 800.0000
  7    0      0    3 3:3:3:0          yes 4200.0000 800.0000

To apply the changes described above, we'll need to edit the GRUB boot loader configuration. You will need sudo to edit this file.

sudo nano /etc/default/grub

I'd suggest to add these parameters right after the iommu parameter.

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt isolcpus=1-3,5-7 nohz_full=1-3,5-7 rcu_nocbs=1-3,5-7 apparmo r=1 security=apparmor udev.log_priority=3"

To persist these changes, rebuild the boot loader and reboot the system.

sudo grub-mkconfig -o /boot/grub/grub.cfg
reboot

Tuning CPU frequency

Manjaro comes with the cpupower utility that can elsewise be installed with sudo pacman -Sy cpupower. There's also a cpupower-gui package available in AUR. I'd suggest to set the CPU governor to "performance" (usually it is set to "powersave" by default, but this might depend on the CPU) in the GUI or by using the command below. If needed you can also increase the frequency on all cores.

sudo cpupower frequency-set -g perfomance

To get detailed current info about your CPU frequency, use the cpupower frequency-info command. The output should be something as follows:

analyzing CPU 0:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 0
  CPUs which need to have their frequency coordinated by software: 0
  maximum transition latency:  Cannot determine or is not supported.
  hardware limits: 800 MHz - 4.20 GHz
  available cpufreq governors: performance powersave
  current policy: frequency should be within 800 MHz and 4.20 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency: Unable to call hardware
  current CPU frequency: 4.09 GHz (asserted by call to kernel)
  boost state support:
    Supported: yes
    Active: yes

Enabling SSH server

SSH is ideal to quickly make small changes to the host when it is currently "not visible" (e.g. when host and guest are sharing a monitor and the guest is currently using it). Note that for the guest to be on the same network as the host, you will need to set up a network bridge and pass it as a NIC to the guest.

On a fresh Manjaro install, OpenSSH should be installed by default, but we will need to enable and start it.

sudo systemctl enable sshd.service
sudo systemctl start sshd.service

Because libvirt required us to install a firewall backend, incoming SSH connections will by default be blocked. Don't forget to whitelist the ports required — in this case 22 (TCP).

Hibernation into swap file

Personally I prefer hibernation over sleep so my PC can be powered off completely in between sessions, but when I power it back on, everything is right where and how I left it. Hibernation has advantages over sleep, but has some disadvantages as well. I did not create a swap partition when installing Manjaro, because a swap file is easier to manage.

First we need to create a file that will be used for swap, and make it so only the root user can write to it. Since we need to be able to dump our entire RAM into this file upon hibernation, its size must be at least the total amount of RAM installed in our system. It's recommended to make it a bit larger though everything depends on your setup and personal preference: if you have a good amount of RAM installed, it is unlikely applications will ever use (much) swap during normal use. Ubuntu's rule of thumb when using hibernation, is to make the swap file size equal to the amount of RAM plus the square root of the amount RAM. I just went with 50 GB since I have plenty of space left on my SSD dedicated to the host.

sudo fallocate -l 50G /swapfile
sudo chmod 600 /swapfile

Next up we'll configure and enable the file as a Linux swap area.

sudo mkswap /swapfile
sudo swapon /swapfile

To make this permanent, we'll add an entry for our swipfile in /etc/fstab. You will need sudo to edit this file.

/swapfile       none     swap     defaults 0 0

We need to set two extra kernel parameters in order to use a swap file for hibernation: the volume where the swap file resides (resume parameter) and its physical offset (resume_offset parameter). These values can be retrieved using the following commands respectively:

findmnt -no UUID -T /swapfile
sudo filefrag -v /swapfile | awk '{ if($1=="0:"){print ($4+0)} }'

Set these parameters in the GRUB boot loader configuration. You will need sudo to edit this file.

sudo nano /etc/default/grub

I'd suggest to add these parameters somewhere in the beginning, prior to add the iommu parameter. Keep in mind that because we are identifying the swap volume by its UUID, we need to include UUID= before the actual UUID in the resume parameter.

GRUB_CMDLINE_LINUX_DEFAULT="quiet resume=UUID=24028771-5be8-4a52-977f-2767b353d3df resume_offset=36872192 intel_iommu=on iommu=pt isolcpus=1-3,5-7 nohz_full=1-3,5-7 rcu_nocbs=1-3,5-7 apparmo r=1 security=apparmor udev.log_priority=3"

To persist these changes, rebuild the boot loader and reboot the system.

sudo grub-mkconfig -o /boot/grub/grub.cfg
reboot

You can check your RAM and swap state using free -h.

              total        used        free      shared  buff/cache   available
Mem:           31Gi        24Gi       292Mi       178Mi       6.0Gi       5.7Gi
Swap:          49Gi       2.0Mi        49Gi
⚠️ **GitHub.com Fallback** ⚠️