Performance tuning - Qrivi/KVM GitHub Wiki
These are some extra steps to optimize your host and QEMU configuration to get the most out of your KVMs. I recommend to do these only after a guest machine was set up successfully or the soonest after following the previous steps regarding the host.
Allow the kernel to isolate specific CPU cores so these can be dedicated to the guest, using the isolcpus
parameter. We'll also isolate these from kernel timer ticks by setting nohz_full
and remove them as candidates for RCU callbacks by specifying rcu_nocbs
. This way only core 0 is preserved for the host and housekeeping. We'll also set transparent_hugepage
to turn off transparent huge pages allowing for faster memory access for the VM.
You can use lscpu --extended
to learn about your CPU architecture. In my case I am using a i7 7700 which has 4 cores and runs 2 threads on each core, making for a total of 8 CPUs. The CPUs on core 0 are CPU 0 and CPU 4.
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ MINMHZ
0 0 0 0 0:0:0:0 yes 4200.0000 800.0000
1 0 0 1 1:1:1:0 yes 4200.0000 800.0000
2 0 0 2 2:2:2:0 yes 4200.0000 800.0000
3 0 0 3 3:3:3:0 yes 4200.0000 800.0000
4 0 0 0 0:0:0:0 yes 4200.0000 800.0000
5 0 0 1 1:1:1:0 yes 4200.0000 800.0000
6 0 0 2 2:2:2:0 yes 4200.0000 800.0000
7 0 0 3 3:3:3:0 yes 4200.0000 800.0000
To apply the changes described above, we'll need to edit the GRUB boot loader configuration. You will need sudo
to edit this file.
sudo nano /etc/default/grub
I'd suggest to add these parameters right after the iommu
parameter.
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt isolcpus=1-3,5-7 nohz_full=1-3,5-7 rcu_nocbs=1-3,5-7 apparmo r=1 security=apparmor udev.log_priority=3"
To persist these changes, rebuild the boot loader and reboot the system.
sudo grub-mkconfig -o /boot/grub/grub.cfg
reboot
Manjaro comes with the cpupower
utility that can elsewise be installed with sudo pacman -Sy cpupower
. There's also a cpupower-gui
package available in AUR. I'd suggest to set the CPU governor to "performance" (usually it is set to "powersave" by default, but this might depend on the CPU) in the GUI or by using the command below. If needed you can also increase the frequency on all cores.
sudo cpupower frequency-set -g perfomance
To get detailed current info about your CPU frequency, use the cpupower frequency-info
command. The output should be something as follows:
analyzing CPU 0:
driver: intel_pstate
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: Cannot determine or is not supported.
hardware limits: 800 MHz - 4.20 GHz
available cpufreq governors: performance powersave
current policy: frequency should be within 800 MHz and 4.20 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: Unable to call hardware
current CPU frequency: 4.09 GHz (asserted by call to kernel)
boost state support:
Supported: yes
Active: yes
SSH is ideal to quickly make small changes to the host when it is currently "not visible" (e.g. when host and guest are sharing a monitor and the guest is currently using it). Note that for the guest to be on the same network as the host, you will need to set up a network bridge and pass it as a NIC to the guest.
On a fresh Manjaro install, OpenSSH should be installed by default, but we will need to enable and start it.
sudo systemctl enable sshd.service
sudo systemctl start sshd.service
Because
libvirt
required us to install a firewall backend, incoming SSH connections will by default be blocked. Don't forget to whitelist the ports required — in this case 22 (TCP).
Personally I prefer hibernation over sleep so my PC can be powered off completely in between sessions, but when I power it back on, everything is right where and how I left it. Hibernation has advantages over sleep, but has some disadvantages as well. I did not create a swap partition when installing Manjaro, because a swap file is easier to manage.
First we need to create a file that will be used for swap, and make it so only the root user can write to it. Since we need to be able to dump our entire RAM into this file upon hibernation, its size must be at least the total amount of RAM installed in our system. It's recommended to make it a bit larger though everything depends on your setup and personal preference: if you have a good amount of RAM installed, it is unlikely applications will ever use (much) swap during normal use. Ubuntu's rule of thumb when using hibernation, is to make the swap file size equal to the amount of RAM plus the square root of the amount RAM. I just went with 50 GB since I have plenty of space left on my SSD dedicated to the host.
sudo fallocate -l 50G /swapfile
sudo chmod 600 /swapfile
Next up we'll configure and enable the file as a Linux swap area.
sudo mkswap /swapfile
sudo swapon /swapfile
To make this permanent, we'll add an entry for our swipfile in /etc/fstab
. You will need sudo
to edit this file.
/swapfile none swap defaults 0 0
We need to set two extra kernel parameters in order to use a swap file for hibernation: the volume where the swap file resides (resume
parameter) and its physical offset (resume_offset
parameter). These values can be retrieved using the following commands respectively:
findmnt -no UUID -T /swapfile
sudo filefrag -v /swapfile | awk '{ if($1=="0:"){print ($4+0)} }'
Set these parameters in the GRUB boot loader configuration. You will need sudo
to edit this file.
sudo nano /etc/default/grub
I'd suggest to add these parameters somewhere in the beginning, prior to add the iommu
parameter. Keep in mind that because we are identifying the swap volume by its UUID, we need to include UUID=
before the actual UUID in the resume
parameter.
GRUB_CMDLINE_LINUX_DEFAULT="quiet resume=UUID=24028771-5be8-4a52-977f-2767b353d3df resume_offset=36872192 intel_iommu=on iommu=pt isolcpus=1-3,5-7 nohz_full=1-3,5-7 rcu_nocbs=1-3,5-7 apparmo r=1 security=apparmor udev.log_priority=3"
To persist these changes, rebuild the boot loader and reboot the system.
sudo grub-mkconfig -o /boot/grub/grub.cfg
reboot
You can check your RAM and swap state using free -h
.
total used free shared buff/cache available
Mem: 31Gi 24Gi 292Mi 178Mi 6.0Gi 5.7Gi
Swap: 49Gi 2.0Mi 49Gi