PCI Passthrough

OpenNebula supports PCI passthrough, allowing virtual machines to have direct access to physical PCI devices. For a detailed understanding of this feature in OpenNebula, please refer to the official documentation on PCI Passthrough.

This document describes the one-deploy feature that automates the configuration required for PCI passthrough on both the hypervisor nodes and the OpenNebula front-end.

How It Works

The PCI passthrough feature in one-deploy is handled by two main roles:

• pci_passthrough/node: This role runs on the hypervisor nodes. It handles the low-level configuration of the specified PCI devices. For each device, it uses driverctl to bind it to the vfio-pci driver, making it available for passthrough. It also creates a udev rule to ensure correct permissions for the VFIO devices.

• pci_passthrough/frontend: This role configures the OpenNebula front-end. It automatically enables the monitoring probe to discover all PCI devices. More importantly, it reads the PCI device information (Vendor and Device ID) from the configured nodes and adds it to each Host's template as a PCI_FILTER. This ensures that OpenNebula can consistently identify devices even if their bus address changes after a reboot.

Ansible Role Configuration

Below is a complete sample inventory file demonstrating how to enable and configure PCI passthrough for different nodes.

---
all:
  vars:
    ansible_user: root
    one_version: '7.0'
    one_pass: opennebulapass
    ds:
      mode: ssh
    vn:
      admin_net:
        managed: true
        template:
          VN_MAD: bridge
          PHYDEV: eth0
          BRIDGE: br0
          AR:
            TYPE: IP4
            IP: 192.168.122.100
            SIZE: 48
          NETWORK_ADDRESS: 192.168.122.0
          NETWORK_MASK: 255.255.255.0
          GATEWAY: 192.168.122.1
          DNS: 1.1.1.1

frontend:
  hosts:
    f1: { ansible_host: 192.168.122.2 }

node:
  hosts:
    n1:
      ansible_host: 192.168.122.3
      pci_passthrough_enabled: true
      pci_devices:
        - address: "0000:09:00.0"
        - address: "0000:0a:00.0"
    n2:
      ansible_host: 192.168.122.4
      pci_passthrough_enabled: true
      pci_devices:
        - address: "0000:09:00.0"
    n3:
      ansible_host: 192.168.122.5
      pci_passthrough_enabled: false

To enable the feature, you need to set the pci_passthrough_enabled variable to true on each hypervisor node where you want to use it.

node:
  hosts:
    n1:
      ansible_host: 192.168.122.3
      pci_passthrough_enabled: true

Next, for each of those nodes, you must provide a list of pci_devices to be configured. The devices are identified by their full PCI bus address (short addresses are not supported).

      pci_devices:
        - address: "0000:09:00.0"
        - address: "0000:0a:00.0"

[!IMPORTANT] The role will automatically install the driverctl package on the nodes, which is required for managing the device drivers.

[!NOTE] The role automatically translates the PCI address (e.g., "0000:09:00.0") into a stable Vendor:DeviceID format. This ID is then used to create a PCI_FILTER attribute on the OpenNebula host, preventing issues if the device address changes.

After providing the above configuration, you can provision your environment as usual:

$ make I=inventory/pci_passthrough.yml