Upstream how to use SF DPDK vdpa device - Mellanox/scalablefunctions GitHub Wiki

This section describes how to use DPDK based VDPA device of SF auxiliary device which were configured in step by step guide.

This section starts a DPDK vdpa example applicaiton to create vdpa socket for qemu virtio net interface, starts another DPDK testpmd to forward traffice between SF representor and PF.

1. List and view SF device

1.1 View the auxiliary device of the SF

There can be hundreds of auxiliary SF devices on the auxiliary bus. Each SF's auxiliary device contains a unique sfnum and PCI information.

Parent PCI BDF (Bus, Device, Function) and the SF number establishes a unique SF device.

Each SF's sfnum can be read using:

$ cat /sys/bus/auxiliary/devices/mlx5_core.sf.4/sfnum
88

1.3 View the RDMA device of the vdpa device

$ ls /sys/bus/auxiliary/devices/mlx5_core.sf.4/infiniband/
mlx5_4

1.4 Build DPDK with vdpa example enabled.

Currently, only mlnx_dpdk 20.11 supports SF. Steps to build DPDK from scratch:

$ git clone https://github.com/Mellanox/dpdk.org.git
$ cd dpdk.org && git checkout origin/mlnx_dpdk_20.11
$ meson -Dexamples=vdpa -Dbuildtype=debug /tmp/build
$ ninja -C /tmp/build

1.5 Use DPDK testpmd forward between SF representor port and PF

DPDK testpmd starts with IO fowarding mode by default if only two ports probed. Representor syntax is "<PCI>,representor=[pfX]sfY", pf section is option in non-lag mode.

$ /tmp/build/app/dpdk-testpmd --file-prefix=xx -w 0000:03:00.0,representor=sf88  -- --stats-period=1

representor syntax guide.

1.6 Use DPDK based vdpa device for the SF

Start DPDK vdpa example with SF devices:

$ /tmp/build/examples/dpdk-vdpa -w auxiliary:mlx5_core.sf.4,class=vdpa -- --iface /tmp/vhost-user-

DPDK VDPA guide. mlx5 VDPA guide.

1.7 Traffic test

Start up qemu with vdpa interface, configure virtio interface ip and ping remote side, the traffic should be captured on wire.