lfx weekly update ‐ task 4 - Aditya-Sarna/lfx_weeklyupdate GitHub Wiki
I checked the lab repository first, then verified the live Dell host (dh5) and the BlueField-3 over rshim. I did not flash a BF-Bundle, reinstall packages, or reboot. BlueField-3 out-of-band Ethernet / 172.22.3.5 is not on the lab management network. This section uses the host PCIe path and rshim only.
| Check | Status |
|---|---|
| Host DOCA and NVIDIA drivers | Done. DOCA 3.4.0112, OFED 26.04-0.8.6 |
| Host NIC firmware | Done from the host. ethtool reports 32.40.1000 on 0000:0d:00.0. That is not a DPU bfver / flint reading |
| DPU DOCA / BF-Bundle | Done earlier over rshim. /etc/mlnx-release is DOCA_2.6.0_BSP_4.6.0_Ubuntu_22.04-5.24-01.prod
|
DPU bfver / DPU dpkg / sample path |
Not captured. The 2026-09-09 probe stopped after mst
|
| Which versions belong together | Live 3.4 host (doca-all) + 2.6 DPU is not in-policy. Same-minor is sufficient, not required (Level 2) |
lspci / mst / mlx5 |
Done. Host detects BlueField-3 |
| Host-to-DPU management | Done over rshim (192.168.100.1 ↔ 192.168.100.2). Not 172.22.3.5
|
doca_dma_copy |
Not run |
On root@dh5 (Ubuntu 24.04.3, kernel 6.8.0-138-generic):
ofed_info -s
doca_caps --version
dpkg -l | grep -iE 'rshim|mft|mlnx-ofed|doca-ofed'
ethtool -i enp13s0f0np0
Expected: a DOCA-Host version, an OFED/mlnx driver version, and a NIC firmware line.
Result:
| Item | Live | When recorded |
|---|---|---|
| Apt source | https://linux.mellanox.com/public/repo/doca/3.4.0/ubuntu24.04/x86_64/ |
2026-09-07 |
doca_caps |
DOCA 3.4.0112 | 2026-09-09 |
ofed_info -s |
OFED-internal-26.04-0.8.6 | 2026-09-09 |
doca-ofed |
3.4.0-085000 | 2026-09-09 |
doca-all / doca-devel / doca-runtime
|
3.4.0-085000 | 2026-09-07 dpkg (listing truncated at doca-sdk-mgmt) |
doca-sdk-dma / doca-samples
|
3.4.0112-1 | 2026-09-07 dpkg
|
mlnx-ofed-kernel-dkms |
26.04.OFED.26.04.0.8.6.1-1 | 2026-09-09 |
MFT (mst) |
4.36.0-147 | 2026-09-09 |
| rshim | 2.7.3-0.g17aaf11.2604086 | 2026-09-09 |
mlx5_core |
26.04-0.8.6 | 2026-09-09 ethtool
|
| NIC firmware | 32.40.1000 (MT_0000000884), bus 0000:0d:00.0
|
2026-09-09 ethtool -i enp13s0f0np0
|
Problems: modprobe rshim fails; rshim is a userspace service, not a kernel module.
Working login is ssh [email protected] from the host. On that login I recorded /etc/mlnx-release:
DOCA_2.6.0_BSP_4.6.0_Ubuntu_22.04-5.24-01.prod
| Item | Live |
|---|---|
| OS | Ubuntu 22.04.3, 5.15.0-1035-bluefield, aarch64 |
| Hostname | localhost.localdomain |
| BF-Bundle / DOCA | DOCA 2.6.0, BSP 4.6.0, Ubuntu 22.04 |
| rshim |
tmfifo_net0 192.168.100.2/30 |
| Data interfaces |
enp3s0f0s0 172.16.3.92, enp3s0f1s0 172.16.3.107 (from an earlier ip -br a) |
bfver / DPU dpkg / flint on the DPU |
Not captured |
Problems: . Planned OOB address 172.22.3.5 does not answer from the VPN.
NVIDIA ships both packages per release on the DOCA Downloads page. Install or upgrade DOCA on the host before installing BF-Bundle on the BlueField (installation guide, BF-Bundle install).
NVIDIA does not require the two sides to be the same minor version in every case. The DOCA Dependency Compatibility Policy (updated 2026-09-01) assigns levels:
| Upper layer | Lower layer | Level |
|---|---|---|
| DOCA-Host: DOCA-OFED profile | BF-Bundle | Level 1 – Perpetual |
| DOCA-Host: OTHER profiles (Networking, All) | BF-Bundle | Level 2 – Limited |
This host has doca-all, so host vs BF-Bundle is Level 2. NVIDIA’s text:
| Pair | Under that policy |
|---|---|
Live: DOCA-Host 3.4.0 (doca-all) + DPU 2.6.0 |
No. 2.6.0 is January 2024 — outside the annual cycle that contains 3.4/3.5 |
| DOCA-Host 3.4.0 + BF-Bundle 3.4.0 | Yes. Matched release. Host is already 3.4.0. Not installed on the DPU |
| DOCA-Host 3.4.0 + BF-Bundle 3.5.0 | Yes, Level 2. April 2026 + July 2026 are in the same annual cycle. Not installed |
| DOCA-Host 3.5.0 + BF-Bundle 3.5.0 | Yes. Matched release. Would also require upgrading the host. Not installed |
Lab BlueField-2 file bf-bundle-2.7.0-33_…
|
No. Wrong DOCA generation and wrong card line |
I have not flashed a bundle to align the versions.
Live lspci on dh5:
| PCI | Function |
|---|---|
| 0d:00.0 | BlueField-3 integrated ConnectX-7 |
| 0d:00.1 | BlueField-3 integrated ConnectX-7 |
| 0d:00.2 | BlueField-3 SoC Management Interface |
/dev/rshim0/misc reports BlueField-3 (Rev 1) at pcie-0000:0d:00.2.
ethtool -i enp13s0f0np0
mst start
mst status -v
| Interface | PCI | RDMA | Driver |
|---|---|---|---|
| enp13s0f0np0 | 0d:00.0 | mlx5_0 | mlx5_core 26.04-0.8.6 |
| enp13s0f1np1 | 0d:00.1 | mlx5_1 | mlx5_core (from mst; ethtool was run on f0 only) |
Host-side data addresses (earlier ip): enp13s0f0np0 172.16.3.65, enp13s0f1np1 172.16.3.66.
Both rows are BlueField3 (rev 1):
| MST device | PCI | NET |
|---|---|---|
| /dev/mst/mt41692_pciconf0 | 0d:00.0 | net-enp13s0f0np0 |
| /dev/mst/mt41692_pciconf0.1 | 0d:00.1 | net-enp13s0f1np1 |
Expected: mst names BlueField-3 and binds mlx5 netdevs to 0d:00.0 and 0d:00.1.
Problems: mst warned that mst_pciconf was already loaded. It printed that the MST PCI module was not loaded while the PCI configuration module was loaded. Devices were still listed. 0d:00.2 is rshim, not a third mlx5 netdev.
The host detects the BlueField-3 correctly over PCIe.
systemctl start rshim
ip link set tmfifo_net0 up
ip addr add 192.168.100.1/30 dev tmfifo_net0
ping -c 2 192.168.100.2
ssh [email protected]
Expected: ping and SSH over rshim.
Result: ping succeeds. SSH succeeds. The DPU prompt is ubuntu@localhost. Inventory lists bf3 as ansible_user=root at 172.22.3.5; that address does not answer. The working login is ubuntu over rshim.