User Tools

Site Tools


nndocs:infiniband:connectx-3

This is an old revision of the document!


SR-IOV

I don't think this can be made to work.

The number of VFs per port is controlled when the module is loaded:

options mlx4_core num_vfs=3,3,0 probe_vf=3,3,0

The triplet is the number of VFs for port 1, the number of VFs for port 2, and the number of VFs that contain both ports. The last number only works if the card is configured for ETH/ETH operation.

You must probe all your VFs. If you set “don't probe this” it will not be possible to change the VF settings.

The PCI devices, network devices visible via `ip a`, and devices visible via `ip link` do not match.

To change the settings on a VF, use driverctl to unbind a VF:

[root]@[shark][00:02:55][~]# driverctl set-override 0000:0e:00.5 none
[root]@[shark][00:03:06][~]# ip link set mlx1 vf 4 mac b6:2e:99:38:a9:66
[root]@[shark][00:04:01][~]# driverctl set-override 0000:0e:00.5 mlx4_core

However, the ONLY traffic I can get the VF to pass is broadcast storm traffic if I add it to a bridge and create a broadcast loop. In all other ways it acts like it's on a network segment by itself.

nndocs/infiniband/connectx-3.1786256247.txt.gz · Last modified: 2026/08/09 06:17 by naptastic