Gfilui

I'm facing some problem with DPDK on Ubuntu 16.04 with a Mellanox 40G NIC.

I'm not able to assign the NIC to an ovs (trying to use vfio-pci driver).

When I try to assign the NIC to the OVS with the following command:

ovs-vsctl add-port br0 ens1 -- set Interface ens1 type=dpdk options:dpdk-devargs=0000:07:00.0

I get the following log message:

Apr 17 15:26:43 olnmalp026Bn001 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-port br0 ens1 -- set Interface ens1 type=dpdk options:dpdk-devargs=0000:07:00.0
Apr 17 15:26:43 olnmalp026Bn001 ovs-vswitchd[56850]: ovs|00184|dpdk|ERR|EAL: Driver cannot attach the device (0000:07:00.0)
Apr 17 15:26:43 olnmalp026Bn001 ovs-vswitchd[56850]: ovs|00185|dpdk|ERR|EAL: No port found for device (0000:07:00.0)

The strangest thing I've noticed, is that if I run dpdk-procinfo, I get the following output:

root@olnmalp026Bn001:/home/sysadmin# dpdk-procinfo
EAL: Detected 56 lcore(s)
EAL: Probing VFIO support...
EAL: cannot connect to primary process!
EAL: VFIO support could not be initialized
EAL: PCI device 0000:02:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 net_e1000_igb
EAL: PCI device 0000:02:00.1 on NUMA socket 0 
EAL: probe driver: 8086:1521 net_e1000_igb
dpdk-procinfo [EAL options] -- -p PORTMASK
 -m to display DPDK memory zones, segments and TAILQ information
 -p PORTMASK: hexadecimal bitmask of ports to retrieve stats for
 --stats: to display port statistics, enabled by default
 --xstats: to display extended port statistics, disabled by default
 --metrics: to display derived metrics of the ports, disabled by default
 --xstats-name NAME: to display single xstat id by NAME
 --xstats-ids IDLIST: to display xstat values by id. The argument is comma-separated list of xstat ids to print out.
 --stats-reset: to reset port statistics
 --xstats-reset: to reset port extended statistics
 --collectd-format: to print statistics to STDOUT in expected by collectd format
 --host-id STRING: host id used to identify the system process is running on
EAL: Error - exiting with code: 1
 Cause: No Ethernet ports - bye

The Mellanox board, does not appear in the PCI devices listed in that command!

root@olnmalp026Bn001:/home/sysadmin# lspci | grep -i mellan
07:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]

But I am able to assign the PCI address to the vfio-pci driver:

root@olnmalp026Bn001:/home/sysadmin# dpdk-devbind -s

Network devices using DPDK-compatible driver
============================================
0000:07:00.0 'MT27710 Family [ConnectX-4 Lx] 1015' drv=vfio-pci unused=uio_pci_generic

Network devices using kernel driver
===================================
0000:02:00.0 'I350 Gigabit Network Connection 1521' if=enp2s0f0 drv=igb unused=vfio-pci,uio_pci_generic
0000:02:00.1 'I350 Gigabit Network Connection 1521' if=enp2s0f1 drv=igb unused=vfio-pci,uio_pci_generic
...

I think the Mellanox driver is properly installed in the machine:

root@olnmalp026Bn001:/home/sysadmin# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
[warn] mst_pciconf is already loaded, skipping
Create devices
Unloading MST PCI module (unused) - Success

root@olnmalp026Bn001:/home/sysadmin# lsmod | grep mst_
mst_pciconf 102400 0

Some data about the system, in case it helps:

root@olnmalp026Bn001:/home/sysadmin# uname -a
Linux olnmalp026Bn001 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@olnmalp026Bn001:/home/sysadmin# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.9.0
DB Schema 7.15.1

root@olnmalp026Bn001:/home/sysadmin# dpkg -l | grep -i dpdk
ii dpdk 17.11-5 amd64 Data Plane Development Kit (runtime)
ii mlnx-ofed-dpdk 4.3-1.0.1.0 all MLNX_OFED dpdk installer package (with DKMS support)
ii openvswitch-switch-dpdk 2.9.0-2 amd64 DPDK enabled Open vSwitch switch implementation