Light At The End Of The Tunnel

This post is to detail how I managed to reliably set the additional settings for  Exablaze’s ExaNIC X40 card, the technique is applicable to other cards. I was performing this on a Dell R730 running CentOS 7.4

The Exablaze cards have additional capabilities like bypass-only mode, which needs to be set every time the system is rebooted.  In the past, I’ve covered this by using udev, but it hasn’t proven to be reliable.

Firstly, systemd-networkd isn’t the default network management system installed by RedHat/CentOS, that is NetworkManager. So you need to remove NetworkManager first and then use systemd-networkd, this has been covered by others.

Typically, on a Dell system you will see the output of exanic-config for an X40 like below:-

Device exanic0:
 Hardware type: ExaNIC X40
 Board ID: 0x00
 Temperature: 49.7 C VCCint: 0.94 V VCCaux: 1.84 V
 Function: network interface
 Firmware date: 20160822 (Mon Aug 22 21:34:58 2016)
 Port 0:
 Interface: enp1s0
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, link active
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:58
 RX packets: 3984824705 ignored: 23380569 error: 0 dropped: 0
 TX packets: 1626531
 Port 1:
 Interface: enp1s0d1
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, link active
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:59
 RX packets: 4117903238 ignored: 36763763 error: 1 dropped: 0
 TX packets: 1626493
 Port 2:
 Interface: enp1s0d2
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5a
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0
 Port 3:
 Interface: enp1s0d3
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5b
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0
 Port 4:
 Interface: enp1s0d4
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5c
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0
 Port 5:
 Interface: enp1s0d5
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5d
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0
 Port 6:
 Interface: enp1s0d6
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5e
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0
 Port 7:
 Interface: enp1s0d7
 Port speed: 10000 Mbps
 Port status: enabled, no SFP, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: off
 Promiscuous mode: off
 Bypass-only mode: off
 MAC address: 64:3f:5f:01:3b:5f
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0

These days I prefer the names of my interfaces to reflect their purpose or what they are connected to. So first steps is to rename them and configure them for normal networking attributes.

cd /etc/systemd/network

create a 50-timing.lnk file like

[Match]
MACAddress=64:3f:5f:01:3b:5f

[Link]
Name=timing
MACAddressPolicy=persistent

The above file tells the system to match on the physical MAC address of the network port, and then rename it to be called timing.

Create a timing.network file

[Match]
Name=timing

[Network]
Address=192.168.3.162/24

This file matches on the interface’s name and then sets it’s IP address and netmask.

OK for this port we also wish it to have bypass-only mode on when the port comes up.

Create /etc/systemd/system/bypass@.service

[Unit]
Description=bypass-only configuration for %i
After=network.target
Requires=sys-subsystem-net-devices-%i.device
After=sys-subsystem-net-devices-%i.device

[Service]
ExecStart=/usr/bin/exanic-config %i bypass-only on
Type=oneshot

[Install]
WantedBy=multi-user.target

The above file describes that the service is not to run until after the network.target is complete and then use exanic-config to set bypass-only on.

Now perform the following:-

systemctl daemon-reload
systemctl enable bypass@timing.service

Now after a reboot we should be able to do exanic-config timing and see :-

Device exanic0:
 Hardware type: ExaNIC X40
 Board ID: 0x00
 Temperature: 65.1 C VCCint: 0.93 V VCCaux: 1.80 V
 Function: network interface
 Firmware date: 20170529 (Mon May 29 23:57:51 2017)
 Port 6:
 Interface: timing
 Port speed: 10000 Mbps
 Port status: enabled, SFP present, no link
 MAC filters: 64 IP filters: 128
 Loopback mode: on
 Promiscuous mode: on
 Bypass-only mode: on
 MAC address: 64:3f:5f:01:3b:5f
 IP address: 192.168.3.162 Mask: 255.255.255.0
 RX packets: 0 ignored: 0 error: 0 dropped: 0
 TX packets: 0

I created a similar service for the promisc option too.

One caveat, for the network interfaces to be renamed in this manner I had to add “net.ifnames=0” to my kernel command line