CentOS – NIC Teaming with LACP

wriedel — Mon, 10 Sep 2018 20:19:16 +0000

A Comparison of Features in Bonding and Team

Feature	Bonding	Team
broadcast Tx policy	Yes	Yes
round-robin Tx policy	Yes	Yes
active-backup Tx policy	Yes	Yes
LACP (802.3ad) support	Yes (active only)	Yes
Hash-based Tx policy	Yes	Yes
User can set hash function	No	Yes
Tx load-balancing support (TLB)	Yes	Yes
LACP hash port select	Yes	Yes
load-balancing for LACP support	No	Yes
Ethtool link monitoring	Yes	Yes
ARP link monitoring	Yes	Yes
NS/NA (IPv6) link monitoring	No	Yes
ports up/down delays	Yes	Yes
port priorities and stickiness (“primary” option enhancement)	No	Yes
separate per-port link monitoring setup	No	Yes
multiple link monitoring setup	Limited	Yes
lockless Tx/Rx path	No (rwlock)	Yes (RCU)
VLAN support	Yes	Yes
user-space runtime control	Limited	Full
Logic in user-space	No	Yes
Extensibility	Hard	Easy
Modular design	No	Yes
Performance overhead	Low	Very Low
D-Bus interface	No	Yes
multiple device stacking	Yes	Yes
zero config using LLDP	No	(in planning)
NetworkManager support	Yes	Yes

Config Snippets:

CentOS 7
No NetworkManager

NEXUS vPC Switch A:

!
interface port-channel223
  description LACP: to nas-a
  switchport access vlan 161
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  bandwidth 80000000
  vpc 223
!
interface Ethernet2/3
  description NAS: nas-a UCS-C240M4-A-A
  no lldp transmit
  no lldp receive
  switchport access vlan 161
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  channel-group 223 mode active
!

!
interface port-channel224
  description LACP: to nas-b
  switchport access vlan 162
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  bandwidth 80000000
  vpc 224
!
interface Ethernet2/4
  description NAS: nas-b UCS-C240M4-B-A
  no lldp transmit
  no lldp receive
  switchport access vlan 162
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  channel-group 224 mode active
!

core-A# sh vpc 223
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
223    Po223       up     success     success                    161         

core-A# sh vpc 224
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
224    Po224       up     success     success                    162

NEXUS vPC Switch B:

!
interface port-channel223
  description LACP: to nas-a
  switchport access vlan 161
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  bandwidth 80000000
  vpc 223
!
interface Ethernet2/3
  description NAS: nas-a UCS-C240M4-A-B
  no lldp transmit
  no lldp receive
  switchport access vlan 161
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  channel-group 223 mode active
!

!
interface port-channel224
  description LACP: to nas-b
  switchport access vlan 162
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  bandwidth 80000000
  vpc 224
!
interface Ethernet2/4
  description NAS: nas-b UCS-C240M4-B-B
  no lldp transmit
  no lldp receive
  switchport access vlan 162
  spanning-tree port type edge
  logging event port link-status
  logging event port trunk-status
  channel-group 224 mode active
!

core-B# sh vpc 223
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
223    Po223       up     success     success                    161         

core-B# sh vpc 224
vPC status
----------------------------------------------------------------------------
id     Port        Status Consistency Reason                     Active vlans
------ ----------- ------ ----------- -------------------------- -----------
224    Po224       up     success     success                    162

CentOS 7 Server – Config: nas-a:

[root@nas-a network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-enp6s0
NM_CONTROLLED=no
DEVICETYPE=TeamPort
MTU=9000
ONBOOT=no
TYPE=Ethernet
TEAM_MASTER=team0
DEVICE=enp6s0
HWADDR=28:AC:9E:E8:60:F0

[root@nas-a network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-enp7s0
NM_CONTROLLED=no
DEVICETYPE=TeamPort
MTU=9000
ONBOOT=no
TYPE=Ethernet
TEAM_MASTER=team0
DEVICE=enp7s0
HWADDR=28:AC:9E:E8:60:F1

[root@nas-a network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-team0
NM_CONTROLLED=no
DEVICE=team0
DEVICETYPE=Team
MTU=9000
BOOTPROTO=none
ONBOOT=yes

# LACP
TEAM_CONFIG='{"runner": {"name": "lacp", "active": true, "fast_rate": true, "tx_hash": ["eth", "ipv4", "ipv6"]},"link_watch":    {"name": "ethtool"},"ports":    {"enp6s0": {}, "enp7s0": {}}}'

# IPv4
IPADDR=192.168.161.231
PREFIX=24
GATEWAY=192.168.161.254
DNS1=192.168.167.244
DNS2=192.168.168.244
DOMAIN=toocoolforyou.net
DEFROUTE=yes

# IPv6
IPV6INIT=yes
IPV6_PRIVACY=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6ADDR=2001:67c:21b0:4161:192:168:161:231/64
IPV6_DEFAULTGW=2001:67c:21b0:4161:192:168:161:254

# Firewall Zone
ZONE=public

CentOS 7 Server – Config: nas-a:

[root@nas-b network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-enp6s0
NM_CONTROLLED=no
DEVICETYPE=TeamPort
MTU=9000
ONBOOT=no
TYPE=Ethernet
TEAM_MASTER=team0
DEVICE=enp6s0
HWADDR=84:3D:C6:77:62:2A

[root@nas-b network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-enp7s0
NM_CONTROLLED=no
DEVICETYPE=TeamPort
MTU=9000
ONBOOT=no
TYPE=Ethernet
TEAM_MASTER=team0
DEVICE=enp7s0
HWADDR=84:3D:C6:77:62:2B

[root@nas-b network-scripts]# cat /etc/sysconfig/network-scripts/ifcfg-team0
NM_CONTROLLED=no
DEVICE=team0
DEVICETYPE=Team
MTU=9000
BOOTPROTO=none
ONBOOT=yes

# LACP
TEAM_CONFIG='{"runner": {"name": "lacp", "active": true, "fast_rate": true, "tx_hash": ["eth", "ipv4", "ipv6"]},"link_watch":    {"name": "ethtool"},"ports":    {"enp6s0": {}, "enp7s0": {}}}'

# IPv4
IPADDR=192.168.162.231
PREFIX=24
GATEWAY=192.168.162.254
DNS1=192.168.167.244
DNS2=192.168.168.244
DOMAIN=toocoolforyou.net
DEFROUTE=yes

# IPv6
IPV6INIT=yes
IPV6_PRIVACY=no
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6ADDR=2001:67c:21b0:8162:192:168:162:231/64
IPV6_DEFAULTGW=2001:67c:21b0:8162:192:168:162:254

# Firewall Zone
ZONE=public

CentOS 7 Server – Show Commands:

[root@nas-a /]# teamdctl team0 state view -v

setup:
  runner: lacp
  kernel team mode: loadbalance
  D-BUS enabled: yes
  ZeroMQ enabled: no
  debug level: 0
  daemonized: no
  PID: 5029
  PID file: /var/run/teamd/team0.pid
ports:
  enp6s0
    ifindex: 2
    addr: 28:ac:9e:e8:60:f0
    ethtool link: 40000mbit/fullduplex/up
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0
        link up delay: 0
        link down delay: 0
    runner:
      aggregator ID: 3, Selected
      selected: yes
      state: current
      key: 0
      priority: 255
      actor LACPDU info:
        system priority: 65535
        system: 28:ac:9e:e8:60:f0
        key: 0
        port_priority: 255
        port: 2
        state: 0x3f
      partner LACPDU info:
        system priority: 32667
        system: 00:23:04:ee:be:a0
        key: 32991
        port_priority: 32768
        port: 16899
        state: 0x3d
  enp7s0
    ifindex: 3
    addr: 28:ac:9e:e8:60:f0
    ethtool link: 40000mbit/fullduplex/up
    link watches:
      link summary: up
      instance[link_watch_0]:
        name: ethtool
        link: up
        down count: 0
        link up delay: 0
        link down delay: 0
    runner:
      aggregator ID: 3, Selected
      selected: yes
      state: current
      key: 0
      priority: 255
      actor LACPDU info:
        system priority: 65535
        system: 28:ac:9e:e8:60:f0
        key: 0
        port_priority: 255
        port: 3
        state: 0x3f
      partner LACPDU info:
        system priority: 32667
        system: 00:23:04:ee:be:a0
        key: 32991
        port_priority: 32768
        port: 515
        state: 0x3d
runner:
  active: yes
  fast rate: yes
  system priority: 65535

[root@nas-a /]# teamdctl team0 config dump -v
{
    "device": "team0",
    "link_watch": {
        "name": "ethtool"
    },
    "ports": {
        "enp6s0": {},
        "enp7s0": {}
    },
    "runner": {
        "active": true,
        "fast_rate": true,
        "name": "lacp",
        "tx_hash": [
            "eth",
            "ipv4",
            "ipv6"
        ]
    }
}

[root@nas-a /]# ls -la /sys/class/net/  
lrwxrwxrwx.  1 root root 0 Sep  8 12:01 enp6s0 -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:01.0/0000:04:00.0/0000:05:00.0/0000:06:00.0/net/enp6s0
lrwxrwxrwx.  1 root root 0 Sep  8 12:01 enp7s0 -> ../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:01.0/0000:04:00.0/0000:05:01.0/0000:07:00.0/net/enp7s0
lrwxrwxrwx.  1 root root 0 Sep  8 12:01 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx.  1 root root 0 Sep  8 12:01 team0 -> ../../devices/virtual/net/team0

LACP – F1-CONSULT

CentOS – NIC Teaming with LACP

A Comparison of Features in Bonding and Team

Config Snippets:

NEXUS vPC Switch A:

NEXUS vPC Switch B:

CentOS 7 Server – Config: nas-a:

CentOS 7 Server – Config: nas-a:

CentOS 7 Server – Show Commands: