bonding mode 4 测试

Myth丶恋晨 2022-06-10 08:09 218阅读 0赞

# mode 4 定义 #

IEEE 802.3ad Dynamic link aggregation.  Creates
        aggregation groups that share the same speed and
        duplex settings.  Utilizes all slaves in the active
        aggregator according to the 802.3ad specification.
    
        Slave selection for outgoing traffic is done according
        to the transmit hash policy, which may be changed from
        the default simple XOR policy via the xmit_hash_policy
        option, documented below.  Note that not all transmit
        policies may be 802.3ad compliant, particularly in
        regards to the packet mis-ordering requirements of
        section 43.2.4 of the 802.3ad standard.  Differing
        peer implementations will have varying tolerances for
        noncompliance.
    
        Prerequisites:
    
        1. Ethtool support in the base drivers for retrieving
        the speed and duplex of each slave.
    
        2. A switch that supports IEEE 802.3ad Dynamic link
        aggregation.
    
        Most switches will require some type of configuration
        to enable 802.3ad mode.

# 配置方法 #

eth0

DEVICE=eth0
    IPV6INIT=no
    MTU=1500
    NM_CONTROLLED=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    USERCTL=no
    BOOTPROTO=none

eth1

DEVICE=eth1
    IPV6INIT=no
    MTU=1500
    NM_CONTROLLED=no
    ONBOOT=yes
    MASTER=bond0
    SLAVE=yes
    USERCTL=no
    BOOTPROTO=none

bond0 \[注, rhel6, rhel7 中, bonding 模式需要在该配置文件中进行指定\]  
由于当前用于 docker container , 因此用到 openvswitch 作为容器之间的网络转换

DEVICE=bond0
    BOOTPROTO=none
    ONBOOT=yes
    BONDING_MASTER=yes
    BONDING_OPTS="miimon=100 mode=4 lacp_rate=fast xmit_hash_policy=layer2+3"
    TYPE=OVSPort
    DEVICETYPE=ovs
    OVS_BRIDGE=vlanbr0

bond0.vlan

DEVICE=bond0.vlan
    ONBOOT=yes
    TYPE=Ethernet
    BOOTPROTO=none
    IPADDR=x.x.x.x
    NETMASK=255.255.255.0
    GATEWAY=x.x.x.x
    VLAN=yes

/etc/modprobe.d/bond.conf \[切记配置改文件, 否则会有问题\]

alias bond0 bonding

# 测试 #

iperf  server :
        [容器]  x.x.x.95
        容器 (4b085ae77152)   [宿主机位置, x.x.x.192 ]
    
    iperf client: 
        [物理机]
        x.x.x.182   x.x.x.183  x.x.x.184  x.x.x.185

## 测试结果: ##

当前网络通道有 eth0,  eth1 (1000M)
    多个主机并发访问,  可以同时利用 eth0,  eth1 进行网络通讯
    假设在多个主机并发访问容器时候,   可能会出现,   其中一个主机独享eth0,  而其他主机在共享 eth1,  
    mode 4 没有负载均衡功能,   由接收方进行 hash 确定数据由那个通道进行通讯

### 测试过程 ###

#### 启动 iperf server: ####

[root@xxxx ~]# docker exec 4b085ae77152 /usr/bin/iperf -s -p 2345

#### 连接 iperf server: ####

iperf  -c x.x.x.95 -p 2345

#### 单主机连接 ####

[  4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28680
    [ ID] Interval       Transfer     Bandwidth
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 28830
    [  5]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec

两次独立测试, 连接带宽最大 937Mbits/sec, 只使用到一个\[随机\]网卡流量

#### 两台主机连接容器测试, ####

[  4] local 10.197.244.95 port 2345 connected with x.x.x.182 port 28928
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45644
    [  4]  0.0-10.0 sec   979 MBytes   818 Mbits/sec
    [  5]  0.0-10.0 sec   288 MBytes   241 Mbits/sec              <- 共 1000M
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 29070
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45762
    [  6]  0.0-10.0 sec   935 MBytes   781 Mbits/sec
    [  4]  0.0-10.0 sec   367 MBytes   307 Mbits/sec              <- 共 1000M
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4054
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45854
    [  5]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec              <- 共 2000M
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.184 port 4524
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 45914
    [  6]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec              <- 共 2000M
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46148
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6406
    [  5]  0.0-10.0 sec   968 MBytes   808 Mbits/sec
    [  4]  0.0-10.0 sec   444 MBytes   372 Mbits/sec              <- 共 1000M
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 46210
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 6878
    [  6]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec              <- 共 2000M

连接时候, iperf client 会随机选择通道, 因此, 双机并发连接时候, 流量为 1000 ~ 2000 Mibits/sec

三台主机同时连接

[  5] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26094
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24020
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48464
    [  5]  0.0-10.0 sec   949 MBytes   793 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                      <- 独享 1000M
    [  6]  0.0-10.0 sec   384 MBytes   321 Mbits/sec
    [  7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 26648
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 24928
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48596
    [  7]  0.0-10.0 sec   941 MBytes   786 Mbits/sec
    [  4]  0.0-10.0 sec   246 MBytes   206 Mbits/sec
    [  5]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                      <- 独享 1000M
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27116
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 25706
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48692
    [  6]  0.0-10.0 sec   960 MBytes   802 Mbits/sec
    [  4]  0.0-10.0 sec   215 MBytes   180 Mbits/sec
    [  5]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                     <- 独享 1000M
    [  7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 27606
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 26558
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 48810
    [  7]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                     <- 独享 1000M
    [  4]  0.0-10.0 sec   942 MBytes   786 Mbits/sec
    [  5]  0.0-10.0 sec   239 MBytes   200 Mbits/sec
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.186 port 4406
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 28322
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.182 port 33646
    [  6]  0.0-10.0 sec   979 MBytes   819 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                   <- 独享 1000M
    [  5]  0.0-10.0 sec   328 MBytes   275 Mbits/sec

三台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec  
\[ 明显地, 在共享同一网络通道的两个 CLIENT 的流量并不平均 \]

四台主机测试

[  7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 29964
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 30564
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49332
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34068
    [  7]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                       <- 独享 1000M
    [  4]  0.0-10.1 sec   563 MBytes   469 Mbits/sec
    [  5]  0.0-10.0 sec   194 MBytes   163 Mbits/sec
    [  6]  0.0-10.0 sec   482 MBytes   404 Mbits/sec
    [  8] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30360
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31230
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49422
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34194
    [  8]  0.0-10.1 sec   709 MBytes   591 Mbits/sec
    [  4]  0.0-10.0 sec   206 MBytes   173 Mbits/sec
    [  5]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                       <- 独享 1000M
    [  6]  0.0-10.0 sec   377 MBytes   316 Mbits/sec
    [  7] local 10.197.244.95 port 2345 connected with x.x.x.185 port 30682
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.184 port 31774
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49500
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34280
    [  7]  0.0-10.1 sec   568 MBytes   472 Mbits/sec
    [  4]  0.0-10.0 sec   157 MBytes   131 Mbits/sec
    [  5]  0.0-10.0 sec   518 MBytes   434 Mbits/sec
    [  6]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                       <- 独享 1000M
    [  8] local 10.197.244.95 port 2345 connected with x.x.x.182 port 34408
    [  4] local 10.197.244.95 port 2345 connected with x.x.x.183 port 49794
    [  5] local 10.197.244.95 port 2345 connected with x.x.x.184 port 32418
    [  6] local 10.197.244.95 port 2345 connected with x.x.x.185 port 31080
    [  8]  0.0-10.1 sec   622 MBytes   518 Mbits/sec
    [  4]  0.0-10.0 sec  1.09 GBytes   937 Mbits/sec                       <- 独享 1000M
    [  5]  0.0-10.0 sec   173 MBytes   144 Mbits/sec
    [  6]  0.0-10.0 sec   491 MBytes   411 Mbits/sec

四台主机同时连接, 流量为 1000 ~ 2000 Mibits/sec  
\[ 明显地, 三台主机在同一通道, 而另外一个主机独享流量\] 即, 没有负载均衡功能