test result of D40185
Contents
Testbeds (1Gbps link and 10Gbps link) used here are from Emulab.net.
1Gbps test
On a 1Gbps link between a FreeBSD-14-current node and a Ubuntu20.04 (Linux) node.
FreeBSD node is the traffic sender (s1) using iperf, and the Linux node (r1) is the traffic receiver.
root@s1:~ # ping -c 5 r1 PING r1-link1 (10.1.1.3): 56 data bytes 64 bytes from 10.1.1.3: icmp_seq=0 ttl=64 time=0.178 ms 64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=0.221 ms 64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=0.153 ms 64 bytes from 10.1.1.3: icmp_seq=3 ttl=64 time=0.158 ms 64 bytes from 10.1.1.3: icmp_seq=4 ttl=64 time=0.155 ms --- r1-link1 ping statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.153/0.173/0.221/0.025 ms
Set packet drop in the Linux receiver to simulate high loss rate.
cc@r1:~ % sudo iptables -A INPUT -p tcp --dport 5201 -m statistic --mode nth --every 100 --packet 0 -j DROP
before change:
root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C newreno Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 20972 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 99.4 MBytes 834 Mbits/sec 704 29.7 KBytes [ 5] 1.00-2.00 sec 96.6 MBytes 811 Mbits/sec 665 46.8 KBytes [ 5] 2.00-3.00 sec 100 MBytes 840 Mbits/sec 839 41.1 KBytes [ 5] 3.00-4.00 sec 96.2 MBytes 807 Mbits/sec 718 29.7 KBytes [ 5] 4.00-5.00 sec 97.2 MBytes 815 Mbits/sec 663 35.4 KBytes [ 5] 5.00-6.00 sec 99.1 MBytes 831 Mbits/sec 790 41.1 KBytes [ 5] 6.00-7.00 sec 97.4 MBytes 817 Mbits/sec 691 39.7 KBytes [ 5] 7.00-8.00 sec 97.4 MBytes 817 Mbits/sec 707 29.7 KBytes [ 5] 8.00-9.00 sec 96.7 MBytes 812 Mbits/sec 707 54.0 KBytes [ 5] 9.00-10.00 sec 96.5 MBytes 809 Mbits/sec 677 31.1 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 977 MBytes 819 Mbits/sec 7161 sender [ 5] 0.00-10.04 sec 976 MBytes 816 Mbits/sec receiver iperf Done. root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C cubic Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 63604 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 66.2 MBytes 555 Mbits/sec 486 27.0 KBytes [ 5] 1.00-2.00 sec 66.5 MBytes 558 Mbits/sec 446 31.0 KBytes [ 5] 2.00-3.00 sec 65.0 MBytes 545 Mbits/sec 481 31.2 KBytes [ 5] 3.00-4.00 sec 66.6 MBytes 558 Mbits/sec 405 21.6 KBytes [ 5] 4.00-5.00 sec 64.3 MBytes 540 Mbits/sec 531 27.7 KBytes [ 5] 5.00-6.00 sec 65.3 MBytes 548 Mbits/sec 462 27.0 KBytes [ 5] 6.00-7.00 sec 65.7 MBytes 551 Mbits/sec 481 29.3 KBytes [ 5] 7.00-8.00 sec 65.0 MBytes 546 Mbits/sec 453 25.5 KBytes [ 5] 8.00-9.00 sec 65.4 MBytes 548 Mbits/sec 486 26.0 KBytes [ 5] 9.00-10.00 sec 66.0 MBytes 554 Mbits/sec 413 32.8 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 656 MBytes 550 Mbits/sec 4644 sender [ 5] 0.00-10.04 sec 656 MBytes 548 Mbits/sec receiver iperf Done.
Compared with newreno, existing cubic has about -33% (lower) performance.
after change:
root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C cubic Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 39818 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 91.5 MBytes 768 Mbits/sec 545 35.1 KBytes [ 5] 1.00-2.00 sec 92.1 MBytes 772 Mbits/sec 583 32.4 KBytes [ 5] 2.00-3.00 sec 92.7 MBytes 777 Mbits/sec 548 42.9 KBytes [ 5] 3.00-4.00 sec 94.8 MBytes 795 Mbits/sec 641 35.7 KBytes [ 5] 4.00-5.00 sec 91.6 MBytes 768 Mbits/sec 526 29.9 KBytes [ 5] 5.00-6.00 sec 94.9 MBytes 796 Mbits/sec 663 39.5 KBytes [ 5] 6.00-7.00 sec 95.2 MBytes 799 Mbits/sec 633 39.5 KBytes [ 5] 7.00-8.00 sec 91.0 MBytes 763 Mbits/sec 588 41.4 KBytes [ 5] 8.00-9.00 sec 92.0 MBytes 771 Mbits/sec 561 39.3 KBytes [ 5] 9.00-10.00 sec 94.7 MBytes 795 Mbits/sec 565 35.8 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 930 MBytes 781 Mbits/sec 5853 sender [ 5] 0.00-10.04 sec 930 MBytes 777 Mbits/sec receiver iperf Done.
Compared with cubic before change, this change improves +42% performance.
Compared with newreno, this change on cubic has around -5% (lower) performance.
10Gbps test
On a 10Gbps link between a FreeBSD-14-current node and a Ubuntu20.04 (Linux) node.
FreeBSD node is the traffic sender (s1) using iperf, and the Linux node (r1) is the traffic receiver.
root@s1:~ # ping -c 5 r1 PING r1-link1 (10.1.1.3): 56 data bytes 64 bytes from 10.1.1.3: icmp_seq=0 ttl=64 time=0.147 ms 64 bytes from 10.1.1.3: icmp_seq=1 ttl=64 time=0.139 ms 64 bytes from 10.1.1.3: icmp_seq=2 ttl=64 time=0.128 ms 64 bytes from 10.1.1.3: icmp_seq=3 ttl=64 time=0.125 ms 64 bytes from 10.1.1.3: icmp_seq=4 ttl=64 time=0.129 ms --- r1-link1 ping statistics --- 5 packets transmitted, 5 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.125/0.134/0.147/0.008 ms
before change:
root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C newreno Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 15248 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1009 MBytes 8.46 Gbits/sec 206 388 KBytes [ 5] 1.00-2.00 sec 1.04 GBytes 8.91 Gbits/sec 159 290 KBytes [ 5] 2.00-3.00 sec 1018 MBytes 8.54 Gbits/sec 182 433 KBytes [ 5] 3.00-4.00 sec 1.04 GBytes 8.96 Gbits/sec 95 402 KBytes [ 5] 4.00-5.00 sec 1.05 GBytes 9.02 Gbits/sec 85 369 KBytes [ 5] 5.00-6.00 sec 1.03 GBytes 8.84 Gbits/sec 156 435 KBytes [ 5] 6.00-7.00 sec 1022 MBytes 8.58 Gbits/sec 208 176 KBytes [ 5] 7.00-8.00 sec 1.03 GBytes 8.81 Gbits/sec 119 125 KBytes [ 5] 8.00-9.00 sec 955 MBytes 8.01 Gbits/sec 196 375 KBytes [ 5] 9.00-10.00 sec 1019 MBytes 8.55 Gbits/sec 210 534 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 10.1 GBytes 8.67 Gbits/sec 1616 sender [ 5] 0.00-10.04 sec 10.1 GBytes 8.63 Gbits/sec receiver iperf Done. root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C cubic Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 56663 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 759 MBytes 6.36 Gbits/sec 52 236 KBytes [ 5] 1.00-2.00 sec 603 MBytes 5.06 Gbits/sec 51 187 KBytes [ 5] 2.00-3.00 sec 688 MBytes 5.78 Gbits/sec 45 159 KBytes [ 5] 3.00-4.00 sec 660 MBytes 5.53 Gbits/sec 61 161 KBytes [ 5] 4.00-5.00 sec 690 MBytes 5.78 Gbits/sec 46 170 KBytes [ 5] 5.00-6.00 sec 643 MBytes 5.40 Gbits/sec 58 180 KBytes [ 5] 6.00-7.00 sec 633 MBytes 5.31 Gbits/sec 86 151 KBytes [ 5] 7.00-8.00 sec 719 MBytes 6.03 Gbits/sec 48 141 KBytes [ 5] 8.00-9.00 sec 575 MBytes 4.82 Gbits/sec 45 158 KBytes [ 5] 9.00-10.00 sec 610 MBytes 5.12 Gbits/sec 55 145 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.42 GBytes 5.52 Gbits/sec 547 sender [ 5] 0.00-10.15 sec 6.42 GBytes 5.44 Gbits/sec receiver iperf Done.
Compared with newreno, existing cubic has about -36% (lower) performance.
after change:
root@s1:~ # iperf3 -c r1 -t 10 -i 1 -C cubic Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 33312 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 908 MBytes 7.61 Gbits/sec 67 367 KBytes [ 5] 1.00-2.00 sec 1.04 GBytes 8.94 Gbits/sec 80 219 KBytes [ 5] 2.00-3.00 sec 822 MBytes 6.89 Gbits/sec 105 180 KBytes [ 5] 3.00-4.00 sec 972 MBytes 8.15 Gbits/sec 153 213 KBytes [ 5] 4.00-5.00 sec 1.02 GBytes 8.78 Gbits/sec 119 538 KBytes [ 5] 5.00-6.00 sec 879 MBytes 7.37 Gbits/sec 155 127 KBytes [ 5] 6.00-7.00 sec 772 MBytes 6.48 Gbits/sec 143 154 KBytes [ 5] 7.00-8.00 sec 799 MBytes 6.70 Gbits/sec 100 197 KBytes [ 5] 8.00-9.00 sec 855 MBytes 7.17 Gbits/sec 141 199 KBytes [ 5] 9.00-10.00 sec 895 MBytes 7.51 Gbits/sec 119 311 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 8.80 GBytes 7.56 Gbits/sec 1182 sender [ 5] 0.00-10.04 sec 8.80 GBytes 7.53 Gbits/sec receiver iperf Done.
Compared with cubic before change, this change improves +37% performance.
Compared with newreno, this change on cubic has around -13% (lower) performance.