23 年买了一台华为 RH2288 V3 在家里当电老虎,服务器上带一个万兆光,支持 SRIOV,这样我可以分出来多个虚拟网卡来加速网络访问,创建一个给 docker,一个给 windows 虚拟机,一个给 linux 虚拟机等。
虽然系统换来换去,最终又是回到了 PVE,因为最近我想跑个飞牛,挺喜欢他家的影视的。
我写了个脚本,开机自动创建了几个网卡。
[Unit] Description=SRIOV service
[Service] Type=oneshot ExecStartPre=-/usr/bin/bash -c '/usr/bin/echo 10 > /sys/class/net/enp2s0f1/device/sriov_numvfs' ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 0 mac 1e:c9:90:ab:15:0b trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 1 mac d2:1b:36:88:c3:17 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 2 mac 6e:03:f8:19:c2:09 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 3 mac 46:ac:a3:40:ce:46 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 4 mac a6:57:31:92:bc:41 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 5 mac 7e:e7:1b:e9:f5:93 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 6 mac 86:09:25:4f:a1:4e trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 7 mac aa:4c:9c:43:03:dc trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 8 mac 7e:38:28:ee:61:64 trust on spoofchk off ExecStartPre=-/usr/bin/ip link set dev enp2s0f1 vf 9 mac 96:70:81:58:67:6a trust on spoofchk off ExecStartPre=-/usr/bin/ip link set enp2s0f1v0 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v1 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v2 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v3 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v4 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v5 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v6 up ExecStartPre=-/usr/bin/ip link set enp2s0f1v7 up ExecStart=-/usr/bin/ip link set enp2s0f1 up [Install] WantedBy=default.target
使用 ip -c a 可以看到创建了许多开头是 enp2s0f1,以 v + 编号结尾的网卡。
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: enp2s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 70:79:90:a8:19:37 brd ff:ff:ff:ff:ff:ff 3: enp2s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000 link/ether 70:79:90:a8:19:38 brd ff:ff:ff:ff:ff:ff 4: enp2s0f1v0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 1e:c9:90:ab:15:0b brd ff:ff:ff:ff:ff:ff inet6 fd6c:f1fa:b46b:d747:1cc9:90ff:feab:150b/64 scope global dynamic mngtmpaddr valid_lft 1789sec preferred_lft 1789sec inet6 fe80::1cc9:90ff:feab:150b/64 scope link valid_lft forever preferred_lft forever 5: enp2s0f1v1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether d2:1b:36:88:c3:17 brd ff:ff:ff:ff:ff:ff inet6 fd6c:f1fa:b46b:d747:d01b:36ff:fe88:c317/64 scope global dynamic mngtmpaddr valid_lft 1789sec preferred_lft 1789sec inet6 fe80::d01b:36ff:fe88:c317/64 scope link valid_lft forever preferred_lft forever
网卡的初始化已经结束了,剩下的就是 lxc 的配置了。我们可以把一个 vf 网卡分配给 lxc,这样不需要走 vmbr 那个网桥,相当于两个物理网卡互相通信,没有 cpu 参与,效率能提高不少。