OpenVSwitch, VLANs, and Addresses - PVE Networking Part 5

Why OpenVSwitch

The integrated Linux Bridge module for PVE works fine in most cases, but for where it doesn’t there’s OpenVSwitch. I’ve always had issues with OVS, for reasons that don’t make a lot of sense to me, but I recently had an issue where I needed to switch over to OVS, since the standard Linux Bridge just couldn’t cut it anymore.

I use the PVE SDN module, and it can cause some conflicts with the way VLANs are handled by the bridging module, specifically adding IPs to VLANs. In short, you cannot have multiple VLAN interfaces connecting to the same VLAN, as it creates a conflicting interface on the bridge. This is where OVS saves the day.

The reason for this is that instead of configuring an interface, OVS functions like plugging something into a switch, except it’s virtual. OVS doesn’t care that multiple things are connecting to the same VLAN.

Network Layout

Now, I want the VMs on this host to have access to VLANs 1 through 10, but I also need the host to have IPs on VLANs 6 and 10. Using the SDN module means I’m not connecting to a bridge with an added-on VLAN tag, I’m connecting to a bridge that’s already on that VLAN. This makes migration and access control a little easier, especially since it cuts down on the bloat in the interfaces file.

The downside with this being experimental is that, while it works, it doesn’t always play nice with the system. For example, it can create conflicts, and because of the way Linux Bridging works, it can even create duplicates, one configuration with the IP, and another without.

Normally this isn’t a problem, since all my other systems have dedicated management and storage networks. However, I have one system which only has a single interface and zero expandability. If I wanted to keep using the SDN system, I needed another solution.

The great thing about the SDN module is that it seems to not care about the back end, it will create the interfaces automatically so that they work with either Linux Bridging or OVS.

PVE Steps

A few packages are needed to get this to work, both the SDN System and OpenVSwitch

apt update
apt install libpve-network-perl ifupdown2 openvswitch-switch

Once those packages are installed, most of the work can be done through the GUI, where the SDN system and the networking are configured.

/etc/network/interfaces

auto lo
iface lo inet loopback

# Singular Gigabit Ethernet port
auto eno1
iface eno1 inet manual
	ovs_type OVSPort
	ovs_bridge vmbr0

# Storage VLAN
auto vlan6
iface vlan6 inet static
	address 10.0.6.8/24
	ovs_type OVSIntPort
	ovs_bridge vmbr0
	ovs_options tag=60

# Management VLAN
auto vlan10
iface vlan10 inet static
	address 10.0.10.8/24
	gateway 10.0.10.1
	ovs_type OVSIntPort
	ovs_bridge vmbr0
	ovs_options tag=10

# Combined OVS Bridge
auto vmbr0
iface vmbr0 inet manual
	ovs_type OVSBridge
	ovs_ports eno1 vlan6 vlan10

source /etc/network/interfaces.d/*

This file was created using the tool in the web GUI, which I want to mention because OVS is very touchy about order. I tried this several times and the only way it worked reliably was after I used the GUI to make sure the order was right.

After this, both the SDN system and the IPs worked perfectly.

More Complications: Bonding

Once I figured out how this worked, I decided to also do this to a testing server I made from an old desktop. It’s pretty robust, but it also only has two 10G ports, and a single 1G, which means there’s no failover if things go bad.

Previously, I would have made a Linux bond, then a bridge to that bond, but the process is slightly different with OVS.

auto lo
iface lo inet loopback

# Gigabit interface, Management, VLAN 10 Access Port
auto enp39s0
iface enp39s0 inet static
        address 10.0.10.7/24
        gateway 10.0.10.1

# Dual-port 10G Ethernet card
auto enp37s0f0
iface enp37s0f0 inet manual

auto enp37s0f1
iface enp37s0f1 inet manual

# Storage Network
auto vlan6
iface vlan6 inet static
        address 10.0.6.7/24
        ovs_type OVSIntPort
        ovs_bridge vmbr0
        ovs_options tag=6

# Bond of the two 10G ports
auto bond0
iface bond0 inet manual
        ovs_bonds enp37s0f0 enp37s0f1
        ovs_type OVSBond
        ovs_bridge vmbr0
        ovs_options lacp=active bond_mode=balance-slb

# The main OVS bridge
auto vmbr0
iface vmbr0 inet manual
        ovs_type OVSBridge
        ovs_ports bond0 vlan6

source /etc/network/interfaces.d/*