diff --git a/doc/howto.md b/doc/howto.md index 563ce18c5..d65c07598 100644 --- a/doc/howto.md +++ b/doc/howto.md @@ -30,6 +30,7 @@ dbus-config netplan-everywhere single-nic-vm-host single-nic-vm-host-with-vlans +multi-nic-vm-host-with-bonds-and-vlans ``` diff --git a/doc/multi-nic-vm-host-with-bonds-and-vlans.md b/doc/multi-nic-vm-host-with-bonds-and-vlans.md new file mode 100644 index 000000000..b486e9475 --- /dev/null +++ b/doc/multi-nic-vm-host-with-bonds-and-vlans.md @@ -0,0 +1,258 @@ +# How to configure a VM host with bonded network interfaces and three VLANs + +This guide shows how to configure a virtual machine (VM) host using Netplan and the `virsh` interface. The host in this scenario has four network interface (NICs). The host uses network bonding and three VLAN networks. + + +## Prerequisites + +Ensure the following prerequisites are satisfied. + + +### System + +- Computer with 4 NICs: + - 1 NIC dedicated to be used in passthrough mode (out of scope of this how to) + - 3 NICs bonded using 802.3ad for the host, VMs, and containers +- Ubuntu Server installed. +- KVM and QEMU installed; see [KVM installation](https://help.ubuntu.com/community/KVM/Installation). +- Administrator privileges. + + +### Networking + +- IPv4: + - VLAN1 untagged (management), IPv4: 192.168.150.0/24 + - VLAN40 tagged (guest), IPv4: 192.168.151.0/24 + - VLAN41 tagged (dmz), IPv4: 192.168.152.0/24 + - DNS1: 1.1.1.1 + - DNS2: 8.8.8.8 +- Switch with [VLAN](https://en.wikipedia.org/wiki/VLAN) and [LACP](https://en.wikipedia.org/wiki/Link_aggregation#Link_Aggregation_Control_Protocol) support +- Router with [VLAN](https://en.wikipedia.org/wiki/VLAN) support + - VLAN1 IPv4: 192.168.150.254/24 + - VLAN40 IPv4: 192.168.151.254/24 + - VLAN41 IPv4: 192.168.152.254/24 + - InterVLAN routing, DNS, and DHCP configured +- Firewall configured; see [UFW](https://help.ubuntu.com/community/UFW). + + +#### Disable netfilter for bridged interfaces + +To allow communication between the host server, its virtual machines, and the devices in the local VLANs, disable netfilter for bridged interfaces: + +1. Add the following lines to the `/etc/systemctl.conf` configuration file: + + ``` + net.bridge.bridge-nf-call-iptables = 0 + net.bridge.bridge-nf-call-ip6tables = 0 + net.bridge.bridge-nf-call-arptables = 0 + ``` + +2. Apply the changes immediately, without rebooting the host. + + ```none + sysctl -p /etc/sysctl.conf + ``` + + +## Netplan configuration + +Configure Netplan: + +- Leave the first NIC unconfigured. +- Disable DHCP on all interfaces. +- Create a 802.3ad bond with three NICs. +- Create two VLANs (40 and 41) under the bond. +- Create three bridge interfaces, and assign IPv4 addresses to them: + - `br0`: bridge on the untagged VLAN1 and the management interface of the server + - `br0-vlan40`: bridge on `vlan40` + - `br0-vlan41`: bridge on `vlan41` +- Configure routes. +- Configure DNS. + +1. To achieve this, modify the Netplan configuration file, `/etc/netplan/00-installer-config.yaml`, as follows: + + ```yaml + # network configuration: + # eno1 - dedicated to virtual firewall WAN + # eno2, eno3, eno4 - bonded interfaces + # bond0 - primary bond for untagged vlan1 + # bond0-vlan40 - vlan interface to connect to tagged vlan40 + # bond0-vlan41 - vlan interface to connect to tagged vlan41 + # br0 - bridge for interface bond0 on untagged vlan1 + # br0-vlan40 - bridge on tagged vlan40 + # br0-vlan41 - bridge on tagged vlan41 + + network: + version: 2 + ethernets: + eno1: + dhcp4: false + eno2: + dhcp4: false + eno3: + dhcp4: false + eno4: + dhcp4: false + bonds: + bond0: + dhcp4: no + interfaces: [eno2, eno3, eno4] + parameters: + mode: 802.3ad + mii-monitor-interval: 1000 + vlans: + bond0-vlan40: + id: 40 + link: bond0 + bond0-vlan41: + id: 41 + link: bond0 + bridges: + br0: + interfaces: [bond0] + dhcp4: false + addresses: [192.168.150.1/24] + routes: + - to: default + via: 192.168.150.254 + metric: 100 + on-link: true + nameservers: + addresses: [1.1.1.1, 8.8.8.8] + search: [] + br0-vlan40: + interfaces: [bond0-vlan40] + dhcp4: false + routes: + - to: 0.0.0.0 + via: 192.168.151.254 + metric: 100 + on-link: true + nameservers: + addresses: [1.1.1.1, 8.8.8.8] + br0-vlan41: + interfaces: [bond0-vlan41] + dhcp4: false + routes: + - to: 0.0.0.0 + via: 192.168.152.254 + metric: 100 + on-link: true + nameservers: + addresses: [1.1.1.1, 8.8.8.8] + ``` + +2. Test the new network settings: + + ```none + netplan try + ``` + +3. Apply the configuration: + + ``` + netplan apply + ``` + + +## Configure virtual networks using `virsh` + +The next step is to configure virtual networks defined for `virsh` domains. This is not necessary, but it makes VM deployment and management easier. + + +### Check networking and delete the default network + +1. Check existing virtual networks: + + ```none + virsh net-list --all + ``` + + There should be one default network as in this example: + + ``` + Name State Autostart Persistent + -------------------------------------------- + default active yes yes + ``` + + If needed, use the `net-info` command to gather more details about the default network: + + ``` + virsh net-info default + ``` + +2. Remove the default network: + + ``` + virsh net-destroy default + virsh net-undefine default + ``` + +3. Check network list to confirm the changes have been applied. There should no networks defined now: + + ```none + virsh net-list --all + ``` + + +### Create bridged networks + +1. Create a directory for VM data. For example: + + ```none + mkdir /mnt/vmstore/ + cd /mnt/vmstore/ + ``` + +2. Define the bridge interface, `br0`, for VLAN1 by creating the `/mnt/vmstore/net-br0.xml` file with the following contents: + + ```xml + + br0 + + + + ``` + +3. Define the bridge interface, `br0-vlan40`, for VLAN40 by creating the `/mnt/vmstore/net-br0-vlan40.xml` file with the following contents: + + ```xml + + br0-vlan40 + + + + ``` + +4. Define the bridge interface, `br0-vlan41`, for VLAN41 by creating the `/mnt/vmstore/net-br0-vlan41.xml` file with the following contents: + + ```xml + + br0-vlan41 + + + + ``` + +5. Enable the virtual (bridged) networks. This consists of three steps (performed for each of the networks): + + 1. Define the network. + 2. Start the network. + 3. Set the network to autostart. + + ``` + virsh net-define net-br0.xml + virsh net-define net-br0-vlan40.xml + virsh net-define net-br0-vlan41.xml + virsh net-start br0 + virsh net-start br0-vlan40 + virsh net-start br0-vlan41 + virsh net-autostart br0 + virsh net-autostart br0-vlan40 + virsh net-autostart br0-vlan41 + ``` + +6. Test the bridged networks. + +Congratulations, the configuration is complete. You can now create a virtual machine, assign the desired network using your preferred VM configuration tool, and run some tests.