2022-03-14 / Bartłomiej Kurek
LXC: Host shared bridge, LAN, DHCP

You might want your LXC containers to be accessible on your LAN, just like any other systems. In this article we're going through the steps to setup LXC networking using the Host Shared Bridge and DHCP controlled by the LAN router. A very good guide can be found at wiki.debian.org.

Host setup (br0)

The main host has a bridge interface called br0 with the default ethernet interface (eth0) attached to it.
The physical ethernet device is set to manual and the bridge is set up to obtain the lease via DHCP.

File /etc/network/interfaces:

allow-hotplug eth0
iface eth0 inet manual

auto br0
iface br0 inet dhcp
   bridge_ports eth0
   bridge_fd 0
   bridge_maxwait 0

LXC setup: lxc-net, lxc-usernet

If not using a dedicated LXC bridge, the specific setting may also be changed in /etc/default/lxc-net:

USE_LXC_BRIDGE="false"

In case of unprivileged containers, the user needs to be allowed to created and attach veth interfaces to br0.
File /etc/lxc/lxc-usernet:

lxcuser veth br0 10

If all of the containers will follow the same scheme, dnsmasq on the main host may be disabled.

systemctl disable dnsmasq

Containers

The containers' network interfaces will be attached to the br0 bridge.
The fragment of my lxc0a container configuration file (.local/share/lxc/lxc0a/config):

# Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br0
lxc.net.0.flags = up
lxc.net.0.hwaddr = 00:00:00:00:00:0A

I called the container lxc0a and I manually set the hardware address to 00:00:00:00:00:0a. I use these in the DHCP server configuration in my LAN router.

Once you start the container, its interface should be attached to a bridge:

# brctl show br0
bridge name     bridge id               STP enabled     interfaces
br0             8000.f0def1a1679f       no              eth0
                                                        veth1004_6GST

LAN Router

If needed, configure your DHCP server. I'm using OpenWRT and opt to configure static leases for most of my hosts.
img-border

DHCP, Firewall

If the main host's default firewall policy is DROP, then DHCP client within the containers might not be working right away. If the container does not get assigned an address, it's time to debug things. I'd suggest using tcpdump (on both - main host and the router (if possible)).

Debugging with tcpdump

tcpdump -v -i br0

This will allow you to trace the packets (DHCP request, DHCP offer and ACK). It looks simiar to this:

2:56:18.194075 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:00:00:00:00:0a (oui Ethernet), length 300, xid 0x6d189230, Flags [none]
          Client-Ethernet-Address 00:00:00:00:00:0a (oui Ethernet)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message (53), length 1: Discover
            Requested-IP (50), length 4: lxc0a.lan
            Hostname (12), length 5: "lxc0a"
            Parameter-Request (55), length 13:
              Subnet-Mask (1), BR (28), Time-Zone (2), Default-Gateway (3)
              Domain-Name (15), Domain-Name-Server (6), Unknown (119), Hostname (12)
              Netbios-Name-Server (44), Netbios-Scope (47), MTU (26), Classless-Static-Route (121)
              NTP (42)

Starting DHCP client

For debugging it is best to stop the dhcp client in the container and start it manually.
Stop:

killall dhclient

Run manually:

/sbin/dhclient -v

This will let you see what is happening and where things go wrong.

Firewall rules (iptables)

You need to forward packets on br0:

-A FORWARD -i br0 -j ACCEPT

The main host might need a rule allowing outgoing DHCP traffic:

-A OUTPUT -p udp --dport 67 -j ACCEPT

Depending on the scenario - you might also need to explicitely allow DHCP ports.

-I INPUT -i br0 -p udp --dport 67:68 --sport 67:68 -j ACCEPT

If using netfilter-persistent - it's best to flush all rules before restarting netfilter.
Flushing the rules can be achived by a simple script:

#!/bin/sh

iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -t raw -F
iptables -t raw -X

I use it this way:

# ./iptables-flush-all; /etc/init.d/netfilter-persistent restart
Restarting netfilter-persistent (via systemctl): netfilter-persistent.service.

It is also useful for testing the new firewall rules and avoiding locking yourself out of the remote machine. Just switch the command order, and use a simple sleep in between, like so:

# /etc/init.d/netfilter-persistent restart ; sleep 60; ./iptables-flush-all

DNS

If your router handles DHCP and DNS - the rest of you machines will be able to resolve the container address by name.

(laptop) $ host lxc0a
lxc0a has address 10.0.3.100

(laptop) $ host lxc0b
lxc0b has address 10.0.3.101