kotfu.net

Redundant OpenBSD Firewalls

Part 3 - Network Services

We will now add some additional services to our single firewall. These are basic configurations which work as-is, but you will almost certainly need to modify and expand them to meet your needs.

Time server

I prefer the reference NTP implementation, but it can only send packets to other time servers from port 123. This won’t work for our use case. If we configure the packet filter on fw1 to accept all UDP traffic on port 123, then the time server on fw2 won’t work.

OpenBSD comes with OpenNTPD, which isn’t as full featured, but it does send outbound UDP packets on high ports. This allows the packet filter to track state on those outbound packets, and send the incoming ones back to the originator, making it possible for us to run two time servers.

Fortunately the NTP protocol design makes it super easy to have redundant servers. We need zero additional software support or configuration to make it happen. We need a NTP server listening on 192.168.13.4.

Make /etc/ntpd.conf look something like this:

#
# configuration file for ntpd

# serve time to our internal network
listen on 127.0.0.1
listen on 192.168.13.4

# time servers
server 0.us.pool.ntp.org
server 1.us.pool.ntp.org
server 2.us.pool.ntp.org
server 3.us.pool.ntp.org

# TLS constraint to mitigate MITM NTP attacks
constraint from "https://9.9.9.9" "2620:fe::9"

And start it up:

fw1# rcctl enable ntpd
fw1# rcctl start ntpd

It may take a few minutes for the daemon to contact all the servers and sync the clock. To check the status of the time service:

fw1$ ntpctl -s all
4/4 peers valid, constraint offset -1s, clock synced, stratum 4

peer
   wt tl st  next  poll          offset       delay      jitter
162.159.200.123 from pool pool.ntp.org
 *  1 10  3 1292s 1529s        -0.217ms    14.496ms     6.526ms
162.159.200.1 from pool pool.ntp.org
    1 10  3 1507s 1525s        -1.301ms    12.377ms     2.488ms
173.0.48.220 from pool pool.ntp.org
    1 10  2 1414s 1503s        22.369ms    46.584ms     4.257ms
23.31.21.164 from pool pool.ntp.org
    1 10  2 1576s 1587s        16.927ms   186.044ms   116.632ms

If you see “clock synced”, then you are in good shape.

Authoritative name server for internal network

I have some internal hosts that I want to use friendly host names for, but I don’t want to publish these hosts in a public name server. So I need an internal only name server, which my internal caching resolver is aware of. The internal authoritative name server only needs to be queried by our internal caching resolver.

OpenBSD uses nsd as an authoritative name server. I won’t go into full details of how to configure a name server and zone files. For this exercise, the key configuration detail is to only listen on localhost port 5353.

Let’s set up nsd:

fw1# nsd-control-setup
fw1# rcctl enable nsd

Then install a basic configuration file in /var/nsd/etc/nsd.conf:

#
# /var/nsd/etc/nsd.conf - configuration for authoritative name server

server:
    hide-version: yes
    verbosity: 1
    database: "" # disable database

    # only listen on localhost
    # the only resolver we care that can access this service
    # is the unbound server running on this same box
    interface: 127.0.0.1@5353
    interface: ::1@5353

remote-control:
    control-enable: yes
    control-interface: 127.0.0.1
    control-port: 8952
    server-key-file:   /var/nsd/etc/nsd_server.key
    server-cert-file:  /var/nsd/etc/nsd_server.pem
    control-key-file:  /var/nsd/etc/nsd_control.key
    control-cert-file: /var/nsd/etc/nsd_control.pem

zone:
    name: kotfu.net
    zonefile: kotfu.net

zone:
    name: 13.168.192.in-addr.arpa
    zonefile: 192.168.13.0

That’s it. Start it up and we are done:

fw1# rcctl start nsd

Caching DNS resolver

In theory, we only need the resolver to run on one IP address. In practice, I have seen stupid network configuration user interfaces on wireless routers that require two name server IP addresses. I have also had cases where I needed to move the resolver to a different piece of hardware. If the resolvers have their own IP addresses, the service can be moved without having to have every DHCP client renew their lease. Finally, most clients that query a resolver will wait a few seconds for it to time out before they try the second resolver in their list. We want to avoid that timeout.

Therefore, we will create a new CARP network interface, and assign it two IP address to be used only for name services. When we add our redundant hardware, we’ll just add it to the CARP group for these IP addresses, and be good to go.

We already have carp0, so let’s create carp1 with two IP addresses.

Put the following into /etc/hostname.carp1:

inet 192.168.13.2 255.255.255.0 192.168.13.255 vhid 132 carpdev em0 pass vhid132passwd description "name servers"
inet alias 192.168.13.3 255.255.255.0

Bring up the interface:

fw1# sh /etc/netstart carp1

Configure unbound and get curl (to be used shortly):

fw1# rcctl enable unbound
fw1# unbound-control-setup
fw1# pkg-add -vv curl

Use curl to get a list of root name servers:

fw1# curl https://www.internic.net/domain/named.root > /var/unbound/etc/root.hints

Create the unbound configuration file /var/unbound/etc/unbound.conf:

#
# /var/unbound/etc/unbound.conf - caching resolver configuration

server:
    interface: 127.0.0.1
    interface: 192.168.13.2
    interface: 192.168.13.3
    interface: 192.168.13.4
    interface: ::1

    access-control: 0.0.0.0/0 refuse
    access-control: 127.0.0.0/8 allow
    access-control: 192.168.13.0/24 allow
    access-control: ::0/0 refuse
    access-control: ::1 allow

    # by default, unbound won't query localhost, let's change that
    do-not-query-localhost: no

    hide-identity: yes
    hide-version: yes

    # root hints
    # curl https://www.internic.net/domain/named.root > /var/unbound/etc/root.hints
    root-hints: /var/unbound/etc/root.hints

    # enable dnssec by specifying a root key file
    auto-trust-anchor-file: /var/unbound/db/root.key
        
	# allow RFC1918 addresses to be returned
	local-zone: 13.168.192.in-addr.arpa transparent

	# don't use dnssec for locally hosted zones
	domain-insecure: kotfu.net
	domain-insecure: 13.168.192.in-addr.arpa

# enable remote control, but only on localhost
remote-control:
    control-enable: yes
    control-interface: 127.0.0.1
    control-port: 8953
    server-key-file:   /var/unbound/etc/unbound_server.key
    server-cert-file:  /var/unbound/etc/unbound_server.pem
    control-key-file:  /var/unbound/etc/unbound_control.key
    control-cert-file: /var/unbound/etc/unbound_control.pem

# stub out our internal zones
stub-zone:
    name: kotfu.net
    stub-addr: 127.0.0.1@5353

stub-zone:
    name: 13.168.192.in-addr.arpa
    stub-addr: 127.0.0.1@5353

Notice that the stub zones are queried from localhost port 5353, which is where we set up nsd to listen. Start it up and test it out:

fw1# rcctl start unbound
fw1# dig @localhost eff.org

Notice that we are also listening on 192.168.13.4. That’s intentional, and we’ll need it later.

DHCP Server

There are two ways to have redundant DHCP servers. One method requires no software support in the servers for any type of failover. Assign two blocks of dynamic IP addresses, configure the first server to hand out addresses in the first block, and configure the second server to hand out addresses in the second block.

The second approach is to use a DHCP server that knows how to sync leases between two instances. You could install Kea or ISC DHCP from the OpenBSD packages repository; both of these DHCP servers support lease synchonization.

You can also use the DHCP server that ships stock with OpenBSD, which also supports lease synchronization. I haven’t used Kea, but the ISC DHCP server requires the synchronization information to be provided in the /etc/dhcpd.conf file, which often requires you to split the configuration information into two files, one that’s the same between your failover servers, and one that’s not. The OpenBSD stock DHCP server uses the command line for the synchronization parameters, simplifying the configuration file and making it the same between the failover servers.

We will use the DHCP server installed by default with OpenBSD. First enable the server:

fw1# rcctl enable dhcpd

Create a configuration file in /etc/dhcpd.conf:

#
# /etc/dhcpd.conf

authoritative;

# set our options
option domain-name "kotfu.net";
option domain-name-servers 192.168.13.2,192.168.13.3;
option ntp-servers 192.168.13.4;
default-lease-time 216000; # 1 hr
max-lease-time 648000; # 3 hr

subnet 192.168.13.0 netmask 255.255.255.0 {
  option routers 192.168.13.1;
  option broadcast-address 192.168.13.255;
  range 192.168.13.32 192.168.13.127;
}

Notice we have used the two IP addresses assigned to our resolver, as well as the NTP server. Your DHCP configuration will likely be more involved than this one (mine is), but it is sufficient for our current example. Finally, we start up the DHCP service:

fw1# rcctl start dhcpd

Test it out with any device on your network to make sure it can dynamically acquire a working network configuration.

Part 2 is now complete. We have a single server which provides:

  • routing
  • packet filtering
  • authoritative name server
  • caching DNS resolver
  • NTP server
  • DHCP server

Along the way, we have done some preparatory work for Part 3, where we add in our second piece of firewall hardware and configure it for redundant routing and packet filtering.

Redundant OpenBSD Firewall Guide