How to implement DNS-Over-HTTPS on PiHole, Ubiquiti USG and dnsmasq devices

April 1, 2018

With the release of the Cloudflare consumer DNS service (1.1.1.1) there is now a great option for using DNS-Over-HTTPS (DoH).

This post will provide an overview on how DNS-Over-HTTPS is an improvement over regular DNS, as well as a guide on how to implement it with a range of configurations, such as:

PiHole (and most Linux Distros based on Debian/RHEL/Fedora)
dnsmasq
Ubiquiti Unifi Security Gateway (USG)

Cloudflare 1.1.1.1 Service

It is the 1st of April, 2018. The people have awoken, ready to discover the most cringeworthy April Fools day "jokes" from the usual tech giants. It looks like Cloudflare has decided to join in this year - "Secure, privacy focused, incredibly fast DNS? Who are they trying to fool?"

But alas, it was not a joke.

Cloudflare have released 1.1.1.1, which completely blows away all previous attempts at a global DNS service out of the water.

It is super fast (in my location it is 40x faster than Google's DNS).
It is privacy focused, writing no query data to disk and wiping all logs every 24 hours (Google sells DNS data for the purposes of advertising)
It supports a myriad of DNS options such as DNSSEC, DNS-over-TLS and DNS-Over-HTTPS, all of which are much more secure and reduce the potential for your ISP or other entities to snoop on your data.

What is DNS-Over-HTTPS?

DNS-Over-HTTPS is a protocol for performing DNS lookups via the same protocol you use to browse the web securely: HTTPS. (If you are not aware of what DNS is, please read this primer before continuing).

With regular DNS, requests are sent in plain-text, with no method to detect tampering or misbehaviour. This means that not only can a malicous actor look at all the DNS requests you are making (and therefore what websites you are visiting), they can also tamper with the response and redirect your device to resources in their control (such as a fake login page for internet banking).

DNS-Over-HTTPS prevents this by using standard HTTPS requests to retrieve DNS information. This means that the connection from the device to the DNS server is secure and can not easily be snooped, monitored, tampered with or blocked.

Configuring DNS-Over-HTTPS

As part of releasing 1.1.1.1, Cloudflare implemented DNS-Over-HTTPS proxy functionality in to one of their tools: cloudflared, also known as argo-tunnel.

In the following sections we will be covering how to install and configure this tool on PiHole, Debian/RHEL/Fedora and Ubiquiti USG devices which use dnsmasq forwarding.

PiHole and Linux

NOTE: I have created an Ansible Role and sample playbook that can be used to automate the following steps. Please find links below:

Installing `cloudflared`

The installation is fairly straightforward, however be aware of what architecture you are installing on (amd64 or arm).

AMD64 architecture (most devices)

Download the installer package, then use apt-get to install the package along with any dependencies. Proceed to run the binary with the -v flag to check it is all working.

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-amd64.deb
sudo apt-get install ./cloudflared-stable-linux-amd64.deb
cloudflared -v

ARM architecture (Raspberry Pi)

Here we are downloading the precompiled binary and copying it to the /usr/local/bin/ directory to allow execution by the cloudflared user. Proceed to run the binary with the -v flag to check it is all working.

wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-arm.tgz
tar -xvzf cloudflared-stable-linux-arm.tgz
cp ./cloudflared /usr/local/bin
chmod +x /usr/local/bin/cloudflared
cloudflared -v

Configuring `cloudflared` to run on startup

Create a cloudflared user to run the daemon.

sudo useradd -s /usr/sbin/nologin -r -M cloudflared

It is good practice to have a configuration file to contain options. Proceed to create a configuration file by copying the following in to /etc/default/cloudflared. This contains the command-line options that get passed to cloudflared on startup.

# Commandline args for cloudflared
CLOUDFLARED_OPTS=--port 5053 --upstream https://1.1.1.1/dns-query

Update the permissions for the configuration file and cloudflared binary to allow access for the cloudflared user

sudo chown cloudflared:cloudflared /etc/default/cloudflared
sudo chown cloudflared:cloudflared /usr/local/bin/cloudflared

Then create the systemd script by copying the following in to /lib/systemd/system/cloudflared.service. This will control the running of the service and allow it to run on startup.

[Unit]
Description=cloudflared DNS over HTTPS proxy
After=syslog.target network-online.target

[Service]
Type=simple
User=cloudflared
EnvironmentFile=/etc/default/cloudflared
ExecStart=/usr/local/bin/cloudflared proxy-dns $CLOUDFLARED_OPTS
Restart=on-failure
RestartSec=10
KillMode=process

[Install]
WantedBy=multi-user.target

Enable the systemd service to run on startup, then start the service and check its status.

sudo systemctl enable cloudflared
sudo systemctl start cloudflared
sudo systemctl status cloudflared

Now test that it is working! Run the following dig command, a response should be returned similar to the one below

dig @127.0.0.1 -p 5053 google.com


; <<>> DiG 9.10.3-P4-Ubuntu <<>> @127.0.0.1 -p 5053 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 65181
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1536
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		299	IN	A	243.65.127.221

;; Query time: 3 msec
;; SERVER: 127.0.0.1#5053(127.0.0.1)
;; MSG SIZE  rcvd: 65

Configuring DNS

If you have gotten to this point, you now have a working DNS-over-HTTPS service. But unfortunately, it's only running locally on the device. The next steps will cover how to implement the service for network-wide DNS lookups via PiHole, dnsmasq or direct.

PiHole and dnsmasq

Create a file /etc/dnsmasq.d/50-cloudflared.conf and fill it with the following:

server=127.0.0.1#5053

Look through all other files within the /etc/dnsmasq.d/ directory, and add a # in front of any lines starting with server=, like so

#server=8.8.8.8#53

This step does not need to be completed if you are not using Pihole. PiHole will automatically regenerate the dnsmasq configuration files when reloaded. To prevent this from conflicting with our manually made changes, we can edit the PiHole configuration file and remove all references to DNS servers. To do this, open /etc/pihole/setupVars.conf in an editor and add a # in front of any lines starting with PIHOLE_DNS, like so

#PIHOLE_DNS1=8.8.8.8

After restarting Dnsmasq (and PiHole if applicable), queries should now be fulfilled using the Cloudflare DNS service. This can be verified by visiting the internet.nl DNSSEC test service.

Direct lookups

It is possible, although not reccomended, to use the DNS Proxy directly.

Do this by editing the port in /etc/default/cloudflared and setting it to 53

# Commandline args for cloudflared
CLOUDFLARED_OPTS=--port 53 --upstream https://1.1.1.1/dns-query

Then proceed to restart the service

sudo systemctl restart cloudflared

You may also be required to open this port in the firewall.

A client device such as a laptop or phone can now be configured to use it as the primary DNS server. You can verify it is working correctly by visiting the internet.nl DNSSEC test service.

Ubiquiti USG

The Ubiquiti USG runs on mips architecture. This makes the process of of installing cloudflared more difficult as it needs to be compiled specifically for this architecture.

Compiling and Installing `cloudflared`

Mac or Linux

On a PC with a bash shell (Mac or Linux), install the Go programming language and Go tools per the instructions on golang.org.

Then, retrieve and build the cloudflared binary for the mips architecture.

go get -v github.com/cloudflare/cloudflared/cmd/cloudflared
GOOS=linux GOARCH=mips go build -v -x github.com/cloudflare/cloudflared/cmd/cloudflared

Docker

(Thanks to Apnar in the comments!) If you have docker installed, the binary can be compiled inside a container by running the following command:

docker run --rm -v "$PWD":/usr/src/myapp -w /usr/src/myapp -e GOOS=linux -e GOARCH=mips golang bash -c "go get -v github.com/cloudflare/cloudflared/cmd/cloudflared; GOOS=linux GOARCH=mips go build -v -x github.com/cloudflare/cloudflared/cmd/cloudflared"

Once built, copy the binary over to the USG

scp cloudflared user@USG:~

Proceed to log in to the USG, and copy the binary to /usr/local/bin. Then run the binary with the -v flag to check it is all working.

ssh user@USG
sudo su -
cp ~user/cloudflared /usr/local/bin
chmod +x /usr/local/bin/cloudflared
cloudflared -v

Configuring `cloudflared` to run on startup

# Commandline args for cloudflared
CLOUDFLARED_OPTS="--port 5053 --upstream https://1.1.1.1/dns-query"

Copy the following init script to /etc/init.d/cloudflared. This will control the running of the service and allow it to run on startup.

#!/bin/sh
### BEGIN INIT INFO
# Provides:
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start cloudflared daemon at boot time
# Description:       Enable service provided by cloudflared.
### END INIT INFO

# Read configuration variable file if it is present
[ -r /etc/default/cloudflared ] && . /etc/default/cloudflared

dir=""
cmd="/usr/local/bin/cloudflared"
user=""

name=`basename $0`
pid_file="/var/run/$name.pid"
stdout_log="/var/log/$name.log"
stderr_log="/var/log/$name.err"

get_pid() {
    cat "$pid_file"
}

is_running() {
    [ -f "$pid_file" ] && ps -p `get_pid` > /dev/null 2>&1
}

case "$1" in
    start)
    if is_running; then
        echo "Already started"
    else
        echo "Starting $name"
        cd "$dir"
        if [ -z "$user" ]; then
            sudo $cmd $CLOUDFLARED_OPTS >> "$stdout_log" 2>> "$stderr_log" &
        else
            sudo -u "$user" $cmd $CLOUDFLARED_OPTS >> "$stdout_log" 2>> "$stderr_log" &
        fi
        echo $! > "$pid_file"
        if ! is_running; then
            echo "Unable to start, see $stdout_log and $stderr_log"
            exit 1
        fi
    fi
    ;;
    stop)
    if is_running; then
        echo -n "Stopping $name.."
        kill `get_pid`
        for i in 1 2 3 4 5 6 7 8 9 10
        # for i in `seq 10`
        do
            if ! is_running; then
                break
            fi

            echo -n "."
            sleep 1
        done
        echo

        if is_running; then
            echo "Not stopped; may still be shutting down or shutdown may have failed"
            exit 1
        else
            echo "Stopped"
            if [ -f "$pid_file" ]; then
                rm "$pid_file"
            fi
        fi
    else
        echo "Not running"
    fi
    ;;
    restart)
    $0 stop
    if is_running; then
        echo "Unable to stop, will not attempt to start"
        exit 1
    fi
    $0 start
    ;;
    status)
    if is_running; then
        echo "Running"
    else
        echo "Stopped"
        exit 1
    fi
    ;;
    *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
    ;;
esac

exit 0

Next, we will update the permissions for the for the init script, enable it to run on startup, and ensure it has started correctly:

chmod 755 /etc/init.d/cloudflared
update-rc.d cloudflared defaults
/etc/init.d/cloudflared start
service cloudflared status

Unfortunately, common DNS diagnostic tools are not installed on the USG, so we will just have to take a leap of faith and assume that if everything looks okay so far, it must be working! If by chance you do have the tools installed, you can use the dig command from the PiHole/Linux section of the guide to ensure it is working.

Configuring `dnsmasq` on the USG

If you have gotten to this point, you should now have a working DNS-over-HTTPS service running. But unfortunately, it's only running locally on the device. The commands below should be run on the USG CLI and will disable the resolv.conf configuration (USG>WAN>DNS in the Unifi controller) and allow the USG to generate the correct dnsmasq configuration.

configure
set service dns forwarding options "no-resolv"
set service dns forwarding options "server=127.0.0.1#5053"
commit
save
exit

sudo /etc/init.d/dnsmasq force-reload

After reloading dnsmasq, queries should now be fulfilled using the Cloudflare DNS service. This can be verified by visiting the internet.nl DNSSEC test service.

Wrapping Up

Congratulations! cloudflared has been succesfully configured. You can now enjoy the extra security, privacy and speed of DNS-Over-HTTPS, as well as some nerd-cred for running an experimental DNS protocol. A big thanks to Cloudflare for creating such a fantastic service!

I hope this guide was helpful. If you have any questions or comments, feel free to leave them below!

How to implement DNS-Over-HTTPS on PiHole, Ubiquiti USG and dnsmasq devices

Cloudflare 1.1.1.1 Service

What is DNS-Over-HTTPS?

Configuring DNS-Over-HTTPS

PiHole and Linux

Installing cloudflared

AMD64 architecture (most devices)

ARM architecture (Raspberry Pi)

Configuring cloudflared to run on startup

Configuring DNS

PiHole and dnsmasq

Direct lookups

Ubiquiti USG

Compiling and Installing cloudflared

Mac or Linux

Docker

Configuring cloudflared to run on startup

Configuring dnsmasq on the USG

Wrapping Up

Installing `cloudflared`

Configuring `cloudflared` to run on startup

Compiling and Installing `cloudflared`

Configuring `cloudflared` to run on startup

Configuring `dnsmasq` on the USG