In general I like the nicolaka/netshoot image for troubleshooting. It has all the tools you need (ip, curl, …). It’s nice for network debugging used with docker run --network=container:$existing_running_containter. Then you have the same ip/traffic like the container you want to debug. If you’re looking for something like top but for containers, I recommend ctop. Just check the aliases below.

Some aliases

DOCKER_FORMAT="table {{ .Names }}\t{{ .Image }}\t{{ .Status }}\t{{ .Ports }}\t{{ .Names }}"

alias dl='docker ps --format "$DOCKER_FORMAT"'
alias dg='docker ps --format "{{ .Names }}" | rg $1'
alias ctop="docker run --rm --name ctop -v /var/run/docker.sock:/var/run/docker.sock -it nicolaka/netshoot ctop"

alias deb=docker_exec_bash
docker_exec_bash() {
    docker exec -it $1 bash
}

alias des=docker_exec_sh
docker_exec_sh() {
    docker exec -it $1 sh
}

alias den=network_debug
network_debug() { # docker exec network (run debug container with network of $1 container)
    docker run --rm --network=container:$1 -it nicolaka/netshoot
}

alias di=container_ips
container_ips() { # show all running containers and their ip addresses
for container in $(docker ps -q)
do
        docker inspect -f '{{ .Name }}: {{range.NetworkSettings.Networks}}{{.IPAddress}} {{end}}' $container;
done
}

Firewalling with Docker

Docker automatically adds iptables rules. When port forwarding is configured, it automatically opens ports in the firewall. Some ways to fix that:

• Use the firewall provided by the hosting platform (some providers allow to set firewall rules)
• Use ufw-docker (Pull Request with v6 support)
• Set "ip": "127.0.0.1" in daemon.json. Then 8080:80 binds to 127.0.0.1 only (docs).
• Do firewalling manually:

1. Set { "iptables": false } in /etc/docker/daemon.json.
2. Use a fixed name for the bridge in docker-compose.yml.

networks:
  nextcloud:
    driver_opts:
      com.docker.network.bridge.name: br-nextcloud

3. Use your favorite firewall tool. I like ferm (example config).

• Use DOCKER-USER chain

Docker’s iptables rules explained and how to deal with it

Let’s say you run sudo docker run -p 8080:80 nginx. Docker creates the following rules for that port forwarding (a bit simplified, as Docker adds these rules into own DOCKER* chains, so the names are a bit different. But it’s easier to understand it this way).

First, there is a port forwarding to the container ip and port (nat table in the PREROUTING chain):

DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 to:172.17.0.2:80

This leads to the first problem: To filter (allow/block) the ports docker automatically opens, you need to add rules to the FORWARD chain and not to the INPUT chain as usually. So you should not forget to block/allow host services like ssh.

The second rule Docker creates is an ACCEPT in the FORWARD chain (filter table)

ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:80

The DOCKER-USER chain in the filter table is intended to be used by the administrator to allow/block Docker traffic. The ACCEPT rule above is evaluated later. So in DOCKER-USER, you can use a generic -i WAN -j DROP rule. Then you can allow single ports. The pitfall: After the port forwarding, the destination port is changed. So now we need to allow port 80, even when we originally wanted to allow port 8080.

The solution is to use the conntrack module:

iptables -I DOCKER-USER -i WAN -p tcp -m tcp -m conntrack --ctorigdstport 8080 -j RETURN

Where ctorigdstport means “the original destination port”.

The next pitfall: The DOCKER-USER chain Docker creates a single line in there: a general

RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

This means that you need to add your rules before the global RETURN rule. You can also flush the DOCKER-USER chain (-F DOCKER-USER) and then re-add the global generic rule. Let’s put everything together:

ext_if="eth0"
iptables -I DOCKER-USER -i ${ext_if} -j DROP
iptables -I DOCKER-USER -i ${ext_if} -m conntrack --ctstate RELATED,ESTABLISHED -j RETURN
iptables -I DOCKER-USER -i ${ext_if} -p tcp -m tcp -m conntrack --ctorigdstport 8080 -j RETURN

We end up with these rules:

root@spring:~ iptables -vnL DOCKER-USER
Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   25  3032 RETURN     all  --  wlan0  *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    2   120 RETURN     tcp  --  wlan0  *       0.0.0.0/0            0.0.0.0/0            tcp ctorigdstport 8080
   50  3000 DROP       all  --  wlan0  *       0.0.0.0/0            0.0.0.0/0           
   32  5695 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

This drops all traffic from outside except for port 8080. With -I we insert rules at position 1. The order of inserting is reversed on purpose to first have the allow/accept rules (RETURN) and then have global DROP (from WAN interface). It’s a bit sad that the Docker documentation does not tell use anything about ctorigdstport.