Fixing networking for Docker

Docker has a known issue where it seems to ignore the network's maximum packet size MTU (Maximum Transmission Unit) and set a default of 1500. This results in situations where the docker container is expecting to be able to send larger packets than the underlying network will send and parts of the packet get trimmed.

Here's an example using the output from running the command ip a:

2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
    link/ether fb:1a:3e:47:80:62 brd ff:ff:ff:ff:ff:ff
    inet 10.17.228.5/24 brd 10.35.228.255 scope global ens3
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:83:20:dc:cc brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever

The important parts are you can see that ens3 (the default network interface) has an MTU of 1450, but Docker created the docker0 interface with a higher MTU of 1500.

The effect of this is that you may get timeouts when connecting to internet services (such as package updates/downloads, Ruby gem installation, etc) or even that files download but don't have the correct size/contents.

Fortunately the fix is quite easy, you need to add a parameter to your launch script for Docker to specify/override the MTU. Using Ubuntu 16.04 for example, the file to edit as root is /lib/systemd/system/docker.service. You need to change the line looking like this:

ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

to look like this:

ExecStart=/usr/bin/dockerd --mtu 1450 -H fd:// --containerd=/run/containerd/containerd.sock

Note: It's quite fine to use the same value as the default network interface (in our example and for our infrastructure 1450), it just can't be higher.

If you are using docker-compose to launch your instances, you will also need to change some config to ensure that they launch with a 1450 MTU.

Your Docker compose file should look something like this:

networks:
  default:
    driver: bridge
    driver_opts:
      com.docker.network.driver.mtu: 1450

After that you need to restart Docker and all will be well with your networking:

sudo systemctl daemon-reload
sudo service docker restart