Issues with Docker's default MTU and its effects
Docker has a known issue where it seems to ignore the network's maximum packet size MTU (Maximum Transmission Unit) and set a default of 1500. This results in situations where the docker container is expecting to be able to send larger packets than the underlying network will send and parts of the packet get trimmed.
How to detect the MTU issue
Here's an example using the output from running the command ip a
:
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000
link/ether fb:1a:3e:47:80:62 brd ff:ff:ff:ff:ff:ff
inet 10.17.228.5/24 brd 10.35.228.255 scope global ens3
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:83:20:dc:cc brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
The important parts are you can see that ens3
(the default network interface) has an MTU of 1450
, but Docker created the docker0
interface with a higher MTU of 1500
.
The effect of this is that you may get timeouts when connecting to internet services (such as package updates/downloads, Ruby gem installation, etc) or even that files download but don't have the correct size/contents.
How to fix the MTU issue with Ubuntu
Fortunately the fix is quite easy, you need to add a parameter to your launch script for Docker to specify/override the MTU. Using Ubuntu 16.04 for example, the file to edit as root is /lib/systemd/system/docker.service
. You need to change the line looking like this:
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
to look like this:
ExecStart=/usr/bin/dockerd --mtu 1450 -H fd:// --containerd=/run/containerd/containerd.sock
Note: It's quite fine to use the same value as the default network interface (in our example and for our infrastructure 1450
), it just can't be higher.
If you are using docker-compose
to launch your instances, you will also need to change some config to ensure that they launch with a 1450
MTU.
Your Docker compose file should look something like this:
networks:
default:
driver: bridge
driver_opts:
com.docker.network.driver.mtu: 1450
After that you need to restart Docker and all will be well with your networking:
sudo systemctl daemon-reload
sudo service docker restart
In Conlusion
This article has provided a solution for an issue caused by Docker ignoring network's MTU setting, resulting in connectivity errors. By modifying the launch script and configuring the Docker-compose, the MTU can be set correctly. Don't forget to restart Docker after editing the launch script to apply the changes