Docker Storage Driver: Don't Use Devicemapper
Wed, Aug 26, 2015DISCLAIMER: I’m not a system expert. What follows is more an aggregation of things I’ve tried and informations gathered on the Internet. I also wrote it to serve as a placeholder for myself. I make sure to provide many links so that you can make your own opinion. Please don’t hesitate to give feedback if you disagree with some statements below.
For some months now, we’ve started deploying more and more Docker services internally to progressively gather experiences.
After encountering a few issues, mainly related to storage, I’ve started to read a lot about the Docker Storage drivers.
Docker default behaviour (with DeviceMapper): Wrong
As much as I love Docker, some may find a pity that the default behaviour is NOT to be used in production (on a non-ubuntu host)!
Indeed, by default, here’s what Docker will choose as a storage driver:
-
AUFS
-
Devicemapper, in loopback mode
BUT, the thing is: though AUFS is apparently great for Docker (was used by DotCloud for their PaaS, before it became public), it’s not in the standard kernel. And is unlikely to be in the future.
For this reason, distributions like RedHat (which is upstream-first) chose to support devicemapper instead, in the so-called thin provisioning mode, aka thinp.
But by default, if AUFS is not found in the current kernel, Docker will fallback to the ubiquitous devicemapper driver. And for the default again, it will create loopback files. This is great for newcomers bootstrapping, but horrible from a least surprise principle perspective: since this mode MUST NOT be used in production.
So, I can use still Devicemapper, if I make sure to use thinp?
Short answer: no.
Longer one: many Docker knowledgeable people have publicly stated that you should prefer other ones. Many have even recommended that you default to using Overlay [1]:
@codinghorror more specifically we’ve never seen devmapper work reliably… Overlayfs support is pretty good now. Also zfs & btrfs.
https://twitter.com/solomonstre/status/604055267303636992
Even Daniel J Walsh aka rhdan, working for RedHat has stated [2]:
Red Hat kernel engineers have been working hard on Overlayfs, (Perhaps others on the outside, but I am not familiar). We hope to turn it on for docker in rhel7.2 BUT […]
Or:
Device Mapper is a complex subsystem
I can’t help thinking that this sentence may tell us more about the subject than his author was thinking. Complexity often being the reason software can not survive years? Might be.
Conclusion: if in doubt, use overlay
I’ve started converting the Docker hosts I work with from devicemapper to overlay. My first impressions are good and everything seems to be working as expected [3].
From all I’ve read, my current wager is that overlay will soon become the default driver. It has the pros of devmapper (no need to create a FS for one) apparently without much of its cons.
Only some specific use cases will still make people choose other drivers like btrfs or zfs. But as these require to create and size real FS to be used, they are unlikely to be used as widely.
Some references
-
Pull Request to promote Overlay to first. Closed since there’re still issues
-
Since 1.7.0, Docker will guess the previously used driver and still use it even if the default changes. This is indeed a cool enhancement to be able to change the Docker default driver to something else than devmapper/aufs/whatever in a future version without breaking existing installations while upgrading.
-
Interesting feedback: Docker with Overlayfs first impression
-
Comprehensive Overview of Storage Scalability in Docker. By the way you can see overlay is indeed very quick, and comparable to btrfs without the FS creation burden.
-
Friends Don’t Let Friends Run Docker on Loopback in Production
-
Pull Request still currently open to make Docker emit Warnings when using devmapper in loopback mode