Docker Storage Driver: Don't Use Devicemapper

#docker #devicemapper #overlay #storage #driver

DISCLAIMER: I’m not a system expert. What follows is more an aggregation of things I’ve tried and informations gathered on the Internet. I also wrote it to serve as a placeholder for myself. I make sure to provide many links so that you can make your own opinion. Please don’t hesitate to give feedback if you disagree with some statements below.

For some months now, we’ve started deploying more and more Docker services internally to progressively gather experiences.

After encountering a few issues, mainly related to storage, I’ve started to read a lot about the Docker Storage drivers.

Docker default behaviour (with DeviceMapper): Wrong

As much as I love Docker, some may find a pity that the default behaviour is NOT to be used in production (on a non-ubuntu host)!

Indeed, by default, here’s what Docker will choose as a storage driver:

  • AUFS

  • Devicemapper, in loopback mode

BUT, the thing is: though AUFS is apparently great for Docker (was used by DotCloud for their PaaS, before it became public), it’s not in the standard kernel. And is unlikely to be in the future.

For this reason, distributions like RedHat (which is upstream-first) chose to support devicemapper instead, in the so-called thin provisioning mode, aka thinp.

But by default, if AUFS is not found in the current kernel, Docker will fallback to the ubiquitous devicemapper driver. And for the default again, it will create loopback files. This is great for newcomers bootstrapping, but horrible from a least surprise principle perspective: since this mode MUST NOT be used in production.

So, I can use still Devicemapper, if I make sure to use thinp?

Short answer: no.

Longer one: many Docker knowledgeable people have publicly stated that you should prefer other ones. Many have even recommended that you default to using Overlay [1]:

@codinghorror more specifically we’ve never seen devmapper work reliably…​ Overlayfs support is pretty good now. Also zfs & btrfs.

Even Daniel J Walsh aka rhdan, working for RedHat has stated [2]:

Red Hat kernel engineers have been working hard on Overlayfs, (Perhaps others on the outside, but I am not familiar). We hope to turn it on for docker in rhel7.2 BUT […​]


Device Mapper is a complex subsystem

I can’t help thinking that this sentence may tell us more about the subject than his author was thinking. Complexity often being the reason software can not survive years? Might be.

Conclusion: if in doubt, use overlay

I’ve started converting the Docker hosts I work with from devicemapper to overlay. My first impressions are good and everything seems to be working as expected [3].

From all I’ve read, my current wager is that overlay will soon become the default driver. It has the pros of devmapper (no need to create a FS for one) apparently without much of its cons.

Only some specific use cases will still make people choose other drivers like btrfs or zfs. But as these require to create and size real FS to be used, they are unlikely to be used as widely.

Some references

1. Previously named Overlayfs, it has been renamed simply Overlay when it got merged in the Kernel
2. in a pull-request by RedHat’s Vincent Batts, one of the most active Docker committers not working for Docker Inc, about putting overlay as the default driver in place of Devicemapper. That may ring yet another bell to you. At least it did for me.
3. I’ve actually had issues with RedHat’s Docker 1.6.x but this disappeared when I upgraded the Fedora Atomic Host I was playing with to Docker 1.7.1
comments powered by Disqus