3 ms·

So, I assume that if you aren't using AUFS then you don't have to deal with potentially bumping up against the 42 layer limit? Or does this update also address

by gexla 13y ago

So, I assume that if you aren't using AUFS then you don't have to deal with potentially bumping up against the 42 layer limit? Or does this update also address the issue with AUFS?

shykes 13y ago
The 42 layer limit is still there... But not for long! There is a fix underway to remove the limitation from all drivers, including aufs. Instead of lifting the limit for some drivers first (which would mean some images on the index could only be used by certain drivers - something we really want to avoid), we're artificially enforcing the limit on all drivers until it can be lifted altogether. If you want to follow the progress of the fix: https://github.com/shykes/docker/pull/66 https://github.com/shykes/docker/pull/66 (This pull request is on my personal fork because that's where the storage driver feature branch lived until it was merged. Most of the action is usually on the main repo).
- m_mueller 13y ago
  One thing I've been wondering is whether the 42 limit was related to performance considerations. If yes, I actually somewhat like it - I'm a proponent to making systems behave in a way that they won't come around and bite you. Will a container based on, say, 200 layers still load and run with reasonable performance?
  - alexlarsson 13y ago
    That depends on the backend a bit i guess. devicemapper "flattens" each layer, so the depth should have zero effect on the performance. For aufs, I have no real data, but I assume that a 42 layer image is somewhat slower than a 1 layer image. The fix for going to > 42 layers is to recreate a full image snapshot (using hard links to share file data) every N layers, so the performance would be somewhere between the performance between 1 and 42 layers depending on how many AUFS layers you got.
    - m_mueller 13y ago
      Doesn't 'flatten' mean that there's going to be a tradeoff between IO performance and storage size? If that's the case I'd like to have some control over what happens, at least with one of the implementations. Say, during development of a Dockerfile just use the layers as before, potentially without any depth limits, but when an image is ready being able to call 'flatten' manually. Some background: Using 0.65 it took me several days to develop a Docker image with manually compiled versions of V8, PyV8, CouchDB, Flask, Bootstrap and some JS Libraries[1]. With no 42 limit it wouldn't have taken so long since I would have had way more caching points - however I'd also be afraid about performance. [1] the image is available on the repository as ategra/xforge-dependencies. I can upload the Dockerfile to github if anyone is interested.
      - alexlarsson 13y ago
        "flatten" is a simplification, what I mean is that the devicemapper thin provisioning module uses a data structure on disk that has a complexity that is independent of the snapshot depth. I did not mean that the Docker image itself is somehow flattened. I don't think there will be any performance problems with deep layering of images, either on dm or ads.
        m_mueller 13y ago
        That's some great news, thanks a lot for the heads up and the great work you guys are doing! I'm really looking forward to what Docker and its ecosystem is becoming - I think it's already quite obvious that it will revolutionize the way people think about linux application rollout - both from the user- as well as the application developer's perspective. It might even make 2014 the year of the linux desktop ;-).
  - shykes 13y ago
    The 42 limit comes from the aufs code itself. My guess is that it's because of stack depth: if you stack these kernel data structures too many levels deep, you might trigger an overflow somewhere. The hardcoded limit is probably a pragmatic way to avoid that. Of course if you had infinite memory and those kernel data structures were designed to never overflow, then you would have a performance problem on your hands instead of a kernel panic :)

by gexla 13y ago

So, I assume that if you aren't using AUFS then you don't have to deal with potentially bumping up against the 42 layer limit? Or does this update also address the issue with AUFS?

shykes 13y ago

The 42 layer limit is still there... But not for long! There is a fix underway to remove the limitation from all drivers, including aufs. Instead of lifting the limit for some drivers first (which would mean some images on the index could only be used by certain drivers - something we really want to avoid), we're artificially enforcing the limit on all drivers until it can be lifted altogether. If you want to follow the progress of the fix: https://github.com/shykes/docker/pull/66 https://github.com/shykes/docker/pull/66 (This pull request is on my personal fork because that's where the storage driver feature branch lived until it was merged. Most of the action is usually on the main repo).

m_mueller 13y ago
One thing I've been wondering is whether the 42 limit was related to performance considerations. If yes, I actually somewhat like it - I'm a proponent to making systems behave in a way that they won't come around and bite you. Will a container based on, say, 200 layers still load and run with reasonable performance?
- alexlarsson 13y ago
  That depends on the backend a bit i guess. devicemapper "flattens" each layer, so the depth should have zero effect on the performance. For aufs, I have no real data, but I assume that a 42 layer image is somewhat slower than a 1 layer image. The fix for going to > 42 layers is to recreate a full image snapshot (using hard links to share file data) every N layers, so the performance would be somewhere between the performance between 1 and 42 layers depending on how many AUFS layers you got.
  - m_mueller 13y ago
    Doesn't 'flatten' mean that there's going to be a tradeoff between IO performance and storage size? If that's the case I'd like to have some control over what happens, at least with one of the implementations. Say, during development of a Dockerfile just use the layers as before, potentially without any depth limits, but when an image is ready being able to call 'flatten' manually. Some background: Using 0.65 it took me several days to develop a Docker image with manually compiled versions of V8, PyV8, CouchDB, Flask, Bootstrap and some JS Libraries[1]. With no 42 limit it wouldn't have taken so long since I would have had way more caching points - however I'd also be afraid about performance. [1] the image is available on the repository as ategra/xforge-dependencies. I can upload the Dockerfile to github if anyone is interested.
    - alexlarsson 13y ago
      "flatten" is a simplification, what I mean is that the devicemapper thin provisioning module uses a data structure on disk that has a complexity that is independent of the snapshot depth. I did not mean that the Docker image itself is somehow flattened. I don't think there will be any performance problems with deep layering of images, either on dm or ads.
      - m_mueller 13y ago
        That's some great news, thanks a lot for the heads up and the great work you guys are doing! I'm really looking forward to what Docker and its ecosystem is becoming - I think it's already quite obvious that it will revolutionize the way people think about linux application rollout - both from the user- as well as the application developer's perspective. It might even make 2014 the year of the linux desktop ;-).
- shykes 13y ago
  The 42 limit comes from the aufs code itself. My guess is that it's because of stack depth: if you stack these kernel data structures too many levels deep, you might trigger an overflow somewhere. The hardcoded limit is probably a pragmatic way to avoid that. Of course if you had infinite memory and those kernel data structures were designed to never overflow, then you would have a performance problem on your hands instead of a kernel panic :)