5 ms·

I am not sure that this belongs on the front page of HN, but the problem seems to stem from the `virtio-blk` driver they're using not supporting TRIM, which cau

by manacit 10y ago

I am not sure that this belongs on the front page of HN, but the problem seems to stem from the `virtio-blk` driver they're using not supporting TRIM, which causes space to never deallocate once the VM writes to a block. Switching to `virtio-scsi` and sending regular TRIMs could fix this issue. It looks like they're also allowing people to configure the maximum size of the qcow2 image, which puts a hard upper bound on how much space the VM will take.

djs55 10y ago
(I'm a Docker employee working on this very issue. I guess my plan to take a break by reading HN failed!) You're completely right -- it's a problem caused by lack of TRIM in the storage path. In the next beta of Docker for Windows (beta 31 due today hopefully) TRIM should be enabled. The Mac will take a little longer as we need to switch protocols and do more work on the host side -- unfortunately the default Apple filesystem doesn't support sparse files so we can't "cheat" by simply passing the TRIM down to the filesystem layer. We'll probably need some kind of explicit block-level compaction to shuffle blocks from the end of the file into holes that have been created by TRIM.
- Cafey 10y ago
  Keep in mind the Apple file system is going to change early 2017! (APFS)
  - winkywooster 10y ago
    It's going to be a long time before it's widely deployed.
    - djs55 10y ago
      I'm looking forward to trying APFS, particularly support for sparse files. But you're right, it'll take while before we can rely on it.
- mzs 10y ago
  Does virtio-scsi and TRIMs work around it?
  - djs55 10y ago
    We'll probably use AHCI since hyperkit https://github.com/docker/hyperkit https://github.com/docker/hyperkit (the Mac hypervisor based on xhyve based on bhyve) lacks a virtio-scsi implementation at the moment. When the VM sends a TRIM ("these blocks aren't used any more, you can free them") we need to make use of this on the host. On Linux/BSD you could tell the host OS that the blocks aren't needed in the file using something like FALLOC_FL_PUNCH_HOLE and the kernel would take care of it e.g. by send TRIMs to the real storage device or shuffling filesystem metadata around. Unfortunately the Mac filesystems in common use don't support this so we'll need to manage this ourselves, e.g. by moving blocks from the end of the file into the middle and shrinking the file.
zitterbewegung 10y ago
Maybe it doesn't but it makes the docker Mac port basically unusable for anyone that runs it on a laptop without much space free.
- ledgerdev 10y ago
  But wait there's more... the mounted volume performance makes it unusable for anyone using mounted volumes for stuff like say dev work. It's a complete joke. https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/240 https://forums.docker.com/t/file-access-in-mounted-volumes-e...

by manacit 10y ago

djs55 10y ago

(I'm a Docker employee working on this very issue. I guess my plan to take a break by reading HN failed!) You're completely right -- it's a problem caused by lack of TRIM in the storage path. In the next beta of Docker for Windows (beta 31 due today hopefully) TRIM should be enabled. The Mac will take a little longer as we need to switch protocols and do more work on the host side -- unfortunately the default Apple filesystem doesn't support sparse files so we can't "cheat" by simply passing the TRIM down to the filesystem layer. We'll probably need some kind of explicit block-level compaction to shuffle blocks from the end of the file into holes that have been created by TRIM.

Cafey 10y ago
Keep in mind the Apple file system is going to change early 2017! (APFS)
- winkywooster 10y ago
  It's going to be a long time before it's widely deployed.
  - djs55 10y ago
    I'm looking forward to trying APFS, particularly support for sparse files. But you're right, it'll take while before we can rely on it.
mzs 10y ago
Does virtio-scsi and TRIMs work around it?
- djs55 10y ago
  We'll probably use AHCI since hyperkit https://github.com/docker/hyperkit https://github.com/docker/hyperkit (the Mac hypervisor based on xhyve based on bhyve) lacks a virtio-scsi implementation at the moment. When the VM sends a TRIM ("these blocks aren't used any more, you can free them") we need to make use of this on the host. On Linux/BSD you could tell the host OS that the blocks aren't needed in the file using something like FALLOC_FL_PUNCH_HOLE and the kernel would take care of it e.g. by send TRIMs to the real storage device or shuffling filesystem metadata around. Unfortunately the Mac filesystems in common use don't support this so we'll need to manage this ourselves, e.g. by moving blocks from the end of the file into the middle and shrinking the file.

zitterbewegung 10y ago

Maybe it doesn't but it makes the docker Mac port basically unusable for anyone that runs it on a laptop without much space free.

ledgerdev 10y ago
But wait there's more... the mounted volume performance makes it unusable for anyone using mounted volumes for stuff like say dev work. It's a complete joke. https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076/240 https://forums.docker.com/t/file-access-in-mounted-volumes-e...