4 ms·

We'd love to get your feedback on the new filesystem engine in the Docker for Mac app. It's been a ton of work to get right, and there a few corner cases in th

by avsm 11y ago

We'd love to get your feedback on the new filesystem engine in the Docker for Mac app. It's been a ton of work to get right, and there a few corner cases in the current beta that we're squashing, but overall things "just work" for my day-to-day Linux development on my Mac using the current beta. At this stage, pointing it to the weirdest and most wonderful filesystem stressers you can find is welcome. We'll leap on any issues you find...

tzaman 11y ago
Will do, I've signed up for the beta (same username as here)
nevir 11y ago
One thing I'm immediately concerned about is having some way of "pausing" xhyve. Purely because of Android development :( Intel's HAXM doesn't (seem to?) play nice, and asks for an exclusive lock. See https://github.com/mist64/xhyve/issues/88 https://github.com/mist64/xhyve/issues/88 and https://code.google.com/p/android/issues/detail?id=197915 https://code.google.com/p/android/issues/detail?id=197915
- justincormack 11y ago
  You can quit and restart (its very quick). There isn't a pause at present though.
  - nevir 11y ago
    That's probably good enough
- avsm 11y ago
  I haven't tried the Android emulator recently, but I am interested in deploying Facebook's Infer tool on our codebase (and they've got a Docker container too for it of course). So I've filed an internal bug for us to look into HAXM and figure out if it plays well with Docker for Mac/Windows. Thanks for the pointer!
STRML 11y ago
What is the filesystem engine? Is it just something fancy on top of NFS, or something completely new?
- shykes 11y ago
  It's something new.
  - ngrilly 11y ago
    Is it related to the work started by Brad Fitzpatrick a few months ago with the goal of implementing client-server gateway between the host and the guest filesystem using FUSE?
    - avsm 11y ago
      The new daemon (dubbed osxfs) is FUSE-based at the moment, but also provides a semantic translation layer between OSX filesystem calls and Linux kernel events. The FUSE layer can be removed in the future in favour of a direct kernel module with this architecture, if it ends up being a bottleneck (its fine right now though)
      - ngrilly 11y ago
        Do you use a custom protocol between the daemon running in OSX and the one running in the container? Is there some kind of caching? If yes, what is the impact on tools like make when checking atime or mtime? If no, is there a perceptible impact on latency, for example when compiling a large project in the container?
hinkley 11y ago
I gotta say I've had about all the VirtualBox I can take in this lifetime. It's caused me pretty dire file handling problems on 3 projects and only 2 of those had any Docker in them. Thanks for working on this. I have an open bug against docker-compose (docker doesn't do the same thing by itself) where the wrong layers are being used, but only on virtualbox. Hopefully this will solve that problem, as well as how to make my dev and prod database handling more homogenous. And I can finally turn sendfile back on in my nginx configs without having to special case anything for dev!
lobster_johnson 11y ago
Is this using the normal VirtualBox "shared folders" functionality? For Vagrant we had to drop VirtualBox in favour of VMware Fusion because VirtualBox suffered cache corruption almost every day. You would write a file on the host, and the file would be corrupt inside the VM. Last I checked, this bug was still open, but I'm not certain (on my phone right now), but it still makes me wary of using VirtualBox again. Have you dealt with this issue at all? Edit: Or is this not using any VirtualBox code at all?
- justincormack 11y ago
  No it does not use any virtualbox code.
dancompton 11y ago
I really appreciate you guys working on that. I have since moved to a debian VM, but might eventually move back if I don't need to frequently restart docker machine hosts.
ngrilly 11y ago
What is the performance of this new filesystem engine, compared to VirtualBox shared folders, which are well-known to be slow?
so0k 11y ago
I've been running Docker on Hyper-V since the start (VBox never worked for me as I had a requirement on Hyper-V) - can you provide a little more info on how unikernel experience was used to make a difference? Is this a custom compiled linux kernel with Alpine distro on top? How would volume mounts compare to something like using the netshare (cifs) with a samba share from the windows host? (which is what I'm currently using)?

by avsm 11y ago

tzaman 11y ago

Will do, I've signed up for the beta (same username as here)

nevir 11y ago

One thing I'm immediately concerned about is having some way of "pausing" xhyve. Purely because of Android development :( Intel's HAXM doesn't (seem to?) play nice, and asks for an exclusive lock. See https://github.com/mist64/xhyve/issues/88 https://github.com/mist64/xhyve/issues/88 and https://code.google.com/p/android/issues/detail?id=197915 https://code.google.com/p/android/issues/detail?id=197915

justincormack 11y ago
You can quit and restart (its very quick). There isn't a pause at present though.
- nevir 11y ago
  That's probably good enough
avsm 11y ago
I haven't tried the Android emulator recently, but I am interested in deploying Facebook's Infer tool on our codebase (and they've got a Docker container too for it of course). So I've filed an internal bug for us to look into HAXM and figure out if it plays well with Docker for Mac/Windows. Thanks for the pointer!

STRML 11y ago

What is the filesystem engine? Is it just something fancy on top of NFS, or something completely new?

shykes 11y ago
It's something new.
- ngrilly 11y ago
  Is it related to the work started by Brad Fitzpatrick a few months ago with the goal of implementing client-server gateway between the host and the guest filesystem using FUSE?
  - avsm 11y ago
    The new daemon (dubbed osxfs) is FUSE-based at the moment, but also provides a semantic translation layer between OSX filesystem calls and Linux kernel events. The FUSE layer can be removed in the future in favour of a direct kernel module with this architecture, if it ends up being a bottleneck (its fine right now though)
    - ngrilly 11y ago
      Do you use a custom protocol between the daemon running in OSX and the one running in the container? Is there some kind of caching? If yes, what is the impact on tools like make when checking atime or mtime? If no, is there a perceptible impact on latency, for example when compiling a large project in the container?

hinkley 11y ago

I gotta say I've had about all the VirtualBox I can take in this lifetime. It's caused me pretty dire file handling problems on 3 projects and only 2 of those had any Docker in them. Thanks for working on this. I have an open bug against docker-compose (docker doesn't do the same thing by itself) where the wrong layers are being used, but only on virtualbox. Hopefully this will solve that problem, as well as how to make my dev and prod database handling more homogenous. And I can finally turn sendfile back on in my nginx configs without having to special case anything for dev!

lobster_johnson 11y ago

Is this using the normal VirtualBox "shared folders" functionality? For Vagrant we had to drop VirtualBox in favour of VMware Fusion because VirtualBox suffered cache corruption almost every day. You would write a file on the host, and the file would be corrupt inside the VM. Last I checked, this bug was still open, but I'm not certain (on my phone right now), but it still makes me wary of using VirtualBox again. Have you dealt with this issue at all? Edit: Or is this not using any VirtualBox code at all?

justincormack 11y ago
No it does not use any virtualbox code.

dancompton 11y ago

I really appreciate you guys working on that. I have since moved to a debian VM, but might eventually move back if I don't need to frequently restart docker machine hosts.

ngrilly 11y ago

What is the performance of this new filesystem engine, compared to VirtualBox shared folders, which are well-known to be slow?

so0k 11y ago

I've been running Docker on Hyper-V since the start (VBox never worked for me as I had a requirement on Hyper-V) - can you provide a little more info on how unikernel experience was used to make a difference? Is this a custom compiled linux kernel with Alpine distro on top? How would volume mounts compare to something like using the netshare (cifs) with a samba share from the windows host? (which is what I'm currently using)?