13 ms·
Speed boost achievement unlocked on Docker Desktop 4.6 for Mac
- NorwegianDude 5y agoI guess it's still insanely slow for windows mounts? Or has that changed too?
- matsemann 5y agoOn Windows I just have the files/repo checked out on the WSL side. At least JetBrains' products handle that very effectively (you get a performance drop the other way with that back to windows side, of course, but IntelliJ/PyCharm/etc handles that with a local cache and some magic making it behave as expected). One other issue this also solves is that volume mounts from windows don't inotify correctly, so hot reloading doesn't work properly. But with files on the WSL side that works fine.
- NorwegianDude 5y agoDocker in WSL is the only way it's half usable on windows, but as you say, performance in jetbrains products are quite slow when it communicates with WSL2. Lets hope it will be possible to communicate more efficiently in the future.
- lmeyerov 5y agoRE:Windows, docker <> wsl2 ubuntu feels just like docker <> native ubuntu. We're able to do all our CUDA dev that way too, the only gap is OpenCL. I don't use docker desktop. As it's annoying to do most modern AI + HPC without nvidia GPUs, as long as you're not doing OpenCL, windows seems to have beaten os x for most data people. In contrast, I largely stopped using docker os x because of the broken fs virtualization here made it impossible to do productive JS dev. Subsecond editing vs 10s-3min is a huge step back. Unclear if the the new patches get it to the native-level feeling of wsl2 (1-3% overhead) or still feeling broken in practice. And of course, apple's vertical control means no nvidia GPUs / CUDA, even eGPUs, so no way for docker os x to support that either.
- 1123581321 5y agoInvalid SSL on WPengine—can someone who saw it briefly summarize?
- sethhochberg 5y agoThe release notes have a good overview: https://docs.docker.com/desktop/mac/release-notes/#docker-desktop-460 https://docs.docker.com/desktop/mac/release-notes/#docker-de... > Docker Desktop 4.6.0 gives macOS users the option of enabling a new experimental file sharing technology called VirtioFS. During testing VirtioFS has been shown to drastically reduce the time taken to sync changes between the host and VM, leading to substantial performance improvements. For more information, see VirtioFS.
- benbristow 5y agoShame you started charging for commercial entities. We've all uninstalled Docker Desktop (Windows) now in our team and just gone to using the software (mainly SQL server) natively on our machines instead.
- smoldesu 5y agoGive Podman a look, if you haven't already. It's FOSS through-and-through, and kinda eats Docker's lunch for a lot of my personal uses. Not sure how it scales in larger environments, but it's pretty close to feature parity with Docker where it counts.
- grawlinson 5y agonerdctl[0] is also worth a look. [0]: https://github.com/containerd/nerdctl https://github.com/containerd/nerdctl
- happymellon 5y agoDoes Podman have a VM wrapper like Docker Desktop for running on MacOS?
- MrWiffles 5y agoLast I looked, yes. AFAIK, MacOS lacks the necessary underlying features in the kernel to implement containers (at least this kind of container) at all, so you have to run a Linux VM under the hood to enable rough feature parity, which is a waste of resources in my view. Yet another reason I'm planning to ditch MacOS for Linux full time very soon.
- bonzini 5y agoIt's not that it doesn't implement containers, it's that most likely your container has Linux binaries in it and thus it cannot run on macOS.
- happymellon 5y ago
- athorax 5y agoI know it isn't a direct replacement, but I switched to rancher desktop[0] on macos and it has been a pretty easy transition just using nerdctl instead of docker cli [0]https://rancherdesktop.io/ https://rancherdesktop.io/
- navels 5y agoSimilarly switched to https://github.com/abiosoft/colima https://github.com/abiosoft/colima and have been happy with it.
- Ocha 5y agoI’ve been trying to use Colima but I get weird networking issue - there is a GitHub issue open for it but no solution for now. Was forced to figure out something else for now
- bak3y 5y agoI've also swapped to colima and the only thing I struggle with sometimes is just remembering the executable name to start it up :D
- Aldo_MX 5y agoThat was pretty straightforward to me: COntainer LInux MAcos
- speedgoose 5y agoYou can also use Docker using Rancher Desktop. It’s a setting.
- kedean 5y agoIt's more than a setting, it's the first thing you are asked when you start it for the first time. I love that RD doesn't default to one or the other and make it harder to use one.
- chrisjc 5y ago
- chemicalnovae 5y agoLots of chatter here about alternatives that all seem to miss the point that the alternatives suffer from the same filesystem performance issues that this release claims to improve. Might not be an issue for you but if you've ever had to work with a volume mount with a ton of small files (like php or node dependencies) then this could be a real life saver.
- flurie 5y agoBoth Kata Containers and UTM support virtio-fs, so this is not strictly true. The former can be used as a stand-in replacement for the runtime used by docker desktop[1]. With the latter, one could use a UTM-backed guest as a docker runtime in macOS[2] or run docker directly on the guest[3]. [1] https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-virtio-fs-with-kata.md https://github.com/kata-containers/documentation/blob/master... [2] https://www.codeluge.com/post/setting-up-docker-on-macos-m1-arm64-to-use-debian-10.4-docker-engine/ https://www.codeluge.com/post/setting-up-docker-on-macos-m1-... [3] https://www.lifeintech.com/2021/11/03/docker-performance-on-m1/ https://www.lifeintech.com/2021/11/03/docker-performance-on-...
- deleted 5y ago[deleted]
- cutler 5y agoIs this an issue with dynamic languages in general? Are Java/Kotlin and Golang, for example, not affected?
- rad_gruchalski 5y agoIt depends what you need to do. If you build a golang binary in a container and run go mod tidy, you get hit by it. If you build a java app with, say, maven, you will pull deps into a volume mount and you get hit by it. Of course, you can mount a host dir to avoid the problem to a certain extent.
- shagie 5y ago
- viraptor 5y agoI was curious what's the situation for Colima. It like like fs sharing is being discussed a lot over the years, but virtiofs depends on Mac's virtualisation.framework and can't be used in qemu currently. https://github.com/lima-vm/lima/issues/20#issuecomment-1068621756 https://github.com/lima-vm/lima/issues/20#issuecomment-10686... Instead 9p experimental support got there a few days ago: https://github.com/lima-vm/lima/issues/20#issuecomment-1066086672 https://github.com/lima-vm/lima/issues/20#issuecomment-10660...
- legodion 5y agoKnow what’s even faster? Configuring your local host server with the same values as your remote production server. It really isn’t that hard.
- afturner 5y agoAre you suggesting that containerization is worthless?
- bhuber 5y agoIt is when your local host server is macos and your remote production server is... anything else. I'm sure the number of people running macos servers in production is quite small in proportion to linux.
- rad_gruchalski 5y agoBeen there, done that. 2014: 20k lines of chef code, followed with tons of ansible yaml soup. Containers are better.
- pjmlp 5y agoLooking at our Kubernetes/Docker yaml, my soup tastes different but it is still a soup nonetheless.
- brimble 5y agoCool. I need to roll back to a version from last year, with entirely different versions of daemons (PostgreSQL, et c). Then run that side-by-side with what's currently deployed. I may end up needing to git-bisect and repeat this process a half dozen times—this afternoon. I want none of those deps or configs to hang around, at all, afterward. And I'll need all that gone next week and a whole different set of services installed because I'll be helping out on some other project entirely. But then I'll come back to this and need all those things installed again. At any moment someone might have a question on something else entirely—a whole suite of services, potentially—that, ideally, I'd be able to fire up an try out locally in, at most, a few minutes, then completely eradicate when I'm done so it's not wasting disk space. What's your solution for that? A bunch of full-fat VMs configured with Ansible or something? Been there, done that, less convenient, more resource-intensive, and far noisier/cruftier than Docker (though the script-configured-VM approach is still far better than running that stuff directly on my machine[s], admittedly)
- xerxes901 5y agoOh hey, I’m the Konstantinos mentioned at the end. The excellent work done by the Asahi Linux folk on discovering demonstrating the incredislow MacBook nvme flush is what prompted me to dig into this for Docker. Nice to see the fixes make it into a release!
- xbar 5y agoThank you for advancing Docker science. And for crediting the Asahi folks.
- gkfasdfasdf 5y agoNice work and thanks! Is there any more info on how the nvme flush issue led to such massive perf improvements? The blog post doesn't seem to go into details.
- xerxes901 5y agoSorry I missed this message - I think the best write up of it is from the issue thread: https://github.com/docker/roadmap/issues/7#issuecomment-1044018513 https://github.com/docker/roadmap/issues/7#issuecomment-1044...
- y0ssar1an 5y agoI was super impressed by your detailed investigation in that thread! Thank you for your contributions!
- iRomain 5y agoI sponsor https://github.com/marcan https://github.com/marcan (Asahi). I encourage anyone invested in the Mac ecosystem to do the same. This discovery is just one of the benefits of having people paid to poke around Apple's stuff. PS: of course you can support Docker also: https://www.docker.com/pricing https://www.docker.com/pricing
- brailsafe 5y agoJust upgraded my mac to give it go. So far, with this change you made Docker Desktop for mac go from barely usable, to very useful. Good game.
- gigatexal 5y agoOn Linux how is this done? Raw no virtio or anything?
- hanikesn 5y agoOn Linux you don't need Docker Desktop and and run docker natively without any VMs. FS is managed via overlayfs which provides near native performance.
- stevebmark 5y ago"Docker for Mac" is actually a full Linux virtual machine. The Docker commands you run on your host mac are sent into the Linux VM, to run the technologies Docker uses ("control groups" and "namespaces" mainly). The Linux VM is hidden as an implementation detail of Docker for Mac, meaning you can't ssh into the VM, you can only interact with it through Docker commands. On Linux itself, Docker simply uses the same core Linux technologies to create the isolated runtime "containers" using "control groups" and "namespaces." Docker on Linux is basically native speed because it doesn't have to jump through hoops like syncing the VM filesystem with the host Mac's filesystem, which is a major point of slowness and a long standing pain point with Docker for Mac.
- AltruisticGapHN 5y agoYou seem to have a good understanding of these containers. Do you know in simple terms why macOS doesn't have such "native containers"? After all isn't it more linux-like than Windows? I saw this website that suggests Apple themselves are not providing such a feature... do they see it as a security issue? https://macoscontainers.org/ https://macoscontainers.org/
- gigatexal 5y agoMacOS is based on a Mach kernel with a BSD userland. It’s unixy but containers are a Linux thing based on namespaces and cgroups among other things all of which are Linux kernel primitives. BSD has jails which are similar but different. So the answer is MacOS doesn’t offer containers like you know them because it’s not linux hence Docker for Mac spins up a VM.
- emptysongglass 5y agoCanonical now offers one-command docker environments with Multipass for those looking at alternatives: https://ubuntu.com/blog/docker-on-mac-and-windows-multipass https://ubuntu.com/blog/docker-on-mac-and-windows-multipass
- fourseventy 5y agoCan docker desktop please stop endlessly pestering me to upgrade? It's impossible to skip the update unless I pay which is total BS.
- kordlessagain 5y agoI've had it uninstall itself twice and then fail to reinstall on update. The second time it did it, I just left it be. Maybe you'll get lucky too if you decide to click.
- jpgvm 5y agoStill think the best approach is to forgo containers locally and use tools like Jib or Bazel that build containers directly without a Docker daemon.
- kccqzy 5y agoDid you mean this one? https://github.com/bazelbuild/rules_docker https://github.com/bazelbuild/rules_docker I was very interested in this Bazel-based way of building containers but its README page says "it is on minimal life support," which does not inspire confidence. How's your experience using it?
- jpgvm 5y agorules_docker has been great for Go and Python 3. Can't comment on other langs. Jib has always been absolutely excellent for Kotlin and Java, really can't fault it in any way. I specifically use it with Gradle though, unsure about the quality of the Maven plugin but I would assume great also.
- sneak 5y agoReminder: Docker Desktop is not free software (it's not even source available) and has embedded spyware that uploads a ton of sensitive info from your system without consent when it crashes.
- stillblue 5y agosource?
- sneak 5y agoThe binary. I assume without verification that Docker Inc's "privacy" policy will corroborate.
- acdha 5y agoWhat evidence do you have that it sends this information before you approve it in the crash report? What kind of sensitive information are you referring to?
- deleted 5y ago[deleted]
- kordlessagain 5y ago> Reminder: Docker Desktop is not free software (it's not even source available) and has embedded spyware that uploads a ton of sensitive info from your system without consent when it crashes. This is a straw man argument, just in case anyone cares. The "reminder" is that Docker isn't free and not "even" Open Source. I think Docker has produced Open Source code, however. As most of us get that Docker is nowadays NOT Open Source, the argument falls to "reminding us" that it "uploads a ton of sensitive info" "without consent". Whatever. Docker has personal info on everyone who uses Docker and uses their online services. Phone numbers. Email addresses. Passwords for various Docker websites. People love this because they are Docker customers or long time users. They don't care because most of these customers/users are DOING BUSINESS. Sure, Docker runs third party analytics on their site and pixel trackers in their emails. Big deal. They are in business to make money. They aren't used by everyone (read the public), and most people that run it (assuming it didn't uninstall itself), run it on their work computer. What info is on these machines that Docker would risk their shareholder value to "steal" or "divulge"? Not much, is the answer. To be clear, that last part of the straw man isn't a reminder, it's a claim. Further evidence of straw man shenanigans is present here in replying to "source?" by saying "the binary". Of course data would come out of a binary, but which binary are you referring to? The Docker binary, which runs containers and crashed, or the binary that handles the crash of the other binary? Or maybe it's the installer binary that crashed? Mine did. Twice. All that said, all you people who just shout "privacy, privacy, grumble, grumble, large company, grumble" are a pain in the ass. If you care about privacy, limit what software you use, put stuff that you worry about on another machine and keep your damn data off it. Privacy advocacy isn't worrying about your privacy, it's understanding how the public, at large, looses access to understanding about where they should share their data. Just telling people to worry about privacy isn't helping, and probably does more harm than good.
- Sytten 5y agoAny chance this will become available to older macos versions? My old 2013 Mac doesn't run version 12.
- ta988 5y agoNo because it relies on some newer kernel features.
- craftoman 5y agoI don't trust Docker and never will, it's better to just download the official binary and install the Daemon into a Linux VM then configure it using docker context.
- asah 5y agosure and I totally agree. But for desktop development docker is a godsend to have a reproducible environment.
- hda111 5y agoPodman and nerdctl seem to be the better option when it must be true open source. I know it’s very hard to build the Docker from source that you can download as binary. I don’t like this. Build process should be documented for everyone for trust reasons.
- stevebmark 5y agoThis is fantastic! Docker for Mac has years of pain with the filesystem speed. I remember first diving into the issue and finding some months long Docker forum thread saying "this is basically unfixible except for micro improvements to the kernel." I use Docker for Mac a lot more now and my company is standardizing on containerizing local development. It's been really helpful to avoid poisoning engineer's machines with hard-to-build / env corrupting software (hi Ruby). Disk heavy operations are of course noticeably slower in these systems and I'm excited to try out this new feature.
- 8fingerlouie 5y agoWhile speed improvements for storage are indeed welcome, it would be nice if the massive resource "leak" occurring with the native mac virtualization would be fixed. I'm aware of some document describing that it's not really a resource leak, and instructs us to look at real mem instead of memory consumption, but sadly MacOS is not conviced, and will happliy use memory and swap space until eventually it crashes with out of memory, so while a memory leak might not be the cause, _something_ is claiming to use a lot of memory and not releasing it again.
- wdavidw 5y agoRunning MacOS on silicon, and after enabling VirtioFS as documented, I see a small but not massive improvement. Launching a command with `npx mocha test/index.js` with an empty test only requiring a module (with a lot of dependencies inside it) leads to: - On MacOS host: around 2s - Inside Docker container with VirtioFS enabled: around 20s - Inside Docker container without VirtioFS enabled: around 25s
- wdavidw 5y ago[correction] The speed measurement of 20s is for a X86 Docker image with the platform field forced to amd64. For a native silicon image, the speed improvement is impressive, I am down to 4s.
- hda111 5y agoI don’t like how the cookie dialogue on the Docker blog says it would take a few minutes to save my cookie preferences. I won’t wait a few minutes to just be able to start reading. This was my last visit on the Docker website.
- brailsafe 5y agoPrevious to this update, I was regularly searching for alternatives, and felt that running docker on my mac was the worst part of my job. The performance improvement is ridiculous, and was totally worth upgrading my mac from big sur for.
- AltruisticGapHN 5y agoI've experienced freezes on macOS + Docker 4.6 + using Vite in a php/apache/nodejs container. When these happened, I can't even do CTRL D in the terminal. The STOP button in Docker doesn't work. Had to restart. Anybody else? Otherwise it does seem quite q bit faster, like `vite` starts up almost instantly instead of taking a second or two (having been used days prior, so it has its cache).