6 ms·
The docker situation on MacOS is so painful. That's the number 1 factor keeping me off it. It's such a fundamental tool to development in this day and age and
by stusmall 4y ago
The docker situation on MacOS is so painful. That's the number 1 factor keeping me off it. It's such a fundamental tool to development in this day and age and Docker Desktop doesn't cut it at all. I used to run into mysterious lock ups of the VM when running decently complex workloads. It would happen maybe 2 or 3 times a week and I would have to blow away all state and set everything back up. Recently I've seen issues where it can't even build some non-trivial images on machines using M-series processors. I've tried Rancher Desktop too but also ran into similar issues. Neither of them can make it past building a decent sized rust project while emulating x86. This doesn't even take in account the performance of these builds that do finish. I'd be happier with slow builds if they at least worked.
Aside from build and stability issues, I've also hit not-so-subtle bugs in MITM proxies they add for connections on some ports.
I get why building Docker on MacOS, especially with the x86 emulation twist, is such a difficult problem. I don't envy the engineers working on this. But I don't get a developer who knows they need Docker would pick a mac for their main machine.
- infotogivenm 4y agoI haven’t noticed locks/hangups but I have noticed that running/building on x86_64 images on ARM macOS (through Rosetta) is excruciatingly slow. I ended up having to switch us to cross compiling, which works okay but changing all our builds over will be a pain. For greenfield images I have not had much issue anecdotally.
- jakear 4y agoHave you had much luck with Podman? I used to do QA for tooling to integrate VSCode with Docker and also couldn't stand the MacOS experience. But from what I understand podman works without the daemon and is generally more stable-ish, while being CLI/API-compatible (at least based on the issue reports I've fielded)
- ollien 4y agoThe reason I hated Docker Desktop was simply that I needed to spin up a VM which used up a bunch of memory (yes it's configurable, but I don't want to mess with that. I don't have to on Linux!). I don't believe the podman Mac client solves this problem (nor can it, given it just simply isn't running on a Linux kernel) On Linux, I quite like Podman! It has a couple of gaps, but for simple things it works great
- zitterbewegung 4y agoHonestly Docket Desktop for macOS has honestly been one of the most frustrating applications I have ever used and it isn’t Apple’s fault. Maybe with the new virtualization layer the developers at docker can fix it. One of the other issues is that it starts to create and enormous docker raw file.
- nicoburns 4y ago> and it isn’t Apple’s fault Eh, yes it is. Apple doesn't provide a containerisation API for their OS, which means Docker is forced to run non-natively!
- oneplane 4y agoEh, no it isn't. apple does have a public container API but the problem is you still can't run Linux binaries on macOS no matter how many containers you put them in. Containers on macOS don't use OCI images or anything like that, but scoped and grouped access to system resources with a separate user land that 'looks' like a full OS has been available for over 5 years. Even without that, XNU has plenty of facilities for it, and even if it didn't you'd still be able to use Launchd environments (which are more akin to LXC). Docker relies on specific implementation details of the Linux kernel. It always has. That is why you always need a Linux OS somewhere to run the containers.
- eptcyka 4y agoThere is an OCI implementation for Windows which allows for Windows native containers. So docker can run on non-linux systems just fine, it's just that nobody cares. People use docker to package Linux applications, not because of some inate dockerness.
- oneplane 4y agoOCI images yes, but that doesn't make them the same. OCI can be used to store many things, and those non-Linux OCI images will not run on Linux. Does that mean that it is Linux at fault for not running arbitrary formats from OCI images? No. This applies everywhere; if you build for a specific target and package it up, not using that target when trying to run it is a PEBCAK. Example: there are artefacts you can store as OCI images that contain Mach-O binaries, and Windows and Linux won't really run those. You can even still PPC code in those binaries, and link to Cocoa frameworks. This means that the OS and architecture you are targeting will not be Windows or Linux, and that is not their fault.
- jkmcf 4y agoJust one data point, but I can't say I've had any development problems with Docker on MacOS. It's frustratingly slow, but it has always worked. Perhaps you are doing stuff way more complicated than just running a few containers.
- geerlingguy 4y agoFor individual containers or smaller local deployments using internal Docker networking, it's usually stable and you can use hacks like NFS reverse mounts for faster shared files. But I've run into issues emulating more complex environments where you need multiple exposed ports, more complex host-to-Docker networking, etc., which generally works on Linux because it's not stuck in a VM shim layer. But most of the time, I don't have any issues, just sometimes slower environments on Mac if I don't tune folder mounts and run apps like Ruby/Node/PHP with hundreds or thousands of files.
- tzs 4y agoYou can use Wireguard to tunnel from the internal Docker networking to Mac's network, so that each Docker container can have a separate IP address that is visible to Mac processes. Here's a description of my setup for that: https://news.ycombinator.com/item?id=33665178 https://news.ycombinator.com/item?id=33665178
- stusmall 4y agoI think this touches on what might have been the heart of the issue of one of my projects. It made use of extremely, complex dynamic docker networking. Both inter-container and host networking. It was a beast
- beebmam 4y agoThe problem is that the Docker runtime doesn't actually run on MacOS. It runs on a Linux VM running alongside MacOS. This can make a few things complex enough that requires changes to applications which interface with docker, especially related to networking. It's the same with Docker on Windows (running Linux containers. Windows containers are a whole other issue)
- foxandmouse 4y agoFunny enough, I switched back to developing on Vagrant because docker wold make my i9 macbook sound like a jet engine.. And if you need docker, running it in vagrant is so much more performant.
- runjake 4y agoSo, what do you actually use?
- stusmall 4y agoUbuntu
- cassianoleal 4y agocolima [0] has been working pretty well for me, much better than Docker Desktop. [0] https://github.com/abiosoft/colima https://github.com/abiosoft/colima
- daitangio 4y agoMe too using (co)lima and no issue. You can mix x86 and ARM images in a transparent way
- pjmlp 4y agoPlenty of development workloads do just fine without Docker.
- bobkazamakis 4y agoEverything works fine without docker, but would you rather line up atoms on your processor with a magnet by hand, or write code?
- pjmlp 4y agoI have managed to write code for 30+ years without Docker, or hand aligning atoms, go figure.
- seiferteric 4y agoI am surprised there has not been an effort to separate docker from linux. It seems like it should be possible by providing the syscall layer in a crosss platform library somehow. I remember years ago there was a "linux userland" project where they extracted parts of the linux kernel (like networking) into userland libraries, but I don't think it exists anymore. WSL 1.0 was probably the closest thing recently.
- paulmd 4y agoThe BSD world has had this for ages in the form of "branded zones" in opensolaris/illumos and linuxulator for jails. It's too complex to thunk windows calls to linux, that is what WSL1 tried and failed at, because windows is just too different from a POSIX paradigm for that to work well. But posix-on-posix isn't really all that hard... nor is other *nix-on*nix.
- schappim 4y ago> that is what WSL1 tried and failed at What did they do differently with WSL2+?
- baq 4y agoEverything, WSL2 is a special hyperv vm. It’s a true Linux kernel with a true Linux file system. The network stack is one way integrated so you don’t have to forward any ports or configure NAT.
- qbasic_forever 4y agoIt runs a hyper-v VM with a Linux kernel.
- thangngoc89 4y agoWSL2 uses VM. So actual Ubuntu distribution under the hood
- nodja 4y agoBroadly speaking WSL1 is linux implemented in the windows kernel, think reverse wine. WSL2 is linux running on a thin VM with several bridge layers that can access windows resources (networking, files, GPU, etc.)
- TimTheTinker 4y agoMy personal fix: Pay for a Parallels Pro license and run Docker in an Ubuntu VM that is configured to start at login and run in the background. I use the Remote SSH plugin in VSCode and configured the VM to forward all container ports to localhost (as well as forward port 22 to port 2222 on localhost). I also have `alias vmlogin=ssh -X -p 2222 localhost` in my .zsh_profile and set up ssh to use a public/private key pair for quick logins.
- turndown 4y agoI know you aren’t denying OPs points but the fact that the solution is to just run Linux(with a $99 fee, from what I can see online) to solve how bad Docker is on OS X is a huge problem.
- TimTheTinker 4y agoI also like how I'm gaining experience using the "developer's OS", which I'm sure will be valuable in the future.
- res0nat0r 4y agoColima on an M1 Mac has been working fine for me. https://github.com/abiosoft/colima https://github.com/abiosoft/colima Just do a 'colima start' once a month when I actually have to reboot my mac, to boot up the background VM, and everything else basically runs as normal.
- baq 4y agoI tried Rancher desktop which is I assume a gui wrapper for Colima. I had an issue where I had to put 9p to some obscure config file for a mount to work and then the whole thing hung under load anyway.
- hakanderyal 4y agoAfter 10+ years of enjoying my time on mac, I switched back to Windows because of Docker. It was working fine for me, maybe requiring one restart per month. But the slowness was unbearable anymore. Don't even get me started on the effects of it on battery life. To anyone looking for a replacement, Windows is usable. I still miss the consistency of MacOS tho. Since I only use laptops, I haven't even bothered to try Linux again, my last experience was not that great with compatibility issues and hibernation problems.
- ggregoire 4y agoJust to add another data point, Docker works fine on mine: MacBook Pro (14-inch, 2021) Chip Apple M1 Pro Memory 16 GB macOS Monterey Docker 20.10.17 I agree it was awful on the previous generations tho.
- stock_toaster 4y agoI found it so painful that I just use a docker context targeting a remote docker instance running on a linux machine. Works well enough for my needs, but certainly isn't a solution for everyone.
- tensor 4y agoI've been using Rancher desktop and it seems to work fine for me. I don't notice anything being particularly slow and no lockups or anything. If Mac wasn't an option I really don't know if I'd try using Linux on the desktop vs Windows with WSL. I have nothing but horrible memories when I had to use Linux on the desktop.
- oneplane 4y agoNo problems here with Docker and macOS. Docker and Windows works too. We have a ratio of about 97% developers on macOS, 2% on windows and 1% on 'else' which is mostly Linux and a single BSD user. This includes multiple editors and IDEs used, multiple languages and multiple frameworks, also multiple types of development (from common devops facilities development to front-end simulation, on-prem and cloud, C#, Python, Java, Go, Rust, Swift, Objective-C, C++, TypeScript, Scala, T-SQL, PL/SQL (yep, RDBMS with on-DB applications), some on plain docker, some on Podman some on local Kubernetes clusters, all using wired and wireless connections (or both), one or more VPN connections), and nobody is having any issues with macOS and Docker on Intel or ARM. The only non-ARM OS we have around is Windows because we can't legally get it. Everything else is multi-arch, except for some legacy Oracle and IBM applications that are being phased out and have been handed off to a third party to maintain while we rip them out. We have the luxury of not really doing and foreign architecture compilation because you can develop locally and CI will do matrix builds so we can run on a mixed spot fleet on AWS. We used to build for POWER8 too (before we handed off the IBM nonsense). Some people do it anyway, especially those who want to test binary protocols (a very small percentage) when they write applications that do IT<-->OT communication and want to make sure that the architecture on the IT side doesn't mess up binary payloads that the OT side won't know what to do with. This is usually done with k3s and skaffold where we attach and 2nd node to a Rancher Desktop instance that runs pure x86 instead of x86-in-ARM via Rosetta. We also have 1 Linux user who does this, but she's just using plain qemu emulation for ARM-on-x86 (instead of the other way around on M1/M2 Macs). So all in all, I think the problem is still valid, but your experience is not really representative it seems. Keep in mind that happy people tend to not go on the internet and tell everyone about all the problems they don't have ;-) So most reports we'll see will be from people having issues with something. The biggest issue we used to have was the AUFS support, but even that was mostly a painpoint for those who could not auto build from the ide or didn't have things like Jib. That has been resolved for over 5 years tho.
- kitsunesoba 4y ago> a single BSD user. This is surprising. I run FreeBSD on a home server but I would imagine that it'd be rather unproductive to run as a web dev workstation OS, since it shares friction points with macOS (mainly, it's not Linux) but doesn't have even a fraction as many users which means you're basically on your own for fixing any issues that crop up. More power to this individual if they're making it work though. It's great to see diversity in the desktop/workstation OS usage… in my view, desktop monoculture will only come back to bite us in the ass.
- duped 4y ago> It's such a fundamental tool to development in this day and age I'd change this to "web" development. There are a lot of us not doing anything that requires even thinking about docker.
- kenjackson 4y agoI’d make that “cloud” development. The web is a part of it — a big part, but just a part.
- Rimintil 4y agoIt’s more effort to deploy docker then to use App Services in Azure. Fortunately no need for a container environment. It adds an unnecessary complexity.
- stusmall 4y agoI wouldn't be so quick to dismiss. It's true you see it a lot more on the cloud side of things. I've found its really useful in any situations where I want to capture the state of the OS along with the program. A good example is creating a CI environment that can be easily reproduced on a developers machine. It cuts out a lot of back and forth of "it works on my machine" when you are just a docker build/docker run away from having a local test image that is identical to whats run in CI. Some of the project I mentioned in the above post were very much not "web" projects. I've also seen it as a method to share difficult to set up development environments like you sometimes see in embedded development. I'm not alone in picking the word "fundamental". In the same survey that is article is based on, Docker was the #1 "other tool" for professional devs. SO had this to say about it: "Last year we saw Git as a fundamental tool to being a developer. This year it appears that Docker is becoming a similar fundamental tool for Professional Developers, increasing from 55% to 69%." I think the comparison works too. Git isn't the best tool for everyone, but is extremely common and a sound default. Docker also isn't the best tool for everyone, but isn't far off on being as ubiquitous.
- nicce 4y agoDocker is used everywhere. Containers allow fluent use of any CLI tool in crossplatform way. No need to bang your head into the wall when fixing your dependencies.
- einpoklum 4y ago> The docker situation on MacOS is so painful Stop trying to get me to switch to MacOS :-P > It's such a fundamental tool to development s/fundamental/gratuitous, undesirable/ . IMNSHO.
- throwaway6734 4y agoAdding a data point: docker on Mac crashes for me whenever I connect to the company vpn. I need to go into activity monitor, kill all docker processes, and reboot. It's not the end of the world, but it is frustrating
- SEJeff 4y agoTry this: brew install podman-desktop https://podman-desktop.io https://podman-desktop.io is a full (free) docker replacement supported by redhat.
- yamtaddle 4y agoDocker-the-daemon not performing well on Mac has always struck me as a weird complaint. It's like complaining that MS-DOS FAT32 defrag doesn't work well on Linux with an ext4 volume. It's not native to the platform and uses alien features, it's cool that it works as smoothly as it does. Meanwhile, the client works just fine and can connect to a headless machine running Linux if you don't want the cost of virtualizing the host OS. Like, if my Linux laptop were having trouble running a Mac copy of Photoshop in a Mac VM (ignore the legality) and it was really important for work and that was getting in my way, I'd... get a Mac. If Docker on Mac isn't cutting it, get a Linux box—it's not Mac-native software and you can still interact with it pretty seamlessly from the Mac.
- withinboredom 4y agoHell. I have buildkit containers running in a k8s dev cluster. It’s perfect for building locally and the only slow part is shipping the build context. But with stuff like devspace, running the whole thing with auto-sync/restart is just simply amazing.
- deleted 4y ago[deleted]
- dmix 4y agoHave the performance issues with Docker been resolved on M1/2 yet? I remember playing around with running our codebase through Docker when M1 first got released and it was really slow (due to ARM and/or shared disk read speeds?). But from what I understood it was something that was to be fixed soon... I'd love to use Docker at work but it was a show stopper.