6 ms·

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways

by dchuk 4y ago

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”. I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

pbowyer 4y ago
Have you tried enabling Virtiofs in Docker Desktop? On my M1 Mac that sorted the performance for me for large PHP apps, and I haven't experienced the permissions problems that some people have (there is a test build that appears to fix those out now).
- charrondev 4y ago
  I have an M1 Pro MacBook and I turned on the 2 experimental settings and that sorted my performance issues. I think the new virtualization framework just came out of beta though so now it’s just VirtuoFS that needs to be enabled.
irusensei 4y ago
Canonical has a product of their own: https://multipass.run/ https://multipass.run/ They all use the underlying OS hypervisor so I’m not sure you can get perceivable performance out of one solution over the other.
- todotask 4y ago
  Sometime it can be buggy on macOS and quite slow to boot with Ubuntu VM, wish there is a cloud VM image.
  - hda111 4y ago
    > wish there is a cloud VM image What do you mean? Multipass only runs Ubuntu server cloud images.
pickleMeTimbers 4y ago
We're using Multipass + Bravetools (https://github.com/bravetools/bravetools https://github.com/bravetools/bravetools) / Docker to build system/application containers. I believe multipass uses SSHFS (https://discourse.ubuntu.com/t/how-to-improve-mounts-performance-in-multipass/26638 https://discourse.ubuntu.com/t/how-to-improve-mounts-perform...) to mount filesystems between the host and the VM. Performance has been excellent.
hamandcheese 4y ago
Because docker for Mac has always kind of sucked, I’ve moved on to making nix-based development environments and I’ve been very pleased so far. Lately I’ve been trying https://devenv.sh/ https://devenv.sh/ and it works great! I haven’t tried it for ruby, though I have used vanilla nix shell for ruby projects before and it worked quite well after I over-rode GEM_PATH and GEM_HOME to the correct values.
- nerdponx 4y ago
  I've been using Colima lately and it's been a great experience. The only thing that didn't quite work was pushing an image to AWS ECR, but pulling images, building images (with BuildKit), and running containers all worked fine, and faster than Docker.
  - hamandcheese 4y ago
    I've tried Colima too but it has always felt sluggish compared to local dev. I don't think any form of cross-host filesystem sharing could ever be fast enough.
  - MuffinFlavored 4y ago
    isn’t Buildkit Docker?
    - MuffinFlavored 4y ago
      https://github.com/moby/buildkit https://github.com/moby/buildkit it looks like it is not, kind of?
      - krab 4y ago
        Moby is Docker.
    - nerdponx 4y ago
      Whatever it is, Colima seems to have faster image builds and faster container startup, which makes my life better.
bsnnkv 4y ago
I've recently switched to using nix-shell and overmind with a Procfile to officially retired docker-compose from all of my projects. Such a breath of fresh air. Like Mitchell Hashimoto (Hashicorp), I no longer do any development on macOS directly; I have a Linux VM where all my projects live and have their environments managed with a shell.nix file. Highly highly recommend it.
- unboxingelf 4y ago
  Would you elaborate on the setup? I recall seeing a comment or article where Mitchell mentioned this setup previously. I’ve been using Make + Docker for development on macOS for a long time now and perhaps it’s time for a change. I’ve looked at nix once or twice over the years but it never really clicked.
  - bsnnkv 4y ago
    Sure, this is a super abbreviated version of my current setup for developing https://notado.app https://notado.app shell.nix - this brings the packages I need to develop into the $PATH when I run `nix-shell` in the repo {pkgs ? import <nixpkgs> {}}: with pkgs; mkShell { buildInputs = [ meilisearch openssl overmind pkg-config postgresql rustup tmux ]; shellHook = '' export OVERMIND_PROCFILE=Procfile.dev export PGDATA="data.pg" export MEILI_DB_PATH="data.ms" ''; } Procfile.dev - this is used to start the background services I need, the format is "name: command" postgres: postgres -K /tmp meilisearch: meilisearch Then I can run `nix-shell` to load all the dependencies and `overmind start` to bring up Postgres + Meilisearch with ignored data dirs relative to the project folder. If any of these packages need to be pinned to a specific version you can do that with this tool[1]. That's pretty much it - with all the background services running you can load your monolith or microservices with cargo/npm/etc. It's all very nice and clean! [1]: https://lazamar.co.uk/nix-versions/ https://lazamar.co.uk/nix-versions/ - I actually just used this in a new shell.nix for my personal website to pin a version of Hugo from 2019 (when I last touched it) so that it would build again
zamalek 4y ago
> slow ass filesystem always ruined it in the past. Make sure that you're running a recent version of Lima, and use a 9p mount. Edit: by editing one of these yamls: https://github.com/lima-vm/lima/tree/master/examples https://github.com/lima-vm/lima/tree/master/examples
tracker1 4y ago
It's less than perfect and being worked on afaik... Main thing to avoid are any volume mounts to the host environment. Declare/use volume containers in your docker-compose.yaml, will help a lot, but still more sluggish than host/native. If you use VS Code the remoting extensions pack includes SSH, for my personal laptop, I'm usually using code to my linux desktop, and editing remotely... it's pretty slick. Code will run a "code server" in the remote environment, and is effectively just displaying the text view to you, the actual work is happening in the remote envornment. You can do similar with containers, or in windows+wsl.
- maherbeg 4y ago
  I'm thinking about giving this a try but using an iPad connected to a monitor setup for this. I rarely code on the go for personal use, so this could work pretty well...
  - tracker1 4y ago
    There are codium forks that will host a web server environment for you... easy enough to VPN home, or another trusted environment, and use a code(ium) server on your host.

by dchuk 4y ago

pbowyer 4y ago

Have you tried enabling Virtiofs in Docker Desktop? On my M1 Mac that sorted the performance for me for large PHP apps, and I haven't experienced the permissions problems that some people have (there is a test build that appears to fix those out now).

charrondev 4y ago
I have an M1 Pro MacBook and I turned on the 2 experimental settings and that sorted my performance issues. I think the new virtualization framework just came out of beta though so now it’s just VirtuoFS that needs to be enabled.

irusensei 4y ago

Canonical has a product of their own: https://multipass.run/ https://multipass.run/ They all use the underlying OS hypervisor so I’m not sure you can get perceivable performance out of one solution over the other.

todotask 4y ago
Sometime it can be buggy on macOS and quite slow to boot with Ubuntu VM, wish there is a cloud VM image.
- hda111 4y ago
  > wish there is a cloud VM image What do you mean? Multipass only runs Ubuntu server cloud images.

pickleMeTimbers 4y ago

We're using Multipass + Bravetools (https://github.com/bravetools/bravetools https://github.com/bravetools/bravetools) / Docker to build system/application containers. I believe multipass uses SSHFS (https://discourse.ubuntu.com/t/how-to-improve-mounts-performance-in-multipass/26638 https://discourse.ubuntu.com/t/how-to-improve-mounts-perform...) to mount filesystems between the host and the VM. Performance has been excellent.

hamandcheese 4y ago

Because docker for Mac has always kind of sucked, I’ve moved on to making nix-based development environments and I’ve been very pleased so far. Lately I’ve been trying https://devenv.sh/ https://devenv.sh/ and it works great! I haven’t tried it for ruby, though I have used vanilla nix shell for ruby projects before and it worked quite well after I over-rode GEM_PATH and GEM_HOME to the correct values.

nerdponx 4y ago
I've been using Colima lately and it's been a great experience. The only thing that didn't quite work was pushing an image to AWS ECR, but pulling images, building images (with BuildKit), and running containers all worked fine, and faster than Docker.
- hamandcheese 4y ago
  I've tried Colima too but it has always felt sluggish compared to local dev. I don't think any form of cross-host filesystem sharing could ever be fast enough.
- MuffinFlavored 4y ago
  isn’t Buildkit Docker?
  - MuffinFlavored 4y ago
    https://github.com/moby/buildkit https://github.com/moby/buildkit it looks like it is not, kind of?
    - krab 4y ago
      Moby is Docker.
  - nerdponx 4y ago
    Whatever it is, Colima seems to have faster image builds and faster container startup, which makes my life better.

bsnnkv 4y ago

I've recently switched to using nix-shell and overmind with a Procfile to officially retired docker-compose from all of my projects. Such a breath of fresh air. Like Mitchell Hashimoto (Hashicorp), I no longer do any development on macOS directly; I have a Linux VM where all my projects live and have their environments managed with a shell.nix file. Highly highly recommend it.

unboxingelf 4y ago
Would you elaborate on the setup? I recall seeing a comment or article where Mitchell mentioned this setup previously. I’ve been using Make + Docker for development on macOS for a long time now and perhaps it’s time for a change. I’ve looked at nix once or twice over the years but it never really clicked.
- bsnnkv 4y ago
  Sure, this is a super abbreviated version of my current setup for developing https://notado.app https://notado.app shell.nix - this brings the packages I need to develop into the $PATH when I run `nix-shell` in the repo {pkgs ? import <nixpkgs> {}}: with pkgs; mkShell { buildInputs = [ meilisearch openssl overmind pkg-config postgresql rustup tmux ]; shellHook = '' export OVERMIND_PROCFILE=Procfile.dev export PGDATA="data.pg" export MEILI_DB_PATH="data.ms" ''; } Procfile.dev - this is used to start the background services I need, the format is "name: command" postgres: postgres -K /tmp meilisearch: meilisearch Then I can run `nix-shell` to load all the dependencies and `overmind start` to bring up Postgres + Meilisearch with ignored data dirs relative to the project folder. If any of these packages need to be pinned to a specific version you can do that with this tool[1]. That's pretty much it - with all the background services running you can load your monolith or microservices with cargo/npm/etc. It's all very nice and clean! [1]: https://lazamar.co.uk/nix-versions/ https://lazamar.co.uk/nix-versions/ - I actually just used this in a new shell.nix for my personal website to pin a version of Hugo from 2019 (when I last touched it) so that it would build again

zamalek 4y ago

> slow ass filesystem always ruined it in the past. Make sure that you're running a recent version of Lima, and use a 9p mount. Edit: by editing one of these yamls: https://github.com/lima-vm/lima/tree/master/examples https://github.com/lima-vm/lima/tree/master/examples

tracker1 4y ago

It's less than perfect and being worked on afaik... Main thing to avoid are any volume mounts to the host environment. Declare/use volume containers in your docker-compose.yaml, will help a lot, but still more sluggish than host/native. If you use VS Code the remoting extensions pack includes SSH, for my personal laptop, I'm usually using code to my linux desktop, and editing remotely... it's pretty slick. Code will run a "code server" in the remote environment, and is effectively just displaying the text view to you, the actual work is happening in the remote envornment. You can do similar with containers, or in windows+wsl.

maherbeg 4y ago
I'm thinking about giving this a try but using an iPad connected to a monitor setup for this. I rarely code on the go for personal use, so this could work pretty well...
- tracker1 4y ago
  There are codium forks that will host a web server environment for you... easy enough to VPN home, or another trusted environment, and use a code(ium) server on your host.