6 ms·

What sort of workloads are people doing where the filesystem access is limiting them? I develop python web apps on a mac and use dockerized postgres and a docke

What sort of workloads are people doing where the filesystem access is limiting them? I develop python web apps on a mac and use dockerized postgres and a dockerized flask app. I don't seem to experience any noticeable issues. When I am developing I mount the source code directory as a volume so code edits are synced live into the running docker container. I also develop frontends using vue, managed by npm. In my experience this doesnt need to be dockerized since npm installs everything in a subdirectory per project. Is there a benefit to running this as a dockerized app?

xtracto 4y ago
Magento in docker on Mac is horrible for this reason.
deleted 4y ago
[deleted]
taf2 4y ago
nginx is pretty unusable via docker - I guess file system cache is the issue…
- osrec 4y ago
  You can turn caching off, I believe.
jeffrom 4y ago
Workloads with a lot of files, for example a large elixir web app with hot reload / fswatch enabled, have upwards of 20 second page load time. More than enough to mess up my flow.
a_t48 4y ago
I work on a _large_ C++ codebase on Linux. If I'm on my main (Linux) machine, things are fine, bind mounts are okay. If I'm stuck on my MacBook then compilation performance is...bad. I suspect it's due to heavy filesystem access from the compiler (reading source, writing object files, etc). At some point I need to confirm this by copying in my source directory.
_-____-_ 4y ago
[dead]
pmontra 4y ago
A customer's Mac with a M1 is only 50% faster than my Intel laptop from 2014 at running Rails tests, because they run in a docker container: 50s vs 75s. The difference between the two machines should be much more than that (CPU, RAM, data bus, etc.)
- brodock 4y ago
  You should try running your dependencies on docker but ruby on the host machine.
  - pmontra 4y ago
    I'd be fine to run Ruby with asdf or rvm on my Linux laptop. I'm also fine to run it in a docker container. Performance is basically the same for me. The choice was made by my customer and it's them using Macs. They deploy in Linux containers though so that's probably why they accept not using all the performance of their hardware: same environment for production and development, no surprises.
zeta0134 4y ago
One issue I ran into at my previous employer was pylint on a large Python codebase. Pylint is slow on the best of days, but the difference on an M1 Mac running under docker (to standardize the version and settings across the team) was something like 10x as slow; several *minutes* to lint the codebase, which we absolutely required before code could be checked in. It finally got a lot better when VirtioFS came out, which, when enabled on an arm64 image, sped up filesystem access dramatically; suddenly my lints were taking seconds again.
- nyrikki 4y ago
  Are your Docker images x86_64? On an M1 Mac x86_64 images run under qemu which is very slow, If an ARM64 image is available it should run a bit faster.
  - nijave 4y ago
    Oh man x86 is so slow it's insane. If you're using a non-ARM base image Docker will happily run the x86 variant for you automatically. Unsurprisingly, running an x86 VM inside an ARM VM on a laptop is very very slow
- nikanj 4y ago
  Are you linting the whole codebase, or just files changed in the commit?
dgunay 4y ago
I tried running our Golang mock generator through Docker on an M1 Mac and it was much slower than running it directly on the host. Probably since it reads every file in the codebase and writes out a file for every interface it finds.
- latchkey 4y ago
  mockery? That thing runs super slow just on its own.
mianos 4y ago
Mentioned in the article, node packages that create tens of thousands, sometimes hundreds of thousands of files.
Arbortheus 4y ago
My company has a large python repo. Anyone that develops in this repo on a Mac is considered a sadist because… Running Pylint on a Linux machine in docker: 3 hours from no cache Running Pylint on a Mac in docker: 9+ hours from no cache unless VirtioFS is used, which makes it closer to 4 hours.

xtracto 4y ago

Magento in docker on Mac is horrible for this reason.

deleted 4y ago

[deleted]

taf2 4y ago

nginx is pretty unusable via docker - I guess file system cache is the issue…

osrec 4y ago
You can turn caching off, I believe.

jeffrom 4y ago

Workloads with a lot of files, for example a large elixir web app with hot reload / fswatch enabled, have upwards of 20 second page load time. More than enough to mess up my flow.

a_t48 4y ago

I work on a _large_ C++ codebase on Linux. If I'm on my main (Linux) machine, things are fine, bind mounts are okay. If I'm stuck on my MacBook then compilation performance is...bad. I suspect it's due to heavy filesystem access from the compiler (reading source, writing object files, etc). At some point I need to confirm this by copying in my source directory.

_-____-_ 4y ago

[dead]

pmontra 4y ago

A customer's Mac with a M1 is only 50% faster than my Intel laptop from 2014 at running Rails tests, because they run in a docker container: 50s vs 75s. The difference between the two machines should be much more than that (CPU, RAM, data bus, etc.)

brodock 4y ago
You should try running your dependencies on docker but ruby on the host machine.
- pmontra 4y ago
  I'd be fine to run Ruby with asdf or rvm on my Linux laptop. I'm also fine to run it in a docker container. Performance is basically the same for me. The choice was made by my customer and it's them using Macs. They deploy in Linux containers though so that's probably why they accept not using all the performance of their hardware: same environment for production and development, no surprises.

zeta0134 4y ago

One issue I ran into at my previous employer was pylint on a large Python codebase. Pylint is slow on the best of days, but the difference on an M1 Mac running under docker (to standardize the version and settings across the team) was something like 10x as slow; several *minutes* to lint the codebase, which we absolutely required before code could be checked in. It finally got a lot better when VirtioFS came out, which, when enabled on an arm64 image, sped up filesystem access dramatically; suddenly my lints were taking seconds again.

nyrikki 4y ago
Are your Docker images x86_64? On an M1 Mac x86_64 images run under qemu which is very slow, If an ARM64 image is available it should run a bit faster.
- nijave 4y ago
  Oh man x86 is so slow it's insane. If you're using a non-ARM base image Docker will happily run the x86 variant for you automatically. Unsurprisingly, running an x86 VM inside an ARM VM on a laptop is very very slow
nikanj 4y ago
Are you linting the whole codebase, or just files changed in the commit?

dgunay 4y ago

I tried running our Golang mock generator through Docker on an M1 Mac and it was much slower than running it directly on the host. Probably since it reads every file in the codebase and writes out a file for every interface it finds.

latchkey 4y ago
mockery? That thing runs super slow just on its own.

mianos 4y ago

Mentioned in the article, node packages that create tens of thousands, sometimes hundreds of thousands of files.

Arbortheus 4y ago

My company has a large python repo. Anyone that develops in this repo on a Mac is considered a sadist because… Running Pylint on a Linux machine in docker: 3 hours from no cache Running Pylint on a Mac in docker: 9+ hours from no cache unless VirtioFS is used, which makes it closer to 4 hours.