8 ms·

If you found this unusable, you probably haven't had to work with the Virtualbox-backed version of Docker for Mac :-) Have you considered storing the npm_modul

If you found this unusable, you probably haven't had to work with the Virtualbox-backed version of Docker for Mac :-) Have you considered storing the npm_modules and running webpack on a tmpfs/volume (so that it doesn't have to go through the shared FS layers) and only copying the end product to a shared volume?

girvo 6y ago
I used to run my own Xhyve based VM just to avoid the brutal VirtualBox Docker system on macOS. Man that’s some bad memories. The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today
- llimllib 6y ago
  > Big node_module directories can still cause pain is there any other kind?
  - booi 6y ago
    Bigger node_modules directories
derefr 6y ago
Creating and mounting a loop-volume within the HGFS and then running NPM in there, would probably also be helpful. It certainly is when wrangling a directory with lots of tiny inodes over SMB. (It’s also what Apple themselves do for running Time Machine over SMB: they create a sparsebundle image on the remote and mount it locally, and write to that.) (The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)
tshaddox 6y ago
Yep, using Webpack in Docker on a Mac is essentially unusable. I see around a 5-10x slowdown in Webpack builds. Using docker-sync seems to help the CPU out a bit, so my MacBook Pro doesn't do the jet engine space heater routine quite as much, but the performance is still nearly unusable. https://docker-sync.readthedocs.io/en/latest/ https://docker-sync.readthedocs.io/en/latest/
- aguilarm 6y ago
  webpack can be made use-able relatively easily if you do not mount node modules over the shared file system. I've been doing this for quite awhile with a volumes declaration in docker-compose that looks like this (running nextjs, assuming /usr/src/app is where your dockerfile has your node stuff): (on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/ and then in the top level volumes key defining node_modules and next_artifacts as blank/default. That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.
  - tshaddox 6y ago
    Interesting. I have always had the same docker-compose setup with a separate node_modules volume inside Docker, but I've always still had the jet engine space heater issue.
- bdcravens 6y ago
  I've been able to improve performance in docker-sync by adding high-churn folders that I don't care about to sync_excludes
GordonS 6y ago
Virtualbox-backed Docker on Windows was my first introduction to Docker - it literally put me off touching Docker for 1-2 years! After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!
mbrumlow 6y ago
Or maybe we can realize that npm and webpack are both trash. And we should stop embracing them and move web dev away from this mess. Don't get me wrong. They were good workaround years ago. But I hope we can do better.

girvo 6y ago

I used to run my own Xhyve based VM just to avoid the brutal VirtualBox Docker system on macOS. Man that’s some bad memories. The newer Docker for Mac is better, but file system perf still could be massively improved. Big node_module directories can still cause pain even today

llimllib 6y ago
> Big node_module directories can still cause pain is there any other kind?
- booi 6y ago
  Bigger node_modules directories

derefr 6y ago

Creating and mounting a loop-volume within the HGFS and then running NPM in there, would probably also be helpful. It certainly is when wrangling a directory with lots of tiny inodes over SMB. (It’s also what Apple themselves do for running Time Machine over SMB: they create a sparsebundle image on the remote and mount it locally, and write to that.) (The efficient thing about this class of solutions, if you’re wondering, is that the client mounting the loop-image ends up owning + managing the loop-image’s internal filesystem’s metadata within its own local disk cache, such that it can coalesce filesystem metadata writes and only push a new copy of the disk-image blocks backing the filesystem metadata after a potentially-huge number of changes. With a network/host-guest filesystem, meanwhile, every filesystem metadata change must become its own synchronous message to the host, to be pushed to the host’s filesystem driver for linearization, so it can succeed or fail relative to other things going on within the host.)

tshaddox 6y ago

Yep, using Webpack in Docker on a Mac is essentially unusable. I see around a 5-10x slowdown in Webpack builds. Using docker-sync seems to help the CPU out a bit, so my MacBook Pro doesn't do the jet engine space heater routine quite as much, but the performance is still nearly unusable. https://docker-sync.readthedocs.io/en/latest/ https://docker-sync.readthedocs.io/en/latest/

aguilarm 6y ago
webpack can be made use-able relatively easily if you do not mount node modules over the shared file system. I've been doing this for quite awhile with a volumes declaration in docker-compose that looks like this (running nextjs, assuming /usr/src/app is where your dockerfile has your node stuff): (on service definition): volumes: - .:/usr/src/app:cached - node_modules:/usr/src/app/node_modules/ - next_artifacts:/usr/src/app/.next/ and then in the top level volumes key defining node_modules and next_artifacts as blank/default. That means I mount everything except node modules and the build artifacts so the shared filesystem does a LOT less work trying to sync stuff. The downside, of course, is that I need to run npm commands both inside the container and outside if i want them in my IDE. A fair trade for decent performance. That setup is still not as fast as native but definitely usable and does not send my machine into space header mode much more than normal usage.
- tshaddox 6y ago
  Interesting. I have always had the same docker-compose setup with a separate node_modules volume inside Docker, but I've always still had the jet engine space heater issue.
bdcravens 6y ago
I've been able to improve performance in docker-sync by adding high-churn folders that I don't care about to sync_excludes

GordonS 6y ago

Virtualbox-backed Docker on Windows was my first introduction to Docker - it literally put me off touching Docker for 1-2 years! After eventually falling in love with it on Windows and Linux, I later tried it on MacOS... oh my. Not fun!

mbrumlow 6y ago

Or maybe we can realize that npm and webpack are both trash. And we should stop embracing them and move web dev away from this mess. Don't get me wrong. They were good workaround years ago. But I hope we can do better.