3 ms·

I've got a 256 thread machine under my desk at the moment, I find myself frequently disabling threads because i'm out of memory. I don't think people fully gra

I've got a 256 thread machine under my desk at the moment, I find myself frequently disabling threads because i'm out of memory. I don't think people fully grasp that lots of applications memory requirements scale with the number of threads. Just running gcc against C make files can burn 1/2 a GB a process, multiply that by the 8 cores most people have and 16G of ram leaves plenty of memory for Firefox/etc. OTOH, 128GB of ram in a 256 thread machine, is borderline out of ram. Build a project where its more like 2G of ram per process (looking at a lot of recent big public google projects) and its more like I need 1/2 a TB and a bunch of swap to keep the project from OOMing. So at this point I might use that as a general rule, 2G per thread. So your looking at another ~$1.5k+ or so in RAM for this machine for most purposes.

teambayleaf 7y ago
I second this. My experience with many core machines (32-64 cores) is kinda dissappointing so far. Yes, running "make -j" on these machines feel amazing at first, but you will soon either 1) run out of memory, or 2) notice that you don't need so many cores anyway due to build dependencies.
- emn13 7y ago
  Hmm, my experience has been resolutely great; we just upgraded to 64 thread 3970x's with 256GB of RAM, and pretty much everything worked smoothly out of the box; I'm very pleasantly surprised by the lack of tuning necessary. We did do the math and decided that for us 4GB per thread is likely safe up front; 2GB per thread might have been pushing it. Another reason to avoid the 3990x (for us) was the tricky scaling of the previous generation's 2990WX; We don't have faith that all of our code would run well on the more memory-channel constrained machine. I mean, if you're just running a single fairly small build, sure, it's likely overkill...
arvinsim 7y ago
Interesting heuristic. That means my 6 core-12 thread CPU will need to have 24GB of RAM. I am on 32GB so it's cool.

teambayleaf 7y ago

I second this. My experience with many core machines (32-64 cores) is kinda dissappointing so far. Yes, running "make -j" on these machines feel amazing at first, but you will soon either 1) run out of memory, or 2) notice that you don't need so many cores anyway due to build dependencies.

emn13 7y ago
Hmm, my experience has been resolutely great; we just upgraded to 64 thread 3970x's with 256GB of RAM, and pretty much everything worked smoothly out of the box; I'm very pleasantly surprised by the lack of tuning necessary. We did do the math and decided that for us 4GB per thread is likely safe up front; 2GB per thread might have been pushing it. Another reason to avoid the 3990x (for us) was the tricky scaling of the previous generation's 2990WX; We don't have faith that all of our code would run well on the more memory-channel constrained machine. I mean, if you're just running a single fairly small build, sure, it's likely overkill...

arvinsim 7y ago

Interesting heuristic. That means my 6 core-12 thread CPU will need to have 24GB of RAM. I am on 32GB so it's cool.