5 ms·

I have the 3970X and use it with Ubuntu. It's fun to have a lot of cores, but in the beginning it's highlighted some areas where open-source programs aren't ass

I have the 3970X and use it with Ubuntu. It's fun to have a lot of cores, but in the beginning it's highlighted some areas where open-source programs aren't assuming you're going to use that many cores (similar to what they talk about for Windows). So you see OOM errors with make, see program crashes from defaults being too low (such as creating too many Postgres connections), things like that. Not taking away from those developers, just sharing my experience. When you do get something that works really well (the newest version of Julia makes it easy to write multithreaded code), it's hilarious running htop and seeing 64 threads of green running.

clSTophEjUdRanu 7y ago
That is definitely not the CPUs problem to worry about. Operating systems need to catch up.
- randyzwitch 7y ago
  Yes, that was generally my point. Even if available code already took advantage of multiple cores, my experience has shown that currently these chips bring out some latent assumptions on what hardware the code will be running on (whether that's OOM or UX or whatever)
- rbanffy 7y ago
  That's why we have things like Rust - so we can write multi-threaded apps without shooting ourselves on the foot.
zeotroph 7y ago
> htop / 64 threads It sure is fun to see the entire top half of htop being taken up by individual CPU bars, but this is another program whose assumptions have been overtaken by the current silicon: At some point it is not that helpful anymore and makes looking at the processes themselves harder. Sadly the development of htop seems to have stalled, patches/PRs for that and many other issues remain unmerged and unanswered.
- dijit 7y ago
  you can easily change the CPU bars to be quarter sized. If you press 'F2' and look at the available meters you should see half and quarter sizes.
  - blattimwind 7y ago
    Even then -- https://gamozolabs.github.io/assets/afl_256thread.png https://gamozolabs.github.io/assets/afl_256thread.png Note that this CPU has 128 threads, so half of that picture. For my usual terminal size I might get 0-2 processes listed below that...
    - irishcoffee 7y ago
      I’ve found htop becomes less useful after about an 8 core count. Is there a reason you’d run htop over top in a normal use case?
      - penagwin 7y ago
        Htop is IMO better overall with more helpful coloring, filtering, etc.
    - thinkmassive 7y ago
      You can replace all of those with a single Avg bar.
      - philsnow 7y ago
        At this number of processors, it starts to make sense to plot min/median/mean/90%ile/99%ile/max instead of gauges for each cpu. I would love to have this kind of problem :) _edit_: or just all the deciles and 1%ile + 99%ile
    - 3fe9a03ccd14ca5 7y ago
      What in the world are you doing when that picture was taken??
StillBored 7y ago
I've got a 256 thread machine under my desk at the moment, I find myself frequently disabling threads because i'm out of memory. I don't think people fully grasp that lots of applications memory requirements scale with the number of threads. Just running gcc against C make files can burn 1/2 a GB a process, multiply that by the 8 cores most people have and 16G of ram leaves plenty of memory for Firefox/etc. OTOH, 128GB of ram in a 256 thread machine, is borderline out of ram. Build a project where its more like 2G of ram per process (looking at a lot of recent big public google projects) and its more like I need 1/2 a TB and a bunch of swap to keep the project from OOMing. So at this point I might use that as a general rule, 2G per thread. So your looking at another ~$1.5k+ or so in RAM for this machine for most purposes.
- teambayleaf 7y ago
  I second this. My experience with many core machines (32-64 cores) is kinda dissappointing so far. Yes, running "make -j" on these machines feel amazing at first, but you will soon either 1) run out of memory, or 2) notice that you don't need so many cores anyway due to build dependencies.
  - emn13 7y ago
    Hmm, my experience has been resolutely great; we just upgraded to 64 thread 3970x's with 256GB of RAM, and pretty much everything worked smoothly out of the box; I'm very pleasantly surprised by the lack of tuning necessary. We did do the math and decided that for us 4GB per thread is likely safe up front; 2GB per thread might have been pushing it. Another reason to avoid the 3990x (for us) was the tricky scaling of the previous generation's 2990WX; We don't have faith that all of our code would run well on the more memory-channel constrained machine. I mean, if you're just running a single fairly small build, sure, it's likely overkill...
- arvinsim 7y ago
  Interesting heuristic. That means my 6 core-12 thread CPU will need to have 24GB of RAM. I am on 32GB so it's cool.

clSTophEjUdRanu 7y ago

That is definitely not the CPUs problem to worry about. Operating systems need to catch up.

randyzwitch 7y ago
Yes, that was generally my point. Even if available code already took advantage of multiple cores, my experience has shown that currently these chips bring out some latent assumptions on what hardware the code will be running on (whether that's OOM or UX or whatever)
rbanffy 7y ago
That's why we have things like Rust - so we can write multi-threaded apps without shooting ourselves on the foot.

zeotroph 7y ago

> htop / 64 threads It sure is fun to see the entire top half of htop being taken up by individual CPU bars, but this is another program whose assumptions have been overtaken by the current silicon: At some point it is not that helpful anymore and makes looking at the processes themselves harder. Sadly the development of htop seems to have stalled, patches/PRs for that and many other issues remain unmerged and unanswered.

dijit 7y ago
you can easily change the CPU bars to be quarter sized. If you press 'F2' and look at the available meters you should see half and quarter sizes.
- blattimwind 7y ago
  Even then -- https://gamozolabs.github.io/assets/afl_256thread.png https://gamozolabs.github.io/assets/afl_256thread.png Note that this CPU has 128 threads, so half of that picture. For my usual terminal size I might get 0-2 processes listed below that...
  - irishcoffee 7y ago
    I’ve found htop becomes less useful after about an 8 core count. Is there a reason you’d run htop over top in a normal use case?
    - penagwin 7y ago
      Htop is IMO better overall with more helpful coloring, filtering, etc.
  - thinkmassive 7y ago
    You can replace all of those with a single Avg bar.
    - philsnow 7y ago
      At this number of processors, it starts to make sense to plot min/median/mean/90%ile/99%ile/max instead of gauges for each cpu. I would love to have this kind of problem :) _edit_: or just all the deciles and 1%ile + 99%ile
  - 3fe9a03ccd14ca5 7y ago
    What in the world are you doing when that picture was taken??

StillBored 7y ago

I've got a 256 thread machine under my desk at the moment, I find myself frequently disabling threads because i'm out of memory. I don't think people fully grasp that lots of applications memory requirements scale with the number of threads. Just running gcc against C make files can burn 1/2 a GB a process, multiply that by the 8 cores most people have and 16G of ram leaves plenty of memory for Firefox/etc. OTOH, 128GB of ram in a 256 thread machine, is borderline out of ram. Build a project where its more like 2G of ram per process (looking at a lot of recent big public google projects) and its more like I need 1/2 a TB and a bunch of swap to keep the project from OOMing. So at this point I might use that as a general rule, 2G per thread. So your looking at another ~$1.5k+ or so in RAM for this machine for most purposes.

teambayleaf 7y ago
I second this. My experience with many core machines (32-64 cores) is kinda dissappointing so far. Yes, running "make -j" on these machines feel amazing at first, but you will soon either 1) run out of memory, or 2) notice that you don't need so many cores anyway due to build dependencies.
- emn13 7y ago
  Hmm, my experience has been resolutely great; we just upgraded to 64 thread 3970x's with 256GB of RAM, and pretty much everything worked smoothly out of the box; I'm very pleasantly surprised by the lack of tuning necessary. We did do the math and decided that for us 4GB per thread is likely safe up front; 2GB per thread might have been pushing it. Another reason to avoid the 3990x (for us) was the tricky scaling of the previous generation's 2990WX; We don't have faith that all of our code would run well on the more memory-channel constrained machine. I mean, if you're just running a single fairly small build, sure, it's likely overkill...
arvinsim 7y ago
Interesting heuristic. That means my 6 core-12 thread CPU will need to have 24GB of RAM. I am on 32GB so it's cool.