3 ms·

Their benchmarks are based on a machine with 72GB of RAM and another with 128GB of RAM - not that it does matter, but I'm really curious if they're outliers or

Their benchmarks are based on a machine with 72GB of RAM and another with 128GB of RAM - not that it does matter, but I'm really curious if they're outliers or if I should bump my computer specs. That's a lot of Electron web apps running in parallel!

mixedCase 3y ago
Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason. If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.
- rakejake 3y ago
  Funnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM. But the 7950X is grossly underutilised, I don't know what to do with it.
  - lolinder 3y ago
    Shouldn't the quantized LLMs be using the CPU to good effect? I'd imagine your Mixtral is substantially faster than it would be on a weaker CPU.
    - rakejake 3y ago
      I have a 4080 with 16GB of VRAM. I experimented with llama.cpp by offloading layers onto GPU and doing the remaining on CPU. I found that it gives me max tokens/sec if I set it to 8 CPU cores as opposed to the 16 available on the 7950X. I guess beyond that, the bookkeeping between the cores might be taking up more time than it is worth.
  - mischief6 3y ago
    the answer i found is run gentoo and compile yocto projects.
- db48x 3y ago
  Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.
- yaantc 3y ago
  You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer. MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizer
pcthrowaway 3y ago
My understanding is that processor performance is more important for rust compilation than available memory, but ultimately the atrocious compilation times are why I struggled so much with Rust.
lolinder 3y ago
Definitely outliers. The Steam hardware survey [0] is obviously not a perfect measurement, but in its February results only ~3% had 64GB and only ~0.3% had more than that. I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB. [0] https://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey
- IshKebab 3y ago
  I have 32GB on my Linux laptop (plus 12GB swap) and regularly run out of RAM with just a handful of VSCode's open (it's generally the language servers using all the RAM) and a couple of dozen Firefox tabs (Firefox seems to be much worse than Chrome RAM-wise). If I run my actual work project, which needs a few gigs then it often just completely runs out of memory and hard reboots. Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows. On Linux I wish I had 64GB.
  - spookie 3y ago
    Have you tried zram for instance? There are a lot of things one can do to better utilise RAM on Linux. I believe Fedora comes ootb with a lot of things to make ram management quite good. Using another distro these days, but Fedora impressed me on that front. (edit: Try to use zram and zswap, check if your i/o scheduler is correct for your hardware, and increase your swapiness value. These should make your system behave better. For other performance improvements, use ananicy and irqbalance.)
    - IshKebab 3y ago
      Yes I have zram enabled. First thing I tried. Even if it is possible to fix this, it's still a horrible bug because you absolutely should not have to blindly tweak settings to stop a desktop computer from hard-rebooting when it runs out of RAM. I think this is a really fundamental problem on Linux though - it would require enormous changes and unprecedented cooperation between different groups for a proper fix. Much easier to say "you're holding it wrong".
nindalf 3y ago
I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/ https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance. So no, 72GB or 128GB makes no difference.
- drewtato 3y ago
  I agree that memory size isn't going to matter much, but memory speed should be noticeable, especially on large projects.
  - deleted 3y ago
    [deleted]
- denysvitali 3y ago
  Yes, maybe I didn't make it clear but I'm pretty sure this is not impacting for Rust. I was just wondering if it's normal to have these specs since my computer has probably "just" 32GB of RAM
lifthrasiir 3y ago
If you are like me and opening too many browser tabs (and other applications, of course), you need a lot of RAM. My desktop has only 48 GB of RAM right now and a typical VM usage is around 80--100 GB. I will probably go for 128 GB or possibly 192 GB of RAM for the next desktop. Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.

mixedCase 3y ago

Some people need it, some don't. When I upgraded to a 7950X I was unable to find an excuse to buy more than 32GB. "That one time a script compiled clang by default with LTO and OOMed" was not a good enough reason. If you use a lot of VMs, it comes in handy. If you do a lot of heavyweight compilation in parallel, same thing. But at least on Linux, popping 3 VS Code projects and 4 more browsers doesn't even hit 50% utilization.

rakejake 3y ago
Funnily enough, RAM I find I can use no matter how much I have, especially now that you have LLMs. I can run a quantised Mixtral 8x7B on 64GB RAM. But the 7950X is grossly underutilised, I don't know what to do with it.
- lolinder 3y ago
  Shouldn't the quantized LLMs be using the CPU to good effect? I'd imagine your Mixtral is substantially faster than it would be on a weaker CPU.
  - rakejake 3y ago
    I have a 4080 with 16GB of VRAM. I experimented with llama.cpp by offloading layers onto GPU and doing the remaining on CPU. I found that it gives me max tokens/sec if I set it to 8 CPU cores as opposed to the 16 available on the 7950X. I guess beyond that, the bookkeeping between the cores might be taking up more time than it is worth.
- mischief6 3y ago
  the answer i found is run gentoo and compile yocto projects.
db48x 3y ago
Yea, very workload dependent. Reposurgeon needed hundreds of gigabytes of ram to convert the GCC repository from SVN to Git, which was a lot of fun, but most of the time I would be fine with 32GB.
yaantc 3y ago
You can try running tests in parallel with the memory sanitizer (MSAN) enabled. MSAN uses a lot of memory, and on a 64 GB machine I have to reduce the amount of parallelism not to trigger Linux OoM killer. MSAN: https://github.com/google/sanitizers/wiki/MemorySanitizer

pcthrowaway 3y ago

My understanding is that processor performance is more important for rust compilation than available memory, but ultimately the atrocious compilation times are why I struggled so much with Rust.

lolinder 3y ago

Definitely outliers. The Steam hardware survey [0] is obviously not a perfect measurement, but in its February results only ~3% had 64GB and only ~0.3% had more than that. I recently upgraded to 64GB to run quantized LLM models, but when I'm not using an LLM I rarely use more than 16GB, much less 32GB. [0] https://store.steampowered.com/hwsurvey https://store.steampowered.com/hwsurvey

IshKebab 3y ago
I have 32GB on my Linux laptop (plus 12GB swap) and regularly run out of RAM with just a handful of VSCode's open (it's generally the language servers using all the RAM) and a couple of dozen Firefox tabs (Firefox seems to be much worse than Chrome RAM-wise). If I run my actual work project, which needs a few gigs then it often just completely runs out of memory and hard reboots. Never have the same problem on Windows even though I only have 16GB there. Feels like Linux is just really bad at memory management. That may be skewing the Steam survey since almost everyone there will be using Windows. On Linux I wish I had 64GB.
- spookie 3y ago
  Have you tried zram for instance? There are a lot of things one can do to better utilise RAM on Linux. I believe Fedora comes ootb with a lot of things to make ram management quite good. Using another distro these days, but Fedora impressed me on that front. (edit: Try to use zram and zswap, check if your i/o scheduler is correct for your hardware, and increase your swapiness value. These should make your system behave better. For other performance improvements, use ananicy and irqbalance.)
  - IshKebab 3y ago
    Yes I have zram enabled. First thing I tried. Even if it is possible to fix this, it's still a horrible bug because you absolutely should not have to blindly tweak settings to stop a desktop computer from hard-rebooting when it runs out of RAM. I think this is a really fundamental problem on Linux though - it would require enormous changes and unprecedented cooperation between different groups for a proper fix. Much easier to say "you're holding it wrong".

nindalf 3y ago

I benchmarked Rusts performance on 2, 4, 8 and 16 cores on several widely used crates (https://arewefastyet.pages.dev/ https://arewefastyet.pages.dev/). Performance generally improved with the number of cores, but memory made no difference. All of these crates topped out at around 1GB and additional memory didn’t improve performance. So no, 72GB or 128GB makes no difference.

drewtato 3y ago
I agree that memory size isn't going to matter much, but memory speed should be noticeable, especially on large projects.
- deleted 3y ago
  [deleted]
denysvitali 3y ago
Yes, maybe I didn't make it clear but I'm pretty sure this is not impacting for Rust. I was just wondering if it's normal to have these specs since my computer has probably "just" 32GB of RAM

lifthrasiir 3y ago

If you are like me and opening too many browser tabs (and other applications, of course), you need a lot of RAM. My desktop has only 48 GB of RAM right now and a typical VM usage is around 80--100 GB. I will probably go for 128 GB or possibly 192 GB of RAM for the next desktop. Rust builds, in comparison, don't actually reach that far, mainly because it is proportional to the maximum number of parallel jobs. In my experience, 48 GB was more than plenty for 8 parallel jobs if I wasn't doing anything else.