4 ms·

The trick with scalable threading is to make the behaviour as independent as is possible. Including memory allocations. “jemalloc” been something worth looking

The trick with scalable threading is to make the behaviour as independent as is possible. Including memory allocations. “jemalloc” been something worth looking at. It’s better to follow the scatter/gather model of MPI (don’t use that) in the same process and do as much amortisation in each thread as possible before collation. When it comes to I/O (with spinning rust), a measured rule is to use twice the cores to account for seek time and use big writes/reads rather than small ones. This can saturate the disks so do measure it. I spent about ten years worrying about these kind of issues on enterprise storage in C++.

nyanpasu64 3y ago
Would snmalloc (https://news.ycombinator.com/item?id=37851210 https://news.ycombinator.com/item?id=37851210) help with scalable threading or not? It claims to be better at allocating memory on a producer and freeing on a consumer thread, and "Freeing memory in a different thread to initially allocated it, does not take any locks and instead uses a novel message passing scheme to return the memory to the original allocator, where it is recycled. This enables 1000s of remote deallocations to be performed with only a single atomic operation enabling great scaling with core count."
- vlovich123 3y ago
  It’s a shame that they don’t compare against mimalloc which is another Microsoft project and it’s unclear which tcmalloc they are comparing against (the gperftools one is stale and performs worse than the current standalone release)
- OnlyMortal 3y ago
  I’ll offer that if you have a long lasting thread, you might consider allocating big blocks to it and having a “pool” it can grab memory from. In C++ you can use a shared pointer to release it back to the pool. This avoids contention in user space. It also reduces fragmentation. You can also bound the memory usage by blocking until memory is free. If memory serves, boost C++ has some code to help there though I did it myself.

nyanpasu64 3y ago

Would snmalloc (https://news.ycombinator.com/item?id=37851210 https://news.ycombinator.com/item?id=37851210) help with scalable threading or not? It claims to be better at allocating memory on a producer and freeing on a consumer thread, and "Freeing memory in a different thread to initially allocated it, does not take any locks and instead uses a novel message passing scheme to return the memory to the original allocator, where it is recycled. This enables 1000s of remote deallocations to be performed with only a single atomic operation enabling great scaling with core count."

vlovich123 3y ago
It’s a shame that they don’t compare against mimalloc which is another Microsoft project and it’s unclear which tcmalloc they are comparing against (the gperftools one is stale and performs worse than the current standalone release)
OnlyMortal 3y ago
I’ll offer that if you have a long lasting thread, you might consider allocating big blocks to it and having a “pool” it can grab memory from. In C++ you can use a shared pointer to release it back to the pool. This avoids contention in user space. It also reduces fragmentation. You can also bound the memory usage by blocking until memory is free. If memory serves, boost C++ has some code to help there though I did it myself.