7 ms·

If you're routinely zeroing this much memory and the performance matters, you might benefit from idle-zeroing it. That is, when you need to zero the massive bl

If you're routinely zeroing this much memory and the performance matters, you might benefit from idle-zeroing it. That is, when you need to zero the massive block, just switch to a different block that has already been zeroed or partly-zeroed in the background. Whatever hasn't already been zeroed, finish synchronously. The background thread doing the zeroing would be scheduled with the lowest priority, so that it only runs when the system otherwise has nothing to do. At first, I thought you might just want to get fresh pages from the kernel (which are always zeroed), but this answer convinced me that might not actually be faster because of the overhead from syscalls and fiddling with virtual memory https://stackoverflow.com/questions/49896578/fastest-way-to-zero-pages-in-linux https://stackoverflow.com/questions/49896578/fastest-way-to-... . And Linux doesn't idle-zero or pre-zero pages (though I believe there's a switch to enable pre-zeroing for purposes of security hardening), so you're probably gonna end up with the OS doing a synchronous[1] zeroing anyway. [1] Synchronous from when you actually first write to each page. My understanding is that when your process gets new pages, they're all mapped to a special zero-page and set to copy-on-write. So there is still some efficiency here in theory: you don't have a long wait for the entire range to be zeroed all at once and you never have to zero pages that you don't modify.

fyp 7y ago
Is that how languages that always give you zeroed arrays (e.g., java) do it?
- deleted 7y ago
  [deleted]
- NohatCoder 7y ago
  There is no canonical answer to that question. Compilers wary wildly in implementation details.
- marcoperaza 7y ago
  I doubt it. Needing to zero ultra large chunks of memory often enough that you care about performance to this extent is a very niche scenario. If I had to guess, Java probably just zeroes the memory upon garbage collecting it or before handing it back out; fresh pages from the OS are already zeroed. But anything is possible. Maybe some big customer used this pattern and now Java detects it and does something fancy like what I suggested. But really, this is an unusual scenario for an application.
- MaxBarraclough 7y ago
  There's a brilliantly-named paper on this: Why Nothing Matters: The Impact of Zeroing. Efficient zeroing is something JVMs put some thought into. Well, with a notable exception, in HotSpot itself: We were able to substantially improve its performance by using memset() to perform the zeroing. [0] (PDF) http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225.9497&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225...
andrewprock 7y ago
Have you tried this and measured it? It feels like the overhead of having to use synchronization might eat up all your savings.
BeeOnRope 7y ago
It's an interesting idea, mostly applicable to long running processes (e.g,. a server) where the concept of "idle time" applies. For something like a process that starts up, does its job as quickly as possible and then stops, it is less obvious how to do it: you could certainly start another thread to do this, but if you are willing to add threading you are probably better off just trying to parallelize the underlying work, rather than doing zero async. The main downsides to this approach: 1) It is cache unfriendly: if you zero the pages synchronously immediately prior to use, you bring them into cache during the zeroing and then they are hot when you use them on the same thread. For idle zeroing this link is lost so you are likely to bring them into cache twice: once during the zeroing and once during the subsequent use. 2) On NUMA systems, doing the idling on some thread/CPU distinct from where you will use it is an anti-pattern: with the usual "first touch" allocation, the memory will be allocated on the node doing the zeroing, potentially different from the node doing the using - which could be an arbitrarily large penalty over time (as you keep using the remote memory for an arbitrarily long time) Note that (1) is a common confounding factor when measuring the cost of zeroing. E.g., if you have something like: zero(mem) do_work(mem) You might measure the zeroing taking 500 (time units) and the work taking 800, total of 1300. Lets say you find a clever way to totally avoid the zeroing (e.g., maybe do_work didn't actually need zeroed memory): you expect only the 800 time units of work to remain - but actually measure 1200: the zeroing more than just zeroing: it was bringing in all the memory into cache and suffering all the cache, TLB misses, etc. When you remove it, that work just moves over to the subsequent work. In some cases do_work alone can even be slower than the sum of zeroing and do_work (e.g, 1,500 time units) - because the interleaving of the work with cache misses can be less efficient in an MLP sense (because the work clogs up the OoO buffers so fewer memory requests fit in the window).
- marcoperaza 7y ago
  I don't think it's about long-running vs. short-running. That might be a heuristic, but what really matters is whether there is idle CPU time to be had between when you need large zeroed chunks of memory, or even if not, if you have some other performance characteristic you want (e.g. stable, consistent throughput) that would benefit from parallelizing the zeroing instead of having to periodically block.
gpvos 7y ago
In that case, why not request freshly zeroed pages from the operating system?
- marcoperaza 7y ago
  Because if you're needing to repeatedly free giant amounts of RAM, the OS probably doesn't have many zeroed pages and will need to zero them itself. Linux, at least, does not idle-zero, so if you want to take advantage of otherwise idle CPU time to do the zeroing, then you need to take matters into your own hands.

fyp 7y ago

Is that how languages that always give you zeroed arrays (e.g., java) do it?

deleted 7y ago
[deleted]
NohatCoder 7y ago
There is no canonical answer to that question. Compilers wary wildly in implementation details.
marcoperaza 7y ago
I doubt it. Needing to zero ultra large chunks of memory often enough that you care about performance to this extent is a very niche scenario. If I had to guess, Java probably just zeroes the memory upon garbage collecting it or before handing it back out; fresh pages from the OS are already zeroed. But anything is possible. Maybe some big customer used this pattern and now Java detects it and does something fancy like what I suggested. But really, this is an unusual scenario for an application.
MaxBarraclough 7y ago
There's a brilliantly-named paper on this: Why Nothing Matters: The Impact of Zeroing. Efficient zeroing is something JVMs put some thought into. Well, with a notable exception, in HotSpot itself: We were able to substantially improve its performance by using memset() to perform the zeroing. [0] (PDF) http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225.9497&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225...

andrewprock 7y ago

Have you tried this and measured it? It feels like the overhead of having to use synchronization might eat up all your savings.

BeeOnRope 7y ago

It's an interesting idea, mostly applicable to long running processes (e.g,. a server) where the concept of "idle time" applies. For something like a process that starts up, does its job as quickly as possible and then stops, it is less obvious how to do it: you could certainly start another thread to do this, but if you are willing to add threading you are probably better off just trying to parallelize the underlying work, rather than doing zero async. The main downsides to this approach: 1) It is cache unfriendly: if you zero the pages synchronously immediately prior to use, you bring them into cache during the zeroing and then they are hot when you use them on the same thread. For idle zeroing this link is lost so you are likely to bring them into cache twice: once during the zeroing and once during the subsequent use. 2) On NUMA systems, doing the idling on some thread/CPU distinct from where you will use it is an anti-pattern: with the usual "first touch" allocation, the memory will be allocated on the node doing the zeroing, potentially different from the node doing the using - which could be an arbitrarily large penalty over time (as you keep using the remote memory for an arbitrarily long time) Note that (1) is a common confounding factor when measuring the cost of zeroing. E.g., if you have something like: zero(mem) do_work(mem) You might measure the zeroing taking 500 (time units) and the work taking 800, total of 1300. Lets say you find a clever way to totally avoid the zeroing (e.g., maybe do_work didn't actually need zeroed memory): you expect only the 800 time units of work to remain - but actually measure 1200: the zeroing more than just zeroing: it was bringing in all the memory into cache and suffering all the cache, TLB misses, etc. When you remove it, that work just moves over to the subsequent work. In some cases do_work alone can even be slower than the sum of zeroing and do_work (e.g, 1,500 time units) - because the interleaving of the work with cache misses can be less efficient in an MLP sense (because the work clogs up the OoO buffers so fewer memory requests fit in the window).

marcoperaza 7y ago
I don't think it's about long-running vs. short-running. That might be a heuristic, but what really matters is whether there is idle CPU time to be had between when you need large zeroed chunks of memory, or even if not, if you have some other performance characteristic you want (e.g. stable, consistent throughput) that would benefit from parallelizing the zeroing instead of having to periodically block.

gpvos 7y ago

In that case, why not request freshly zeroed pages from the operating system?

marcoperaza 7y ago
Because if you're needing to repeatedly free giant amounts of RAM, the OS probably doesn't have many zeroed pages and will need to zero them itself. Linux, at least, does not idle-zero, so if you want to take advantage of otherwise idle CPU time to do the zeroing, then you need to take matters into your own hands.