4 ms·

Put everything in contiguous memory and load it, it'll be in cache

IgorPartola 5y ago
That not a great solution. My working data set is likely larger than 192MB. But my application is much more likely to know what data is in the hot path than the CPU is to guess it. I might want to put my DB index into this cache, or I might want to put only a part of it. I might want to preload the data then work on it for a bit, then load the next chunk. I might want one core preparing the data in RAM while another core is getting ready to work on it in this cache. Essentially I want full access to it independent of the main RAM because I can do a lot more with that. Think of this as the difference between RAM and disk: is RAM only useful as a cache to what’s on disk?
- vlovich123 5y ago
  We know both positions are correct in practice. Most automatic prediction/management algorithms perform better on 90% of the code that’s run on them. However, careful performance tuning over time by experts can outperform generic solutions, sometimes dramatically. I generally err on the side of “I’m not as smart as I think I am” in these kinds of discussions. It’s not that I can never do it. I’m sure if I studied it a bit I could. The reason is that 99% of the lines of code I’ve ever written don’t warrant that kind of attention. I’m sure there are uses like when you’re writing some core fundamental algorithm in the fields of compression, crypto algorithm, hash, video processing, etc etc etc. I don’t work in those spaces though so the benefit is much more marginal. It’s possible 192mb is enough to start needing some explicit memory management. It makes the coding model much more complex though. And in a greedy software system where your code isn’t the only one running, such complexity doesn’t necessarily net overall wins. It’s the reason we have drivers and OSes even though we started with each piece of content bundling explicit HW support (at much better perf generally)

That not a great solution. My working data set is likely larger than 192MB. But my application is much more likely to know what data is in the hot path than the CPU is to guess it. I might want to put my DB index into this cache, or I might want to put only a part of it. I might want to preload the data then work on it for a bit, then load the next chunk. I might want one core preparing the data in RAM while another core is getting ready to work on it in this cache. Essentially I want full access to it independent of the main RAM because I can do a lot more with that. Think of this as the difference between RAM and disk: is RAM only useful as a cache to what’s on disk?

vlovich123 5y ago
We know both positions are correct in practice. Most automatic prediction/management algorithms perform better on 90% of the code that’s run on them. However, careful performance tuning over time by experts can outperform generic solutions, sometimes dramatically. I generally err on the side of “I’m not as smart as I think I am” in these kinds of discussions. It’s not that I can never do it. I’m sure if I studied it a bit I could. The reason is that 99% of the lines of code I’ve ever written don’t warrant that kind of attention. I’m sure there are uses like when you’re writing some core fundamental algorithm in the fields of compression, crypto algorithm, hash, video processing, etc etc etc. I don’t work in those spaces though so the benefit is much more marginal. It’s possible 192mb is enough to start needing some explicit memory management. It makes the coding model much more complex though. And in a greedy software system where your code isn’t the only one running, such complexity doesn’t necessarily net overall wins. It’s the reason we have drivers and OSes even though we started with each piece of content bundling explicit HW support (at much better perf generally)