4 ms·
288MB Cache. I wonder if we'll ever be given some control about how and what computations are cached. It seems like a lot of memory to leave to simple heuristic
by thomasahle 7y ago
288MB Cache. I wonder if we'll ever be given some control about how and what computations are cached. It seems like a lot of memory to leave to simple heuristics.
- TaylorAlexander 7y agoI’ve heard these “simple heuristics” are multi-megabyte neural networks these days. I believe I heard that claim in this video: https://youtu.be/ymcOLL2qEg8 https://youtu.be/ymcOLL2qEg8
- pas 7y agoUsually branch prediction is what ANNs are used for in CPUs. Caches usually operate as complicated pseudo-LRUs. https://stackoverflow.com/questions/22597324/what-cache-invalidation-algorithms-are-used-in-actual-cpu-caches https://stackoverflow.com/questions/22597324/what-cache-inva... It's possible that Tesla does something different that makes sense in their specific situation, but general purpose CPUs - as far as I know - don't use NNs for this.
- Sesse__ 7y agoRyzen and some ARMs use perceptrons for branch prediction. It's essentially a single-layer neural network with no nonlinearity at the end. Multi-megabyte NNs for a normal branch predictor would be impossible; way too high latency to be useful.
- szatkus 7y agoIt's 32MB L3 per die and 512KB L2 per core, they sum them for marketing effect. Effectively one core can access "only" 32.5MB.
- szatkus 7y agoErrata: as on one die there are two CCXs, one core can access 16.5MB
- neysofu 7y agoDidn't Intel try precisely that with Itanium, only for it to fail spectacularly?
- masklinn 7y agoWasn’t itanium mostly the reverse, expectations that compiler heuristics would take care of the architectural complexity?
- jzwinck 7y agoSuch control has been available for many years. On x86 you have "non-temporal" load and store instructions [1] and the Cache Disable bit in Control Register 0 [2] which can be used to suggest which bytes are not suitable for caching. C also has posix_madvise() [3] which is somewhat relevant. [1] https://stackoverflow.com/questions/37070/what-is-the-meaning-of-non-temporal-memory-accesses-in-x86 https://stackoverflow.com/questions/37070/what-is-the-meanin... [2] https://en.wikipedia.org/wiki/Control_register https://en.wikipedia.org/wiki/Control_register [3] http://man7.org/linux/man-pages/man3/posix_madvise.3.html http://man7.org/linux/man-pages/man3/posix_madvise.3.html
- bob1029 7y agoI feel the cache in these modern CPUs is being used extremely efficiently. If they have a branch predictor that is 99% accurate, I have faith in letting the same engineers manage my cache eviction strategy. The extreme scope of modern OoO execution strategies would probably make detailed CPU cache management more of a liability than an asset. One simple policy would be to try to ensure your application is as small as reasonably possible. If you can fit your entire executable image in a fraction of your L3 you are probably sitting in a really good spot.