3 ms·

"the lesson learned is mostly that it [scratchpad memory] ends up chronically underutilized. A cache speeds up nearly all code written for any system" I recall

"the lesson learned is mostly that it [scratchpad memory] ends up chronically underutilized. A cache speeds up nearly all code written for any system" I recall that when that technology came out, there was a heavy marketing stunt involved. For the boast's sake, performance had to be conjured. And as seamlessly as possible. And as noticeable as possible. That really affected the design development, and not in a good way. The problem (in what I quoted) here is the poor measurement base, the wrong comparison base, and what is counted or discounted as value. So that cache really speeds up nearly all the code written for any system. Such a flat model does not, however, make much distinction between a memory hogging application that might not require performance and some server process that has to be up and running with as little lag as possible. What is counted there in the terms of cache performance really? That 0.1% (that someone bothered to hand optimize the memory access patterns for) might be the only one really needing that cache. That, however, is not how you sell technology to the public, isn't it? You have to show them a general speed-up, even when it doesn't matter. You have to show them that the same application that used to do something in a given period of time does it now in less. That's how you sell. But that's not how you develop, hopefully.

tveita 10y ago
> That 0.1% (that someone bothered to hand optimize the memory access patterns for) might be the only one really needing that cache. I am going to claim that it isn't. I think you underestimate the amount of speedup the cache gives you. Few applications require performance so little that they will tolerate a 50-100 times slowdown. Even if they could, who's to say the right 0.1% ends up optimized? You add a new function without profiling it, or a user has a slightly different use case than you do, and suddenly the hot path is waiting for main memory in every loop. This was all decided a long time ago, general purpose caches won and scratch memory lost. > You have to show them that the same application that used to do something in a given period of time does it now in less. That's how you sell. Yes, that sounds like a swell idea. What is wrong with that?
- restalis 10y ago
  "Few applications require performance so little that they will tolerate a 50-100 times slowdown." Applications and users had or still have to wait a lot more than 100 times access slowdown when it gets to trashing (excessive number of page faults), memory swapping, or simply normal waits for some slow actions. "Even if they could, who's to say the right 0.1% ends up optimized?" I haven't said it's about only 0.1%, and that it has to be dictated by some policy. For all I care, even if they all would be manually cache managed, I don't expect the result to be worse than the caching mechanism currently employed, which works without really knowing what's going on. "This was all decided a long time ago, general purpose caches won and scratch memory lost." A long time ago they had a different world. Their reasoning might have been true back then. We are creatures of reason, however, aren't we? "Yes, that sounds like a [sell] idea. What is wrong with that?" Salesmanship and the engineering of future tech have at their cores different objectives. At least in my book.

tveita 10y ago

> That 0.1% (that someone bothered to hand optimize the memory access patterns for) might be the only one really needing that cache. I am going to claim that it isn't. I think you underestimate the amount of speedup the cache gives you. Few applications require performance so little that they will tolerate a 50-100 times slowdown. Even if they could, who's to say the right 0.1% ends up optimized? You add a new function without profiling it, or a user has a slightly different use case than you do, and suddenly the hot path is waiting for main memory in every loop. This was all decided a long time ago, general purpose caches won and scratch memory lost. > You have to show them that the same application that used to do something in a given period of time does it now in less. That's how you sell. Yes, that sounds like a swell idea. What is wrong with that?

restalis 10y ago
"Few applications require performance so little that they will tolerate a 50-100 times slowdown." Applications and users had or still have to wait a lot more than 100 times access slowdown when it gets to trashing (excessive number of page faults), memory swapping, or simply normal waits for some slow actions. "Even if they could, who's to say the right 0.1% ends up optimized?" I haven't said it's about only 0.1%, and that it has to be dictated by some policy. For all I care, even if they all would be manually cache managed, I don't expect the result to be worse than the caching mechanism currently employed, which works without really knowing what's going on. "This was all decided a long time ago, general purpose caches won and scratch memory lost." A long time ago they had a different world. Their reasoning might have been true back then. We are creatures of reason, however, aren't we? "Yes, that sounds like a [sell] idea. What is wrong with that?" Salesmanship and the engineering of future tech have at their cores different objectives. At least in my book.