6 ms·
> How do you teach people to effectively use a profiler on code with pernicious caching? Profile the function that is called if object is not found in cache ?
by ilyt 4y ago
> How do you teach people to effectively use a profiler on code with pernicious caching?
Profile the function that is called if object is not found in cache ? Have pluggable cache with plugin that just doesn't cache? That is not hard problem to solve.
> They get really uncomfortable six months later when I'm reporting 2-3x in code that can't be optimized further. And in each case everything was made more difficult by the people ahead of me chumming the waters
Well, they delivered (I'm guessing) 10x improvement in a month by adding cache. Caching is entirely reasonable first step and it can be good enough to be last one.
And frankly ability to use profiler and knowing some architectural optimizations seems to be pretty rare skill.
- zasdffaa 4y agoExactly! People here seem to be making a complex thing out of a simple thing.
- hinkley 4y agoWhat makes caching one of the hardest parts of computing is this assertion that it's easy. Everything is easy when you oversimplify it. Some things explode in complexity when you try to force them to be simple.
- andrekandre 4y ago> Some things explode in complexity when you try to force them to be simple. do you mean to say iow "simple in the small but when assembled (in the large) is more complex" whereas what we should be aiming for is perhaps "a little more complex in the small but with the benefit that it enables a simpler (and probably more scalable) architecture once assembled (in the large)"??
- TeMPOraL 4y ago> Profile the function that is called if object is not found in cache ? Have pluggable cache with plugin that just doesn't cache? That is not hard problem to solve. The more the dynamic behavior of the profiled system and the real system deviate, the harder it is to isolate what's causing the slowdowns. > Well, they delivered (I'm guessing) 10x improvement in a month by adding cache. Yes, but by doing this they also tremendously increased the complexity of the project, and (unless the team has someone like GP in it) prevented further optimizations. It may still be a good trade-off, but the overall sorry state of pretty much all software everywhere suggests it may not be a good one usually. > And frankly ability to use profiler and knowing some architectural optimizations seems to be pretty rare skill. That's kind of reinforcing the parent's point, though.
- zasdffaa 4y ago> The more the dynamic behavior of the profiled system and the real system deviate Maybe. I've never found it a problem, but maybe. When I've added a cache, the cache comes with instrumentation for exactly that. > Yes, but by doing this they also tremendously increased the complexity of the project It depends. EG if you don't need to worry too much about cache invalidation, adding caching can be as simple as adding a single function call eg like python's lru_cache. It depends on the use case.
- ilyt 4y ago>> Profile the function that is called if object is not found in cache ? Have pluggable cache with plugin that just doesn't cache? That is not hard problem to solve. >The more the dynamic behavior of the profiled system and the real system deviate, the harder it is to isolate what's causing the slowdowns. Whose quote you repeat without thinking ? It's not that relevant if you know how profiling works... >> Well, they delivered (I'm guessing) 10x improvement in a month by adding cache. >Yes, but by doing this they also tremendously increased the complexity of the project Not really in most cases its pretty simple to add it, and if you profiled correctly you'd start with only adding extra code in hottest part of the app > and (unless the team has someone like GP in it) prevented further optimizations. Uh, just no. It might've fixed problem well enough to not be touched but it ain't stopping anything, just a different solution. > It may still be a good trade-off, but the overall sorry state of pretty much all software everywhere suggests it may not be a good one usually. The assumption here that the same developers that fucked up caching would somehow manage to achieve same speedup with "just" optimizations and less bugs is bold one. Hell, many low level optimizations are also basically "using cache" (but CPU one) well, althought compiler usually gets you out of most egregious mistakes (unless you decide to write multithreaded C/C++ carelessly I guess. >> And frankly ability to use profiler and knowing some architectural optimizations seems to be pretty rare skill. > That's kind of reinforcing the parent's point, though. What point exactly ? That average developer is bad at their job ? Poster above explicitly said that the worse developers claimed it can't be optimized more in the first place, what else you expect from them ? Cache was the most cost-efficient solution most likely.
- hinkley 4y ago> Profile the function that is called if object is not found in cache ? Have pluggable cache with plugin that just doesn't cache? That is not hard problem to solve Okay, now I have a profile that says 90% of the time is spent in the code the cache avoids. Now what? Narrator's note: When the cache was put in place it accounted for 40% of run time, but now people use it because it's 'free' and so the code has grown from looking up the data twice to looking it up five to seven times depending on conditional logic. > Well, they delivered (I'm guessing) 10x improvement in a month by adding cache. No, they didn't. That's the problem. No cache gets you 10x. 10x for one step of a 20 step process is a positive improvement but rarely more than 2x for the first one, 4x at the outside. If it does more that that then someone is being titanically stupid. From then on each cache is going to net you less, and less, and less. Only if someone measured wrong or you are blocked by some architecture change that's needed so that caches even make sense, will you see a bigger improvement this year than you did last. But the pathology I'm discussing above didn't even have anything at all to do with caching. It's the psychology of (ir)rationalizing around performance, the assertion that you've done everything that can be done because you've done everything you know to do. It's a bizarre love child of learned helplessness and ego self defense. It's gotten way worse in a world where people reach for caches first (and thus, I assert, last), but it was always there. I have a couple coworkers who are trying to be clever with a new cache architecture. Irritatingly by taking a statement of fact made by me 3+ years ago and jumping to the wrong conclusion. They promised 50% in 18 months (I was suggesting we target 90% in three years). Now we are somewhere past 30 months and so far they've delivered lower numbers than I did in 5 man weeks spread out from Christmas to mid March, while juggling other priorities and suffering from Seasonal Affective Disorder. Another coworker, usually cache averse, was making noises about adding a cache for some other pain point, and I started pulling on the thread for a bit until he was satisfied. It'll be next March before they surpass the 20% I turned in with about 8 modest changes. And the architectural improvement from both efforts will be roughly on a par. Top down is sexy, bottom up is not. But it's hard to do the former without the latter. Eat your vegetables.
- zasdffaa 4y ago> No cache gets you 10x I've done much better. Think sticking a a cache in front of a database with seriously complex queries. Yes you need to be be aware of staleness, no it's not a magic wand, but it delivered massive improvements. > When the cache was put in place it accounted for 40% of run time If looking shit up in a hash table, at a few hundred or thousand cpu ticks, cost 40% of your runtime then I smell PEBKAC[1], not cache problems. [1] Not you personally but someone