4 ms·

> That, and... performance tuning is hard work Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary

by tpxl 4y ago

> That, and... performance tuning is hard work Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer. The problem isn't 'hard work', it's nobody cares about this.

otikik 4y ago
That is not the lowest hanging fruit (where I would put trivial changes like inlining functions or switching an of-the-shelf library by a faster alternative). But it is medium-low hanging fruit. Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway. Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time. Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.
Xeamek 4y ago
Yes, last part of performance tuning is relatively simple process as you described, but that's only because apps where performance matters are developed from the ground app with performance in mind, so entire architecture is designed in a way that supports performance. Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base. You really think all you will talk about are cache misses?
- aastronaut 4y ago
  no, but you can definitely talk about it. i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care. finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.
bayindirh 4y ago
That’s just good software development practices. Game developers and scientific software developers borderline abuse the software (language, drivers, etc.) and hardware to get these performance numbers. For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU. I need to test this with every CPU generation and test whether it works as expected performance-wise. To achieve this, I need to rewrite half of the said software. The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.
rwmj 4y ago
Isn't it another example of misaligned incentives? Your website renders on my computer, making rendering performance mostly my problem. Your website is only financially viable because it includes Google adverts and tracking, and Google again isn't paying to run those on my computer.
- leidenfrost 4y ago
  Backend developer here. Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess. The process goes like this: - Devs ditch the legacy (often monolithic) code - They start creating microservices fron scratch - They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write. - They do that with the idea that the legacy code is "bad" and the new code is "good" - Requirements either scale or downright change - To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good" Repeat Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again
BackBlast 4y ago
> Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer. That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly. I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs. I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s. Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.
- tpxl 4y ago
  GC has not been an issue in Java for years. I haven't seen memory issues, except for services running in ridiculously CPU or memory starved kubernetes containers.
  - LtdJorge 4y ago
    This could be Python or another language.
    - tpxl 4y ago
      I know, just sharing my specific experience. A sibling poster also comments that dictionaries are too slow. They might be in a game setting, but definitely not in a Java backend service.
barbariangrunge 4y ago
Well, for starters... dictionary hashing is too slow to be ideal. We don't swear off dictionaries, but we try to get our constant-time lookups without needing a hashing step when possible. The dictionaries can also lead to extra cache misses, since the memory often won't be compacted as you step through your data. In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!
- generichuman 4y ago
  > dictionary hashing is too slow to be ideal Even when you use SIMD intrinsics? For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions https://en.wikipedia.org/wiki/Intel_SHA_extensions A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics https://github.com/noloader/SHA-Intrinsics
  - JacobiX 4y ago
    Not the OP, but those extensions implements cryptographic hash functions, they are not very suitable (too slow) for hash-tables and other hash based data structures.
    - generichuman 4y ago
      4 CPU cycles is too slow? Because SHA256RNDS2 is 4 clock cycles on Zen3 CPUs. 6 clock cycles on Intel Alder Lake (12th gen). Source is the measurements here: https://uops.info/table.html https://uops.info/table.html
      - JacobiX 4y ago
        Yes, because SHA256RNDS2 aid the implementation of the update function of SHA256. We should invoke it multiple times on the message blocks, it performs two rounds out of the 64 SHA256 rounds.

by tpxl 4y ago

otikik 4y ago

That is not the lowest hanging fruit (where I would put trivial changes like inlining functions or switching an of-the-shelf library by a faster alternative). But it is medium-low hanging fruit. Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway. Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time. Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.

Xeamek 4y ago

Yes, last part of performance tuning is relatively simple process as you described, but that's only because apps where performance matters are developed from the ground app with performance in mind, so entire architecture is designed in a way that supports performance. Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base. You really think all you will talk about are cache misses?

aastronaut 4y ago
no, but you can definitely talk about it. i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care. finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.

bayindirh 4y ago

That’s just good software development practices. Game developers and scientific software developers borderline abuse the software (language, drivers, etc.) and hardware to get these performance numbers. For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU. I need to test this with every CPU generation and test whether it works as expected performance-wise. To achieve this, I need to rewrite half of the said software. The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.

rwmj 4y ago

Isn't it another example of misaligned incentives? Your website renders on my computer, making rendering performance mostly my problem. Your website is only financially viable because it includes Google adverts and tracking, and Google again isn't paying to run those on my computer.

leidenfrost 4y ago
Backend developer here. Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess. The process goes like this: - Devs ditch the legacy (often monolithic) code - They start creating microservices fron scratch - They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write. - They do that with the idea that the legacy code is "bad" and the new code is "good" - Requirements either scale or downright change - To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good" Repeat Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again

BackBlast 4y ago

> Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer. That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly. I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs. I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s. Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.

tpxl 4y ago
GC has not been an issue in Java for years. I haven't seen memory issues, except for services running in ridiculously CPU or memory starved kubernetes containers.
- LtdJorge 4y ago
  This could be Python or another language.
  - tpxl 4y ago
    I know, just sharing my specific experience. A sibling poster also comments that dictionaries are too slow. They might be in a game setting, but definitely not in a Java backend service.

barbariangrunge 4y ago

Well, for starters... dictionary hashing is too slow to be ideal. We don't swear off dictionaries, but we try to get our constant-time lookups without needing a hashing step when possible. The dictionaries can also lead to extra cache misses, since the memory often won't be compacted as you step through your data. In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!

generichuman 4y ago
> dictionary hashing is too slow to be ideal Even when you use SIMD intrinsics? For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions https://en.wikipedia.org/wiki/Intel_SHA_extensions A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics https://github.com/noloader/SHA-Intrinsics
- JacobiX 4y ago
  Not the OP, but those extensions implements cryptographic hash functions, they are not very suitable (too slow) for hash-tables and other hash based data structures.
  - generichuman 4y ago
    4 CPU cycles is too slow? Because SHA256RNDS2 is 4 clock cycles on Zen3 CPUs. 6 clock cycles on Intel Alder Lake (12th gen). Source is the measurements here: https://uops.info/table.html https://uops.info/table.html
    - JacobiX 4y ago
      Yes, because SHA256RNDS2 aid the implementation of the update function of SHA256. We should invoke it multiple times on the message blocks, it performs two rounds out of the 64 SHA256 rounds.