3 ms·
For databases, I'm reluctant to rely on PGO (profile-guided optimization) because the workloads are so varied. There's a risk of over-fitting to the profiled wo
by jeff-davis 4y ago
For databases, I'm reluctant to rely on PGO (profile-guided optimization) because the workloads are so varied. There's a risk of over-fitting to the profiled workloads at the expense of others.
Though there may be a lot of opportunity with some database subsystems that have a more consistent usage pattern.
Edit: also, PGO is closely related to JIT techniques, which are based on current runtime information rather than profiles generated a long time ago on a workload that may or may not be representative.
- vlovich123 4y agoI think in practice enabling PGO will be a net gain even if it’s suboptimal on some workloads (ie you should still see some performance gain across the board even if the specific workload isn’t profiled). The reason is that it’s using the profiles to make decisions in lieu of heuristics which should be a win even for non profiled workloads because heuristics are essentially just general case profiles (ie tuned to a bunch of OSS software out there). I’m unaware of any research showing PGO being worse than not doing it even if your profile isn’t the workload (you’d probably have to try to specially build such a situation and unlikely to come up in practice). Have you actually seen otherwise?
- Filligree 4y agoThroughput isn't everything, and improving OPS at the expensive of tail latency can be a problem. Depends on your specific workload, but it isn't something I'd enable by default.
- summerlight 4y agoPGO will still likely improve general performance even with biased workloads because in many cases we want compilers to focus on optimizing happy paths, but this is not always well executed even when it's pretty trivial for human eyes.