3 ms·

This looks like seriously impressive technology, and yields impressive results, but: I don't think I'd be comfortable with the idea of something rewriting my da

This looks like seriously impressive technology, and yields impressive results, but: I don't think I'd be comfortable with the idea of something rewriting my database in production. What I -would- probably be OK with is having dynamize analyze my workload in a staging/load test environment, and then producing a new binary for me which I could then run through its paces. Beyond actual errors being produced, I'm wondering what'd happen in weird scenarios such as one where by primary gets heavily optimized for its write load and creeps up to, say, 80% CPU or so at peak.. What then happens then if my replica which has been heavily optimized for its read load gets promoted in a failure scenario and gets pegged at high CPU? Final thought here is if this tech really is solid, when is AWS going to start shipping it with my VM?

davidyeager 8y ago
This does not rewrite your database. It optimizes the live in-memory machine code of the mysqld (MySQL Server) process. It must run on the same OS host as the mysqld (MySQL Server) process being optimized. So if you are using this on the master and not on the replica, the replica won’t be touched. Hope that makes sense.
- grogers 8y ago
  I think the point was that profile guided optimization relies on the workload staying relatively fixed. If the workload suddenly changes (like promoting a readonly slave to be the writable master) the assumptions made by PGO may not be valid, and performance could be worse than if no modifications were made in the first place. I think you'd just have to measure scenarios before using it in production.
  - davidyeager 8y ago
    Dynimizer can react to drastic changes in workload and reoptimize depending on how you configure it.
- martin_ 8y ago
  grogers explained my intent well, especially around the promotion of a replica to a primary work load (thank you). I think we're arguing semantics between optimizing machine code and rewriting mysqld. I want to emphasize that Dynamizer looks super awesome, and I do intend to try it, so no offense was intended. Well done!
  - davidyeager 8y ago
    Thanks! Sorry for terse tone... just responding quickly from a smartphone while travelling :-) Your concerns are definitely valid and we are working on improving the situation. Dynimizer can detect a drastic workload change as you described and reoptimize in response. That’s the default operation which can be turned off. What has been observed is that if we optimized for say a write-heavy workload and then change it to read-only without reoptimizing (or vice versa), it will still show an improvement, just not as much. Hope that makes sense.
deleted 8y ago
[deleted]
davidyeager 8y ago
For a future update we will be optionally caching optimizations to forgo the optimizations/warmup period. You can then move those cached optimizations from test to production to reduce uncertainty. Coming soon. Many advantages there over PGO which is very difficult to use.

davidyeager 8y ago

This does not rewrite your database. It optimizes the live in-memory machine code of the mysqld (MySQL Server) process. It must run on the same OS host as the mysqld (MySQL Server) process being optimized. So if you are using this on the master and not on the replica, the replica won’t be touched. Hope that makes sense.

grogers 8y ago
I think the point was that profile guided optimization relies on the workload staying relatively fixed. If the workload suddenly changes (like promoting a readonly slave to be the writable master) the assumptions made by PGO may not be valid, and performance could be worse than if no modifications were made in the first place. I think you'd just have to measure scenarios before using it in production.
- davidyeager 8y ago
  Dynimizer can react to drastic changes in workload and reoptimize depending on how you configure it.
martin_ 8y ago
grogers explained my intent well, especially around the promotion of a replica to a primary work load (thank you). I think we're arguing semantics between optimizing machine code and rewriting mysqld. I want to emphasize that Dynamizer looks super awesome, and I do intend to try it, so no offense was intended. Well done!
- davidyeager 8y ago
  Thanks! Sorry for terse tone... just responding quickly from a smartphone while travelling :-) Your concerns are definitely valid and we are working on improving the situation. Dynimizer can detect a drastic workload change as you described and reoptimize in response. That’s the default operation which can be turned off. What has been observed is that if we optimized for say a write-heavy workload and then change it to read-only without reoptimizing (or vice versa), it will still show an improvement, just not as much. Hope that makes sense.

deleted 8y ago

[deleted]

davidyeager 8y ago

For a future update we will be optionally caching optimizations to forgo the optimizations/warmup period. You can then move those cached optimizations from test to production to reduce uncertainty. Coming soon. Many advantages there over PGO which is very difficult to use.