3 ms·

> We have cheap and usable machine learning; it can run on neat hardware if you need it to. Meanwhile, databases are hard to tune. It seems that we could combin

by collyw 8y ago

> We have cheap and usable machine learning; it can run on neat hardware if you need it to. Meanwhile, databases are hard to tune. It seems that we could combine the two to tune automagically database systems. People are working on this. It sounds a bit crazy, but I am actually excited about it and I plan to do some of my own work in this direction. Aren't databases well understood in comparison to machine learning? I know it would take me a lot less time to look up how to tune a database than it would to get a machine learning system up and running.

contingencies 8y ago
This was the only point that struck me as well. I believe the author was referring to self-tuning within a general case (ie. as part of the DB software to be deployed against many new databases) rather than one-off ML-based tuning (ie. as part of a manual optimization effort for an existing database). Of course, the real optimization should be for legibility and maintainability... human time being far more important than machine time. But that wouldn't allow fetish discussions diving in to low level efficiencies, now would it? Any sufficiently complex system acts as a black box when it becomes easier to experiment with than to understand. Hence, black-box optimization has become increasingly important as systems become more complex. - Google Vizier: a service for black-box optimization Golovin et al., KDD'17
jacques_chester 8y ago
I think it's mostly down to the fact that the configuration space for database tuning is very large and that it can vary a lot from machine to machine. There's a point up to which this would be better handled with a rules system. And after that "clever" stuff can search for rules. In any case the real problem with using ML is getting enough data. Most folks don't feel like spending millions of dollars waiting for their database autotuner to climb billions of hills, they'd rather someone else did it for them. Which brings us back to prebaked rules systems.

by collyw 8y ago

contingencies 8y ago

This was the only point that struck me as well. I believe the author was referring to self-tuning within a general case (ie. as part of the DB software to be deployed against many new databases) rather than one-off ML-based tuning (ie. as part of a manual optimization effort for an existing database). Of course, the real optimization should be for legibility and maintainability... human time being far more important than machine time. But that wouldn't allow fetish discussions diving in to low level efficiencies, now would it? Any sufficiently complex system acts as a black box when it becomes easier to experiment with than to understand. Hence, black-box optimization has become increasingly important as systems become more complex. - Google Vizier: a service for black-box optimization Golovin et al., KDD'17

jacques_chester 8y ago

I think it's mostly down to the fact that the configuration space for database tuning is very large and that it can vary a lot from machine to machine. There's a point up to which this would be better handled with a rules system. And after that "clever" stuff can search for rules. In any case the real problem with using ML is getting enough data. Most folks don't feel like spending millions of dollars waiting for their database autotuner to climb billions of hills, they'd rather someone else did it for them. Which brings us back to prebaked rules systems.