6 ms·

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but tha

by euske 10y ago

I've been saying that ML is much like alchemy than science. They've pretty much given up to understand the underlying mechanism because it's so complex, but that doesn't stop them experimenting because they still get something that looks like a result. And hey, they can get paid for it. Eventually it might grow into a full-fledged science, but it will probably take an awful lot of time.

Udo 10y ago
I disagree about the underlying mechanism being complex. Machine learning algorithms are a class of equation-based systems and strategies for configuring these equations for a specific task. All our math and reasoning tools still apply to this class of algorithms, in principle. Where we pass into alchemy though is the interplay of these basic components with each other and the parameters they encounter while running, this is where complexity happens. Part of this lies in the very nature of the tasks we use them for: we basically push a cart of raw data in front of a set of "AI" solvers and expect them to do something with it. When that doesn't work, start over, tweak parameters, and try again. I agree that there is no sufficiently useful intellectual framework for creating these artificially intelligent components, and that shows not only in the uneven success rates and performance, but also in the surprising fact that experts in very different AI systems can usually create components with similar performance characteristics for a given problem, despite using very disparate strategies.
yummyfajitas 10y ago
They haven't given up at all. And more importantly, they've developed a huge number of "if it looks good it is good" mathematical theorems. This means that under many circumstances, you can build a model that satisfies certain abstract properties, test it, and have a high probability of generalizability. I.e. we've circumvented the "understanding" stage. In fact, I'd say that the core of machine learning (as opposed to merely statistics) is exactly this.
Houshalter 10y ago
I think you mean it's much more like science than math. Not that it's much more like alchemy than science. Machine learning research is very empirical. With a philosophy of doing lots of experiments and tests, and discovering what methods work best. That's what science is. No scientific field starts off with complete knowledge and understanding, they have to do a lot of experiments to discover the general laws. Some people dislike empiricism and want pure, provable math. Machine learning isn't a field of mathematics, at least not in practice. But that doesn't mean it's not a science.

by euske 10y ago

Udo 10y ago

I disagree about the underlying mechanism being complex. Machine learning algorithms are a class of equation-based systems and strategies for configuring these equations for a specific task. All our math and reasoning tools still apply to this class of algorithms, in principle. Where we pass into alchemy though is the interplay of these basic components with each other and the parameters they encounter while running, this is where complexity happens. Part of this lies in the very nature of the tasks we use them for: we basically push a cart of raw data in front of a set of "AI" solvers and expect them to do something with it. When that doesn't work, start over, tweak parameters, and try again. I agree that there is no sufficiently useful intellectual framework for creating these artificially intelligent components, and that shows not only in the uneven success rates and performance, but also in the surprising fact that experts in very different AI systems can usually create components with similar performance characteristics for a given problem, despite using very disparate strategies.

yummyfajitas 10y ago

They haven't given up at all. And more importantly, they've developed a huge number of "if it looks good it is good" mathematical theorems. This means that under many circumstances, you can build a model that satisfies certain abstract properties, test it, and have a high probability of generalizability. I.e. we've circumvented the "understanding" stage. In fact, I'd say that the core of machine learning (as opposed to merely statistics) is exactly this.

Houshalter 10y ago

I think you mean it's much more like science than math. Not that it's much more like alchemy than science. Machine learning research is very empirical. With a philosophy of doing lots of experiments and tests, and discovering what methods work best. That's what science is. No scientific field starts off with complete knowledge and understanding, they have to do a lot of experiments to discover the general laws. Some people dislike empiricism and want pure, provable math. Machine learning isn't a field of mathematics, at least not in practice. But that doesn't mean it's not a science.