4 ms·

This sounds like a pile of equivocations designed to try to minimize an unfortunate truth about learned functions: the learning process does not care whether it

This sounds like a pile of equivocations designed to try to minimize an unfortunate truth about learned functions: the learning process does not care whether it corresponds to human thinking, so learned functions do not necessarily correspond to human-written functions. Yes, the black-box nature of machine learning is a state of mind; the alternative is to view all software as black boxes and then throw up our hands and claim to know nothing whatsoever. This is clearly facetious and meant to protect machine-learning developers from the responsibility of grokking their systems.

vamin 9y ago
You’re setting up a bit of a false dichotomy, in my opinion. Treating something like a black box doesn’t mean you have to “throw up our hands and claim to know nothing whatsoever.” Rather, it means you have to change your approach to understanding the system. People who are concerned about “black boxes” seem to think that we need a first principles or causal-mechanistic explanation for what’s going on in a machine learning system to have any confidence in it. That couldn’t be further from the case. By interrogating the inputs and outputs of a “black box” you can learn all you need about how it works. Much (if not most) of our understanding of the physical world comes from carefully probing black box systems—systems for which we have no a priori knowledge of mechanism. So, the alternative is not to “throw up your hands,” it is to take a considered, scientific approach to understanding the relationship between inputs and outputs in your model: in what situations it succeeds, in what situations it fails, how changing a single variable affects the output, etc. Yes, that can be difficult for a complex model, but why should anyone expect it to be simple?

4 ms·

This sounds like a pile of equivocations designed to try to minimize an unfortunate truth about learned functions: the learning process does not care whether it

by EtDybNuvCu 9y ago

vamin 9y ago
You’re setting up a bit of a false dichotomy, in my opinion. Treating something like a black box doesn’t mean you have to “throw up our hands and claim to know nothing whatsoever.” Rather, it means you have to change your approach to understanding the system. People who are concerned about “black boxes” seem to think that we need a first principles or causal-mechanistic explanation for what’s going on in a machine learning system to have any confidence in it. That couldn’t be further from the case. By interrogating the inputs and outputs of a “black box” you can learn all you need about how it works. Much (if not most) of our understanding of the physical world comes from carefully probing black box systems—systems for which we have no a priori knowledge of mechanism. So, the alternative is not to “throw up your hands,” it is to take a considered, scientific approach to understanding the relationship between inputs and outputs in your model: in what situations it succeeds, in what situations it fails, how changing a single variable affects the output, etc. Yes, that can be difficult for a complex model, but why should anyone expect it to be simple?