5 ms·

The article calls machine learning "conservative" because it only tells us what is, and not what should be. I don't think that's a useful framing. It's more acc

by knzhou 7y ago

The article calls machine learning "conservative" because it only tells us what is, and not what should be. I don't think that's a useful framing. It's more accurate to say that, like all statistical techniques, machine learning is descriptive, not prescriptive. Not everything in the world has to be prescriptive.

avs733 7y ago
The article is not addressing what machine learning can and cannot do but how individuals and organizations conceptualize what machine learning can and cannot do. It, as the current top comment mentions, is a philosophical rather than technical argument. It doesn't matter if all machine learning can do is describe the current state of things of it is presumed by those interpreting the output to provide insight into some great and universal truth. Most science degree programs have a philosophy of science course as a degree requirement. As an engineering faculty member I have found huge resistance to and equivalent course becoming part of engineering or computer science degrees. Those degrees, instead, take an apparoch to scientific/technical knowledge that is less science and more religion. The comments here are a reminder why such courses have a place.
- abdullahkhalids 7y ago
  It is not just a philosophy course which is missing from software engineering degrees. Software has real impact on individuals and societies that use them. Therefore, software engineers must necessarily be taught about the interface of psychology/sociology with software and the ethics surrounding this interface. The world would be a better place for it. Please read this thread https://twitter.com/yonatanzunger/status/975545527973462016 https://twitter.com/yonatanzunger/status/975545527973462016 "computer science is a field which hasn't yet encountered consequences."
visarga 7y ago
We generally want ML to imitate us (or human generated data), thus we teach it to do so. But there are also generative ML (like GANs) and simulation based ML (like AlphaGo) which can be more creative. There is nothing stoping us from letting agents evolve in a complex environment and be creative. It's just not commercially useful to do that yet. Doctorow writes like he doesn't understand much math behind ML, yet has strong opinions on it. Every time a random number is involved in the training process (stochastic selection of mini batches, noise injection, epsilon-greedy action selection), a form of exploration is taking place, finding novel paths of solving the problem at hand. The mixing of noise with data is paradoxically useful, even necessary in learning.
- knzhou 7y ago
  Sure, but no matter what, you can't derive an "ought" from an "is". At the end of the day we merely tell these algorithms what is the case. No matter what goes on inside, they cannot output moral prescriptions.
  - logicchains 7y ago
    >Sure, but no matter what, you can't derive an "ought" from an "is". The same applies to humans, no?
    - Iolaum 7y ago
      No it doesn't. When a humam gets wronged by what "is" they can likely feel or imagine a better "ought".
      - jmmcd 7y ago
        They can feel or imagine an "ought", but the point of Hume is that they can't argue for it.
        goatlover 7y ago
        And yet humans do, all the time. It's similar to Hume's attack on causality. You can't show that A caused B, but yet we all act like it's the case, when B always follows A. Kant's critique comes next.
      - naveen99 7y ago
        You don’t think an machines can learn to classify situations as good or bad for them ?
- Reelin 7y ago
  > ... finding novel paths of solving the problem at hand ... This only applies to reinforcement learning (and similar). Classification problems (and similar) are fundamentally conservative, because you feed them with a finite set of preexisting training data that they must extrapolate from.
meritt 7y ago
> Not everything in the world has to be prescriptive. Unfortunately we're still on the rise of the current hype cycle around Machine Learning, and prescriptive solutions are precisely what nearly every company who use the phrase "machine learning" are promising.
- sgt101 7y ago
  I think recasting prescription from "so you should" to "and here are the inferences now available" can be reasonable. The issue is when the inferences from discovered patterns are invalid and implicit - as per predictive policing or follow about ads on the web. An inference such as "this is almost certainly the component that is causing the most faults and will cost the least to replace" is reasonable and valid to me, and so a machine offering it would be valuable.
ckcheng 7y ago
Maybe the pure science of ML is descriptive and not prescriptive. But if the rest of the world uses ML prescriptively in application, then in practice it's basically prescriptive. It's a bit like dictionaries: they "learn" the current meaning of words. It gets annoying though when everyone starts running around policing the meaning of words based heavily on that dictionary: it would conservatively restrict the natural evolution of word meanings, when we know meaning of words do change over time, and sometimes should change over time.

by knzhou 7y ago

avs733 7y ago

The article is not addressing what machine learning can and cannot do but how individuals and organizations conceptualize what machine learning can and cannot do. It, as the current top comment mentions, is a philosophical rather than technical argument. It doesn't matter if all machine learning can do is describe the current state of things of it is presumed by those interpreting the output to provide insight into some great and universal truth. Most science degree programs have a philosophy of science course as a degree requirement. As an engineering faculty member I have found huge resistance to and equivalent course becoming part of engineering or computer science degrees. Those degrees, instead, take an apparoch to scientific/technical knowledge that is less science and more religion. The comments here are a reminder why such courses have a place.

abdullahkhalids 7y ago
It is not just a philosophy course which is missing from software engineering degrees. Software has real impact on individuals and societies that use them. Therefore, software engineers must necessarily be taught about the interface of psychology/sociology with software and the ethics surrounding this interface. The world would be a better place for it. Please read this thread https://twitter.com/yonatanzunger/status/975545527973462016 https://twitter.com/yonatanzunger/status/975545527973462016 "computer science is a field which hasn't yet encountered consequences."

visarga 7y ago

We generally want ML to imitate us (or human generated data), thus we teach it to do so. But there are also generative ML (like GANs) and simulation based ML (like AlphaGo) which can be more creative. There is nothing stoping us from letting agents evolve in a complex environment and be creative. It's just not commercially useful to do that yet. Doctorow writes like he doesn't understand much math behind ML, yet has strong opinions on it. Every time a random number is involved in the training process (stochastic selection of mini batches, noise injection, epsilon-greedy action selection), a form of exploration is taking place, finding novel paths of solving the problem at hand. The mixing of noise with data is paradoxically useful, even necessary in learning.

knzhou 7y ago
Sure, but no matter what, you can't derive an "ought" from an "is". At the end of the day we merely tell these algorithms what is the case. No matter what goes on inside, they cannot output moral prescriptions.
- logicchains 7y ago
  >Sure, but no matter what, you can't derive an "ought" from an "is". The same applies to humans, no?
  - Iolaum 7y ago
    No it doesn't. When a humam gets wronged by what "is" they can likely feel or imagine a better "ought".
    - jmmcd 7y ago
      They can feel or imagine an "ought", but the point of Hume is that they can't argue for it.
      - goatlover 7y ago
        And yet humans do, all the time. It's similar to Hume's attack on causality. You can't show that A caused B, but yet we all act like it's the case, when B always follows A. Kant's critique comes next.
    - naveen99 7y ago
      You don’t think an machines can learn to classify situations as good or bad for them ?
Reelin 7y ago
> ... finding novel paths of solving the problem at hand ... This only applies to reinforcement learning (and similar). Classification problems (and similar) are fundamentally conservative, because you feed them with a finite set of preexisting training data that they must extrapolate from.

meritt 7y ago

> Not everything in the world has to be prescriptive. Unfortunately we're still on the rise of the current hype cycle around Machine Learning, and prescriptive solutions are precisely what nearly every company who use the phrase "machine learning" are promising.

sgt101 7y ago
I think recasting prescription from "so you should" to "and here are the inferences now available" can be reasonable. The issue is when the inferences from discovered patterns are invalid and implicit - as per predictive policing or follow about ads on the web. An inference such as "this is almost certainly the component that is causing the most faults and will cost the least to replace" is reasonable and valid to me, and so a machine offering it would be valuable.

ckcheng 7y ago

Maybe the pure science of ML is descriptive and not prescriptive. But if the rest of the world uses ML prescriptively in application, then in practice it's basically prescriptive. It's a bit like dictionaries: they "learn" the current meaning of words. It gets annoying though when everyone starts running around policing the meaning of words based heavily on that dictionary: it would conservatively restrict the natural evolution of word meanings, when we know meaning of words do change over time, and sometimes should change over time.