5 ms·

The debate is mostly about: Are opaque probabilistic models scientific? David Mumford's stance: "This paper is a meant to be a polemic which argues for a ver

The debate is mostly about: Are opaque probabilistic models scientific? David Mumford's stance: "This paper is a meant to be a polemic which argues for a very fundamental point: that stochastic models and statistical reasoning are more relevant i) to the world, ii) to science and many parts of mathematics and iii) particularly to understanding the computations in our own minds, than exact models and logical reasoning" https://www.dam.brown.edu/people/mumford/beyond/papers/2000b--DawningAgeStoch-NC.pdf https://www.dam.brown.edu/people/mumford/beyond/papers/2000b...

foobarqux 4y ago
> The debate is mostly about: Are opaque probabilistic models scientific? No the debate is about whether probabilistic models are scientific when applied to the human language faculty (they aren't). Probabilistic models are scientific when they tell you something about the natural world. In some cases they do and in others they don't.
- hackandthink 4y ago
  Yes, the scope of the Norvig - Chomsky ist the human language faculty. But I think Mumford's more general discussion is relevant. And Mumford explicitly refers to speech: "This approach denies that statistical inference can have anything to do with real thought ... The new applications of Bayesian statistics to vision, speech, expert systems and neural nets have now started an explosive growth in these ideas."
  - foobarqux 4y ago
    It isn't relevant. There isn't any evidence that the human language faculty is a statistical process at its core, in fact there is evidence that it isn't. Vision, etc are different processes but in any case you have to show evidence of biology using a statistical process not simply postulate it. There is lots of talk like "these statistical models could yield information about how the mind or some other system works" but they never do. In fact no one even tries. People don't really care about doing science but because science is high prestige they want to make sure to be classified in that way.
    - candiodari 4y ago
      What's wrong with "humans/biologists have no evidence how the human language works. AI practicioners have one important piece of evidence: they can demonstrate somewhat-human-like language processing using statistical techniques, therefore they have the best available evidence". Nobody else can demonstrate it. But of course, this is simply the non-religious version of "humans have a soul, and machines, by definition don't". If necessary people drag quantum physics into that argument ... The truly forbidden argument is that we don't have any definition of a soul, and in fact plenty of evidence humans don't have a soul, such as large "soul"/character changes with occuring with physical damage to the neocortex. This also means the discussion is moot: people are now using LLM's to pass the Turing test on a large scale for all sorts of purposes. From scamming people to management (let's assume there's a difference there). From programming to teaching ugly bags of mostly water new languages. The point where people discover their wife/kids/... have more of a relationship with an LLM speaking for them than with them personally is now just a matter of time away.
      - foobarqux 4y ago
        AI practitioners have no evidence because they can't relate the thing they produced to how humans work. And it's false that scientists have no evidence, they don't have much but some of the little evidence they do have demonstrates that humans don't use statistical processes as the core of human language.
        candiodari 4y ago
        ChatGTP, someone gave me this answer. It's about you and the algorithm you run: "AI practitioners have no evidence because they can't relate the thing they produced to how humans work. And it's false that scientists have no evidence, they don't have much but some of the little evidence they do have demonstrates that humans don't use statistical processes as the core of human language." Do you think he's right? > While it is true that the workings of artificial intelligence algorithms and the workings of the human brain are not identical, this does not necessarily mean that AI practitioners have no evidence to support their work. In fact, there is a large body of research that has been done on the effectiveness and limitations of different machine learning algorithms, as well as on their applications in various domains. > Additionally, while it is true that human language processing is not solely based on statistical processes, it is also not accurate to say that humans do not use statistical processes as part of their language processing. There is evidence to suggest that humans do rely on statistical learning to some extent in order to acquire and use language. > Overall, it is important to approach these topics with nuance and recognize that the relationship between AI and human cognition is complex and multifaceted. This blows anything biological researchers can do to reproduce human behavior out of the water by a margin from here to Proxima Centauri and back. Therefore I'll believe the model behind this is a far closer approximation to human behavior than anything every to come out of any other field of research, not using humans themselves. Hell, I would comfortably declare this algorithm (far) more intelligent than our closest living relatives, primates.
jsenn 4y ago
Interesting paper, thanks for linking. I think this is slightly tangential to the Norvig-Chomsky controversy though. What Mumford is saying is that probability and statistics is a more useful basis for modelling natural phenomena than classical logic. I don't think Chomsky would disagree with this! What he disagrees with is the idea (also raised by Mumford) that merely reproducing surface-level aspects of a given natural phenomenon (like human language) with no insight or understanding is sufficient for a scientific theory. It isn't sufficient: one also has to show that the theory cannot produce phenomena that are not natural, and give some insight into what's going on. In fact, it's not even necessary: Galileo advanced physics by imagining a frictionless plane. This doesn't and can't exist in the real world, but it helps us understand what does happen in the real world. As an example, Mumford talks about how particle filters work much better than any "classical AI" technique for certain tracking tasks. This is true, and particle filters are still important in engineering for this reason, but it's not a theory of how humans accomplish this for the simple reason that particle filters can just as happily do other things that humans don't/can't.

foobarqux 4y ago

> The debate is mostly about: Are opaque probabilistic models scientific? No the debate is about whether probabilistic models are scientific when applied to the human language faculty (they aren't). Probabilistic models are scientific when they tell you something about the natural world. In some cases they do and in others they don't.

hackandthink 4y ago
Yes, the scope of the Norvig - Chomsky ist the human language faculty. But I think Mumford's more general discussion is relevant. And Mumford explicitly refers to speech: "This approach denies that statistical inference can have anything to do with real thought ... The new applications of Bayesian statistics to vision, speech, expert systems and neural nets have now started an explosive growth in these ideas."
- foobarqux 4y ago
  It isn't relevant. There isn't any evidence that the human language faculty is a statistical process at its core, in fact there is evidence that it isn't. Vision, etc are different processes but in any case you have to show evidence of biology using a statistical process not simply postulate it. There is lots of talk like "these statistical models could yield information about how the mind or some other system works" but they never do. In fact no one even tries. People don't really care about doing science but because science is high prestige they want to make sure to be classified in that way.
  - candiodari 4y ago
    What's wrong with "humans/biologists have no evidence how the human language works. AI practicioners have one important piece of evidence: they can demonstrate somewhat-human-like language processing using statistical techniques, therefore they have the best available evidence". Nobody else can demonstrate it. But of course, this is simply the non-religious version of "humans have a soul, and machines, by definition don't". If necessary people drag quantum physics into that argument ... The truly forbidden argument is that we don't have any definition of a soul, and in fact plenty of evidence humans don't have a soul, such as large "soul"/character changes with occuring with physical damage to the neocortex. This also means the discussion is moot: people are now using LLM's to pass the Turing test on a large scale for all sorts of purposes. From scamming people to management (let's assume there's a difference there). From programming to teaching ugly bags of mostly water new languages. The point where people discover their wife/kids/... have more of a relationship with an LLM speaking for them than with them personally is now just a matter of time away.
    - foobarqux 4y ago
      AI practitioners have no evidence because they can't relate the thing they produced to how humans work. And it's false that scientists have no evidence, they don't have much but some of the little evidence they do have demonstrates that humans don't use statistical processes as the core of human language.
      - candiodari 4y ago
        ChatGTP, someone gave me this answer. It's about you and the algorithm you run: "AI practitioners have no evidence because they can't relate the thing they produced to how humans work. And it's false that scientists have no evidence, they don't have much but some of the little evidence they do have demonstrates that humans don't use statistical processes as the core of human language." Do you think he's right? > While it is true that the workings of artificial intelligence algorithms and the workings of the human brain are not identical, this does not necessarily mean that AI practitioners have no evidence to support their work. In fact, there is a large body of research that has been done on the effectiveness and limitations of different machine learning algorithms, as well as on their applications in various domains. > Additionally, while it is true that human language processing is not solely based on statistical processes, it is also not accurate to say that humans do not use statistical processes as part of their language processing. There is evidence to suggest that humans do rely on statistical learning to some extent in order to acquire and use language. > Overall, it is important to approach these topics with nuance and recognize that the relationship between AI and human cognition is complex and multifaceted. This blows anything biological researchers can do to reproduce human behavior out of the water by a margin from here to Proxima Centauri and back. Therefore I'll believe the model behind this is a far closer approximation to human behavior than anything every to come out of any other field of research, not using humans themselves. Hell, I would comfortably declare this algorithm (far) more intelligent than our closest living relatives, primates.

jsenn 4y ago

Interesting paper, thanks for linking. I think this is slightly tangential to the Norvig-Chomsky controversy though. What Mumford is saying is that probability and statistics is a more useful basis for modelling natural phenomena than classical logic. I don't think Chomsky would disagree with this! What he disagrees with is the idea (also raised by Mumford) that merely reproducing surface-level aspects of a given natural phenomenon (like human language) with no insight or understanding is sufficient for a scientific theory. It isn't sufficient: one also has to show that the theory cannot produce phenomena that are not natural, and give some insight into what's going on. In fact, it's not even necessary: Galileo advanced physics by imagining a frictionless plane. This doesn't and can't exist in the real world, but it helps us understand what does happen in the real world. As an example, Mumford talks about how particle filters work much better than any "classical AI" technique for certain tracking tasks. This is true, and particle filters are still important in engineering for this reason, but it's not a theory of how humans accomplish this for the simple reason that particle filters can just as happily do other things that humans don't/can't.