3 ms·
> I take Chomsky's points to be the following: 1. Statistical language models have had engineering success, but that is irrelevant to science. Well, deep neura
by sampo 4y ago
> I take Chomsky's points to be the following: 1. Statistical language models have had engineering success, but that is irrelevant to science.
Well, deep neural nets are not statistical models, so shouldn't Chomsky now be at least a little bit happier with ChatGPT?
- mxwsn 4y agoHow so? Deep neural nets and ChatGPT are very much statistical models.
- hackinthebochs 4y agoWhat does it mean to be a statistical model? Deep nets are deterministic, for example. We can use them to model probability distributions, but they are not intrinsically statistical.
- sampo 4y ago> How so? They don't fit the definition [1,2,3]. But I looked at the internet, and apparently many people consider neural nets to be statistical models, or "a kind of" statistical models. [1] https://en.wikipedia.org/wiki/Statistical_model https://en.wikipedia.org/wiki/Statistical_model [2] https://www.stat.uchicago.edu/~pmcc/pubs/AOS023.pdf https://www.stat.uchicago.edu/~pmcc/pubs/AOS023.pdf [3] https://www.statlect.com/glossary/statistical-model https://www.statlect.com/glossary/statistical-model