3 ms·
I don't think the comparison with human learning holds. NNs and humans don't learn the same way - humans can fairly quickly generalise what they have learned a
by a_bonobo 3y ago
I don't think the comparison with human learning holds.
NNs and humans don't learn the same way - humans can fairly quickly generalise what they have learned and, most importantly, go beyond what they've learned. I haven't see that happen with neural networks or GPTs; at best, you're getting the average of what it has 'learned'. There's human learning and there's neural network 'learning' and they're a different thing.
- nl 3y agoNN's absolute can go beyond what they have learned and aren't just producing the "average". Some good examples outside the typical LLM/images work: * Deep Mind's work on AlphaFold, which generates predictions on proteins that haven't been seen before * AlphaGo which plays games better than any human (so clearly can't be "the average") If we look at LLMs, something like writing code in the style of Shakespear isn't really something that's been seen before.
- a_bonobo 3y ago> Deep Mind's work on AlphaFold, which generates predictions on proteins that haven't been seen before I have used AlphaFold a bit in my own work, and if I showed it 'unusual' proteins like rare mutants it usually generated garbage. Some evidence for this exists in the literature; see for example https://www.biorxiv.org/content/10.1101/2021.09.19.460937v1 https://www.biorxiv.org/content/10.1101/2021.09.19.460937v1 or https://academic.oup.com/bioinformatics/article/38/7/1881/6517779 https://academic.oup.com/bioinformatics/article/38/7/1881/65... or >AlphaFold recognizes a 3D structure of the examined amino acid sequence by a similarity of this sequence (or its parts) to related sequences with already known 3D structures https://www.biorxiv.org/content/10.1101/2022.11.21.517308v1 https://www.biorxiv.org/content/10.1101/2022.11.21.517308v1