4 ms·
Yeah, no person has ever read anything like every textbook ever written, but that's pretty much table stakes for training sets. Clearly there's something missin
by thfuran 3y ago
Yeah, no person has ever read anything like every textbook ever written, but that's pretty much table stakes for training sets. Clearly there's something missing aside from more virtual reading. (I suspect it has something to do with the half a billion years of pre-training baked into the human neural architecture and the few extra orders of magnitude in scale but who knows)
- all2 3y agoI'm sure I've read about specialized neural networks being created. The human brain has (apparently) a bunch of different kinds of neurons in it that specialize in processing different information. I'm not sure how that would work with our current architectures, though.
- bpiche 3y agoWell Jeff Hawkins has been working on this for a while, in terms of biomimetic neural networks. They've done some great work but they don't have anything like modern language models in terms of abilities + performance. https://www.youtube.com/watch?v=cz-3WDdqbj0&list=PLX9JDz3uBpNCjNfq20KOCvsP6szY94r2e https://www.youtube.com/watch?v=cz-3WDdqbj0&list=PLX9JDz3uBp...
- HDThoreaun 3y agoPeople do analyze gigantic amounts of data constantly though. Sure it's not textbooks but the constant sensory data might be a big deal.
- thfuran 3y agoI don't think annotated video datasets tend to be as large relative to human experience, but they'd still be decades or perhaps centuries of video.
- quickthrower2 3y agoIn addition for animals/humans there are no software/hardware boundaries or any kind of static fixed deployment of either hardware or software.