5 ms·
So, almost every possible ngram ever used is a 'input'? Can someone describe what the lexical reality of 3.5T inputs actually means? I feel like this is 'Deep
by jariel 6y ago
So, almost every possible ngram ever used is a 'input'?
Can someone describe what the lexical reality of 3.5T inputs actually means?
I feel like this is 'Deep Memorization' instead of 'Deep Learning'.
Like a Doctor who passes everything merely by memorizing the textbook with absolutely no ability beyond that.
- visarga 6y agoEven so, memorisation implies word by word information retrieval and interpolation, it's not a hash table.
- jariel 6y agoYes, I mean, that's cool, but I'm just thinking that this isn't quite the AI we were thinking about before. It's like a 'new form of storage and lookup' as opposed to the kind of 'magic algorithm' we usually think of when we think of AI. Or maybe that's just me.
- azinman2 6y agoMagic algorithms don’t exist. That’s the true reality of AI. I also felt the same thing and became dismayed as an undergrad at what was going on in computation having also studied neurobiology. But as it turns out, a shit ton of data with some stats can get you very, very far.
- jariel 6y agoI didn't mean 'magic' (I know it's not that) - I just meant to imply that I think of AI as a 'function' not a 'lookup'. Inputs -> Outputs not Search -> Response Like if you train AI on a small dataset, it feels like what it's doing afterwords is a 'function' or 'algorithm' using what is in the end some arcane algebra. But if you train on all the data in the world, with a trillion parameters ... well ... I kind of feel that 'all the data' is in an AI-style datastructure, that we are 'querying' with AI. But that's just an observers abstraction.
- azinman2 6y agoSee Searle’s Chinese room.
- feanaro 6y agoMagic algorithms (or perhaps magic machines) do exist, though. See: humans.
- mlajtos 6y agoWe have ~150 trillion synapses. If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines.
- qayxc 6y ago> If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines. It's a poor comparison, though, since neurons and synapses aren't the same as parameters in a computational network. It's the same trap that news outlets routinely fall into when citing "storage capacity of the brain" in TBs and other such nonsense.
- deleted 6y ago[deleted]