3 ms·
I was wondering on what basis @Sama keeps saying they are near AGI, when in reality LLMs just calculate sequences and probabilities. I really doubt this bubble
by sreekanth850 1y ago
I was wondering on what basis @Sama keeps saying they are near AGI, when in reality LLMs just calculate sequences and probabilities. I really doubt this bubble is going to burst soon.
- jjtheblunt 1y agoI'm unaware of any proof (in the mathematician sense, for example) that _we_ aren't just kickass machines calculating sequences at varying probabilities, though. perhaps that is how the argument persists?
- sreekanth850 1y agoEfficiency matters. We do it with a fraction of the processing power.
- jjtheblunt 1y agotrue in the caloric/watts sense, but we might well have way higher computational power architecturally?
- sreekanth850 1y ago100%, we do a lot more in real life. There are many circumstances, where you work without prior training. This is how new inventions happening everytime.
- lispybanana 1y agoHumans do this but this is not all they do. How do we explain humans who invent new concepts, new words, new numerical systems, new financial structures, new legal theories. These are not exactly predictions (since they don't exist in a training set) but they may be composed from such sets.
- in-silico 1y ago> How do we explain humans who invent new concepts Simple: they are hallucinations that turn out to be correct or useful. Ask ChatGPT to create a million new concepts that weren't in its training data and some of them are bound to be similarly correct or useful. The only difference is that humans have hands and eyes to test their new ideas.
- lispybanana 1y agoHow does NN create a concept not in its training data? (Does it explore negative idea-space?) What if a concept uses a word not yet invented? How does LLM produce that word and what cosine similarity would such a word have if it's never appeared next to others? How would we know if such a word is useful?