5 ms·

It can only predict words, is like saying all neuron's are just switches, all matter is just protons, electrons and neutrons. It's just reductive. What we're s

It can only predict words, is like saying all neuron's are just switches, all matter is just protons, electrons and neutrons. It's just reductive. What we're seeing from LLMs is simply amazing considering what they are supposed to just do. Whatever LLMs are capable of, big or small it exceeds such reductive thinking as 'it predicts words'.

clarge1120 4y ago
One expects the academic community to step in and provide this kind of analysis, namely that an LLM combined with a great deal of computing capacity exhibits capabilities greater than the sum of its parts.
- ftxbro 4y ago
  I'm a long time LLM enjoyer and by far the best analysis I've seen for that is https://generative.ink/posts/simulators/ https://generative.ink/posts/simulators/ but it's way "too much" for normal people who are learning about stochastic parrots and blurry jpegs instead.
janalsncm 4y ago
Exactly. These kinds of reductive arguments always bother me because they ignore emergent effects. Chess engines are “just” running minimax? Well guess what, they’re evaluating millions of positions per second. If this whole intelligence thing can be reduced to “just” predicting the next word, maybe we’re not as special as we thought. In fact anyone who reads the Wikipedia article on cetacean intelligence will quickly realize that we aren’t the hot shit we once thought we were.
kenjackson 4y ago
And "predicting words" given a huge corpus of interesting text by itself seems like it could be amazing. Predicting words effectively can mean that it has internalized all the logic used to generate the original texts. At some extrema simply predicting words could mean more than just "knowing facts", but actually having the accumulated smarts of those that made large contributions to the corpus. Allow me to participate in some hyperbole here, but "predicting words" could result in a model that is "smarter" in most capacities than most humans.
- blueorange8 4y ago
  It's basically creating an accurate world view

clarge1120 4y ago

One expects the academic community to step in and provide this kind of analysis, namely that an LLM combined with a great deal of computing capacity exhibits capabilities greater than the sum of its parts.

ftxbro 4y ago
I'm a long time LLM enjoyer and by far the best analysis I've seen for that is https://generative.ink/posts/simulators/ https://generative.ink/posts/simulators/ but it's way "too much" for normal people who are learning about stochastic parrots and blurry jpegs instead.

janalsncm 4y ago

Exactly. These kinds of reductive arguments always bother me because they ignore emergent effects. Chess engines are “just” running minimax? Well guess what, they’re evaluating millions of positions per second. If this whole intelligence thing can be reduced to “just” predicting the next word, maybe we’re not as special as we thought. In fact anyone who reads the Wikipedia article on cetacean intelligence will quickly realize that we aren’t the hot shit we once thought we were.

kenjackson 4y ago

And "predicting words" given a huge corpus of interesting text by itself seems like it could be amazing. Predicting words effectively can mean that it has internalized all the logic used to generate the original texts. At some extrema simply predicting words could mean more than just "knowing facts", but actually having the accumulated smarts of those that made large contributions to the corpus. Allow me to participate in some hyperbole here, but "predicting words" could result in a model that is "smarter" in most capacities than most humans.

blueorange8 4y ago
It's basically creating an accurate world view