5 ms·
I won't speak for what's "all the way down", but on whatever level humans perform reflective reasoning, it's self-evident that we don't simply "predict the next
by root_axis 4y ago
I won't speak for what's "all the way down", but on whatever level humans perform reflective reasoning, it's self-evident that we don't simply "predict the next word" in the way a LLM does.
Whether or not a LLM is capable of "creating anything new" is a different argument altogether. "New" is in eye of the beholder and it is often said that nothing is new under the sun, but we already know that unthinking processes within nature are capable of producing beauty far and beyond anything that even the greatest human artist could ever hope to accomplish. That a large LLM is unthinking doesn't preclude it from producing art.
- tambourine_man 4y agoPlease provide evidence that intelligent speech isn’t simply predicting the next word, as it’s obviously not self-evident to me.
- lukev 4y agoI'm not sure it counts as "evidence" because I can't show it to you, but based on personal introspection, whenever I write something it starts with a barely-linguistic thought I want to communicate that gradually gets expanded into as much prose as is necessary to get across the concept whilst establishing context and defending from the most probable detractors. I literally never just start writing and then just keep adding words.
- tambourine_man 4y agoIt’s not evidence, sure, but what you describe, the “feel of it” is most of what we have to work with at the moment, unfortunately. But it clearly is not self-evident that intelligence is or isn’t prediction. Nothing is self-evident in a field we know so very little about. That’s my main objection, we should be humble and curious, not dogmatic.
- orwin 4y agoIt's pretty much self-evident when you worked with 3-6 that this isn't how it work. The idea come before the vocalization. Or even when you talk in a foreign language you're semi-competent with.
- tambourine_man 4y agoI’ve spent many years with infants and learned 3 languages aside from my mother tongue. I don’t find it self-evident at all. You’re gonna need to elaborate why you think so if you’re willing to convince someone.
- orwin 4y agoThat the idea form before it's vocalized? A three year old reaching toward any toy, then saying 'mine!'. Stumbling on a specific word, stopping your sentence, then proceed to replace it with a combination of words. Or any time you paraphrase basically.
- tambourine_man 4y agoNot true in my experience. To me, words come first, the idea is a consequence I realize after a few of them have been stringed together. Knowing where the words come from, how “the brain” knows the next one before my conscious self does has always been a mystery to me. I’ve heard Sam Harris describing a similar feeling in his podcast.
- orwin 4y agoYou never paraphrased something despite knowing the exact word? [edit] Or explained high level CS concepts to an undergrad at your school?
- tambourine_man 4y agoSure, but how's that indicative that prediction isn't taking place?
- phaedrus 4y agoI've always hated the idea promoted by some AI researchers that "intelligence is just prediction." Specifically in the context of your challenge, since a brain must decide the next word before uttering it, is it not vacuously true that any conceivable method of producing speech is "predicting" it? You're asking for evidence that it isn't; I'm asking for evidence that question has any meaning.
- tambourine_man 4y agoSure, any production in that case can be classified as prediction. What I’m saying is that such distinction, if at all possible, is far from self-evident.
- breckenedge 4y agoAnytime you say “uh, what’s that word again?” So you’re thinking of a concept, you can picture it and it’s impact on your environment, but can’t quite remember the word for it.
- tambourine_man 4y agoOr, you reached a point in the Markov chain where none of the next guesses rank high enough if the whole paragraph is taken into account. It often feels like that, in fact, you can’t figure it out until you take another “route”. Of course, my anecdote is just as valid as yours. My point is we have no freaking idea what’s going on and shouldn’t jump to conclusions.
- root_axis 4y agoThe content of intelligent speech is not driven by statistical relationships between the spoken lexemes, this is self-evident. For example, let's say you're planning to approach your boss about receiving a raise, the contents of your speech will be driven by a personal theory of mind regarding how your boss perceives you, your own memory of who you are and how you like to be perceived, the cadence and tenor of speech you imagine your boss is receptive to, information contextually relevant to the purpose of the speech like your proven history of success within the organization, as well as a hundred other variables driven by physiological factors, culture, social habituation and more. "Predict the next word" just doesn't describe intelligent speech in any meaningful way.
- tambourine_man 4y agoSo, you thrown in theory of mind, physiological factors and culture without defining what that means or showing that they don't all derive from the same underlying statistical relationships in order to make your argument more robust? That's not how it works. Defining intelligence is fundamental philosophical problem, a discussion that's thousands of years old in the making. To call something in this field self-evident is either ignorance or self denial.
- deleted 4y ago[deleted]
- tarsinge 4y agoSomeone deaf and blind still form thoughts but don’t output them in words. Words follow cognition, it’s sometimes obvious when you have a clear concept in mind but forget the word. Edit: also when programming, I form mental structures and code them without verbalizing them, same with maths or when drawing or making music. A thought is maybe a prediction, but language seems not to be the abstraction it’s operated on.
- tambourine_man 4y agoOf course they do. They still have language. How else would they interact with the world?
- pnt12 4y agoYou're asking for the evidence of a non existing factor in a complex process not fully understood... I'd say it's on you to provide it. Why is predicting the next word proof of intelligent speech?
- tambourine_man 4y agoThat’s not what I’m asking. The grandparent said it’s self-evident that humans do not simply predict the next word. I asked for evidence of that self-evidence.