4 ms·
>represents UNDERSTANDING of the input language. It does not have an understanding, it pattern matches the "idea shape" of words in the "idea space" of trainin
by qsera 8mo ago
>represents UNDERSTANDING of the input language.
It does not have an understanding, it pattern matches the "idea shape" of words in the "idea space" of training data and calculates the "idea shape" that is likely to follow considering all the "idea shape" patterns in its training data.
It mimics understanding. It feels mysterious to us because we cannot imagine the mapping of a corpus of text to this "idea space".
It is quite similar to how mysterious a computer playing a movie can appear, if you are not aware of mapping of movie to a set of pictures, pictures to pixels, and pixels to co-ordinates and colors codes.
- basch 8mo agoSemantics. Its a encoded position that represents meaning in a way that is useful and reusable. That is "understanding." It's a mathematical representation of grasp.
- qsera 8mo agoYea, semantics is important. It is not "understanding" any more than a microphone+ADC is hearing.
- basch 8mo agoagree to disagree. encoding a meaning is understanding. I cited a source using the word in the same way.
- qsera 8mo ago>agree to disagree. Yea >encoding a meaning is understanding. encoding a meaning is encoding. Nothing more!
- basch 8mo agowhat is understanding but encoded meaning distilled into pure structure, in both cases a property of a pattern? No need to gatekeep the word "understanding" behind subjective human experience eg qualia.
- qsera 8mo ago> No need to gatekeep the word "understanding" behind subjective human experience eg qualia. Yea, I think gatekeeping is needed exactly for the same reason. Make up another word if you want..
- joquarky 8mo agoIf you make up a word, nobody will know what it means.
- qsera 8mo agoAll of these communication can be done without using such words. It would just appear less "magical". This is done is the guise of dumbing it down, but people sometimes take it quite literally, which is what the marketing wants anyway..
- fc417fc802 8mo agoThe distinction you're making reads like substance dualism to me. Are you able to provide a clear and objective metric for assessing "understanding"? If not then you're just handwaving an effectively meaningless semantic distinction.
- qsera 8mo ago>objective metric for assessing "understanding" It should involve consciousness. You would not call an AI reacting to red color as "seeing" red. Same thing.
- fc417fc802 8mo agoAnd where is this objective metric for consciousness? Last I checked we didn't even have a sensible definition for it. It seems to me you're just kicking the can. Setting that issue aside. While I certainly don't believe LLMs to be conscious (an entirely subjective and arbitrary take on my part I admit) I don't see any reason that concepts such as "intelligence" and "understanding" should require it. When considering how we apply those terms to humans it seems to me they are results based and highly contextual (ie largely arbitrary).
- qsera 8mo ago>humans it seems to me they are results based and highly contextual (ie largely arbitrary). Is that right? It seems that we generally say that "the computer is programmed to do", instead of "the computer understand" or "the computer knows", even if the programmed computer can produce the same result as a human who does it.
- gkbrk 8mo agoA microphone + ADC is hearing though, that's the whole reason we even produce microphones. So that our electronics can hear sound.
- qsera 8mo agoSo according to you when can you qualify something as capable of hearing 1. Vibrate according input to the sound, is that hearing? 2. Generate electrical signals according to the sound, is that hearing? 3. Amplify electrical signal, does we cross the hearing mark? 4. Record the signal to a cassete tape (or use an ADC -> mp3), are we hearing yet? 5. Play it back through a speaker. Sure, we should be hearing now! At which point exactly would you say the thing is definitely hearing?
- fc417fc802 8mo agoYou can reduce the human auditory process to a similar mechanical list. At which specific point would you say a human is hearing? You've fallen into the trap of human exceptionalism but you don't seem to be aware of that fact. Are you a substance dualist or not?
- qsera 8mo ago>You can reduce the human auditory process to a similar mechanical list. You can't. Because we don't know at which point sound gets registered in consciousness.
- fc417fc802 8mo agoBecause you can't even define what consciousness is, let alone objectively test for it. You are entirely wrong though. You most certainly _can_ reduce the human auditory process to a (bio)mechanical list. You have unilaterally, arbitrarily, and without justification added consciousness to that list.
- qsera 8mo ago
- aqua_coder 8mo agoI am not knowledgeable on how transformer works but, what if, us humans just do the same thing in our minds as well ? What if our feeling of "understanding" is merely just the emotional response to a pattern matching as you just said?
- qsera 8mo agoYea, you said it. It is the feeling of understanding and feeling/sensing implies consciousness. Why does it matter? I don't know. All I know is that it is not the same thing, because a chunk of metal cannot feel. So I don't want it to be called by the same name. When AI marketing (ab)uses the word, it is to project the appearance of human equivalence. And I don't like to fall for it.
- dnautics 8mo ago> pattern matches the "idea shape" of words in the "idea space it does much more than this. first layer has an attention mechanism on all previous tokens and spits out an activation representing some sum of all relations between the tokens. then the next layer spits out an activation representing relations of relations, and the next layer and so forth. the llm is capable of deducing a hierarchy of structural information embedded in the text. not clear to me how this isn't "understanding".