5 ms·
If someone asks me "What is your name?" my reply is also simply just guessing what token would go well next in the full text of the conversation.
by vasco 3y ago
If someone asks me "What is your name?" my reply is also simply just guessing what token would go well next in the full text of the conversation.
- miningape 3y agoNo you "know" what your name is and you retrieve that information. It's infinitely different to rolling a dice and picking a name based on what number comes up
- agumonkey 3y agoSometimes I'm tempted to think that my brain is as fuzzy if not more than an LLM
- realusername 3y agoI feel like I work in similar ways, if you ask my name there's a probability that I'll answer with the shorten version or the longer one pretty much randomly and there's no conscious effort about it. While the LLMs do have hallucinations, basic stuff like this will never trigger any. Where I feel the most differences are aren't in the token concept but rather on the deep reasoning (which we don't use as much in my opinion)
- astrange 3y agoThis is two different parts of the system. > No you "know" what your name is and you retrieve that information. The LLM has this (literally - most of it is a key value store). It outputs a probability for all possible next tokens. > It's infinitely different to rolling a dice and picking a name based on what number comes up The sampling algorithm running the LLM then does this part, and adds randomness to make it more "creative". So if you want factual information then don't be so random you skip over the right answer.
- poniko 3y agoIt's really not though, generally maybe but you also give it a split second to think about if it's a good time to lie, make a joke or maybe just not reveal your name.
- vasco 3y agoAnd any of those choices will be your best guess at the next tokens in the text of the conversation respecting not only the conversation but also your self image and surroundings.
- vrighter 3y agowhat if you're the only john in a world of steves? What would happen then?
- vasco 3y agoI'd say, hi I'm john
- input_sh 3y agoIf asked repeatedly, are you gonna answer that question with a different name, depending on which "memory" you randomly pick from?
- kevindamm 3y agoSomebody with MPD might.
- edflsafoiewq 3y agoThat's an interesting example, since an LLM has no concept of a "self" and literally does not know who it is. It can only answer it "correctly" if you prefix it with a prompt telling it who and what it is.
- throaway893 3y agoYou also have been told your name, most likely by your parents. It also had to be explained to you what you are, at some point in your life.