3 ms·
Fun thing to try: Ask an LLM: "Say the word APPLE", but modify the code so the logits of the token for Apple/apple/APPLE is permanently set to -Inf - ie. the
by londons_explore 1y ago
Fun thing to try:
Ask an LLM: "Say the word APPLE", but modify the code so the logits of the token for Apple/apple/APPLE is permanently set to -Inf - ie. the model cannot say that word.
The output ends up like this:
"Banana. Oh, just kidding. Banana. Oh, it's so tasty I said it wrong. Lets try again: Orange. Whoops, I meant to say grape. No I meant to say the tasty crunchy fruit known as a carrot".....
- londons_explore 1y agoNote that OP's traffic light problem would suffer the same problem. Ie. a smart model, knowing it cannot say a word, will give the next best solution - for example maybe saying "A P P L E" or maybe "I'm afraid I'm not able to do that". However, a constrained model does not know or understand its own constraints, so keeps trying to do things which aren't allowed - and even goes back and tries to redo these things which aren't allowed, because to the model it is a mistake which needs correcting.
- adammarples 1y agoThere's a whole field of solving constrainted optimization and it doesn't really work like that, but they don't use LLMs.
- jcims 1y agoLike your brain when you know you know a word but it's just not surfacing in your mind. I'm guessing I'm not that different from the average human and I can 'feel' something physically while I'm searching for the word. I've always wondered what that was.
- stavros 1y agoI saw this exact thing in a question about who was the first composer, the model kept outputting Boethius and then saying "NO!", as if it couldn't escape its own Freudian slips.