3 ms·
It's more like someone whispered dog into your ears while you were unconscious, and you were unable to recall any conversation but for some reason you were thin
by DangitBobby 1y ago
It's more like someone whispered dog into your ears while you were unconscious, and you were unable to recall any conversation but for some reason you were thinking about dogs. The thought didn't enter your head through a mechanism where you could register it happening so knowing it's there depends on your ability to examine your own internal states, i.e., introspect.
- munro 1y agoI'm more looking at the problem more like code https://bbycroft.net/llm https://bbycroft.net/llm My immediate thought is when the model responds "Oh I'm thinking about X"... that X isn't from the input, it's from attention, and thinking this experiment is simply injecting that token right after the input step into attn--but who knows how they select which weights