5 ms·
Chain of thought is very weird and very impressive. When you watch it work, it looks like a series of flash-frozen human voices being recruited onto the page. W
by BriggyDwiggs42 2y ago
Chain of thought is very weird and very impressive. When you watch it work, it looks like a series of flash-frozen human voices being recruited onto the page. We know, of course, that these are the ones that happen to produce the responses we want at the end of the chain, but there isn’t imo an underlying meaning being communicated through the words. It’s a Chinese room responding to itself, built to produce desirable paths. Humans mostly don’t work that way. The subconscious can talk to itself without words, and it’s only the most blunt, underspecified ideas that get put into internal monologue.
- drakenot 2y agoPeople have proposed doing CoT not with output tokens, but to do it in the latent space of the model.
- BriggyDwiggs42 2y agoIf they can get that working, I’d expect it to be much more powerful.
- pillefitz 2y agoIt's been done already https://aipapersacademy.com/chain-of-continuous-thought/ https://aipapersacademy.com/chain-of-continuous-thought/