3 ms·
Interesting that he came to this conclusion (CoT should be done in latent space) well before the release of OpenAI's o1, which made explicit CoT reliable in the
by cubefox 11mo ago
Interesting that he came to this conclusion (CoT should be done in latent space) well before the release of OpenAI's o1, which made explicit CoT reliable in the first place. At the time the blog post was written, CoT was only achieved via a "reason step by step" instruction, which was highly error prone compared to modern o1-like reasoning. (And before InstructGPT/ChatGPT, it was achieved by prompting the model with "let me reason step by step".)
- jdthedisciple 11mo agoFair to say he demonstrates good CoT reasoning capabilities?
- cubefox 11mo agoIsn't this a form of intuitive intelligence which doesn't rely on "reasoning"? To me "reasoning" sounds like intentionally trying to solve some explicit problem, while another form of ... insight is the ability to figure out that something is a problem in the first place. That's, by the way, something LLMs are very much not good at. They possess a superhuman amount of knowledge covering all areas of academia, including math, science, philosophy, engineering, computer science, social sciences and so on, but that doesn't cause them to come up with novel hypotheses and theories. Something that would be easy for a smart human even with a fraction of the academic knowledge of an LLM.