3 ms·

Could someone ELI5? It sounds like they generate a compressed token which represents a whole "thought" rather than elaborating the entire "thought" in actual la

• ipunchghosts 2y ago
  Currently, when AI models solve problems, they write out long chains of thoughts (like showing their work in math). While helpful, this takes up a lot of computing power. Heima does something clever - instead of writing out long explanations, it compresses each step of thinking into a single "thinking token." Think of it like using a shorthand symbol instead of writing out a full sentence.
  • Ancapistani 2y ago
    I've been doing a lot of introspection lately about how I think. I lack the terminology here unfortunately, but your description here sounds super familiar: > instead of writing out long explanations, it compresses each step of thinking into a single "thinking token." Think of it like using a shorthand symbol instead of writing out a full sentence. I have clear memories of how cognition worked for me before I understood spoken language. I recall thinking in concepts - kind of a weird mix of forms, motions, and intent. I know this sounds metaphysical, but that's not my intent. I just don't have the words to explain it. I wish I did, though, because my very early memories of self-awareness certainly seem to map well onto the current state of AI development.
    • fzzzy 2y ago
      Plenty of people don't think with an internal monologue or internal imagery.