4 ms·
> and markov chain don't work on token embedding that allows some encoding of meaning. Working on an "encoding of meaning" sure sounds a lot like reasoning.
by AlecSchueler 4mo ago
> and markov chain don't work on token embedding that allows some encoding of meaning.
Working on an "encoding of meaning" sure sounds a lot like reasoning.
- tanseydavid 4mo agoConsider the classic ambiguity example from Natural Language Processing: "Time flies like an arrow; fruit flies like a banana". Markov chains are mostly about memorized local co-occurrence, while transformers are much better at context-sensitive disambiguation and compositional structure. I do not have deep knowledge about these subjects but this is what I think was meant by "encoding of meaning" (LLMs). I also do not think this capability has to be thought of as "reasoning".