3 ms·
Well, that an autoregressive model has parameters does not mean it is not autoregressive. LLMs are not Markovian.
by bjourne 1y ago
Well, that an autoregressive model has parameters does not mean it is not autoregressive. LLMs are not Markovian.
- sailingparrot 1y agoAt no point have I argued that LLMs aren’t autoregressive, I am merely talking about LLMs ability to reason across time steps, so it seems we are talking past each other which won’t lead anywhere. And yes, LLM can be studied under the lens of Markov processes: https://arxiv.org/pdf/2410.02724 https://arxiv.org/pdf/2410.02724 Have a good day