3 ms·
What a weirdly snarky comment (it's like your third one in this thread, maybe you need to take a breather). I wasn't comparing them, though there's an obvious s
by dvt 4y ago
What a weirdly snarky comment (it's like your third one in this thread, maybe you need to take a breather). I wasn't comparing them, though there's an obvious stochastic argument to be made; I was just commenting on the former's lack of complexity and still fascinatingly good results, I'm not sure what your point is.
- krackers 4y agoMarkov chains are actually not a bad comparison from what i can see as an utter layman. At a high level seems like the difference between transformer-models and markov chains is that instead of the next token being determined purely based on n-gram frequencies on the training corpus, you have the attention mechanism so in some sense it has access to the entire previous context as well as some latent space of learned embeddings. But at the end of the day it still is just a fancy way to predict the next token, so that's why it's surprising you get all this neat behavior.