4 ms·
> I had the model look at some existing code and rank the probability of each token appearing given the previous tokens. ... is the "AI" actually just a markov
by blueflow 4y ago
> I had the model look at some existing code and rank the probability of each token appearing given the previous tokens.
... is the "AI" actually just a markov chain?
- eschluntz 4y agoLarge language models have the same interface as a Markov chain - they're just predicting likelihoods of next tokens. You generate text by sampling from that likelihood distribution.
- pigscantfly 4y agoNo, an autoregressive language model is conditioned on all prior states, not the previous one.
- blueflow 4y agoMultiply out the states, "all prior states" is then the "previous one". Easy to model as Markov chain.
- adgjlsfhk1 4y agothis is technically true but the Markov chain would be too big to store even with petabytes of storage.
- tgv 4y agoIndeed. The argument boils down to: since it's finite, I can turn it into a FSA. Not only is that unhelpful, it doesn't tell you how to construct it, i.e. the learning process.
- The_Amp_Walrus 4y agoAlso 'easy' to model as a lookup table containing all possible solutions.
- deleted 4y ago[deleted]
- devmor 4y agoModels like this are essentially extremely efficient markov chains. Calling them AI is disingenuous, like most things we call AI lately.