That's quite an interesting comparison. I like the description of both as Sequential Monte-carlo sampling from a desired distribution. But I think there are two

That's quite an interesting comparison. I like the description of both as Sequential Monte-carlo sampling from a desired distribution. But I think there are two crucial differences. First, in Bayesian sampling, the initial value and first samples are not sampled from the desired distribution. In a well trained LLM, the prompt is given and the first response is sampled from the desired distribution (of text that is likely to follow the prompt). Second, in Bayesian sampling, the fact that the samples aren't independent is an unwelcome but unsolvable problem. We want independent samples but can't generate them, so we settle for conditionally independent samples. In an LLM, we want each sample to be dependent on the preceding text, in particular the prompt. In summary: Bayesian sampling - poorly chosen "prompt" (the initial sample), future samples would ideally be independent of the prompt and each other. LLM sampling - carefully chosen prompt, future samples are ideally dependent on the prompt and on each other. And in conclusion: The warm up period helps a Bayesian sampler find values that are less dependent on the initial "prompt", which we definitely don't want in an LLM.