The probabilities are fun (and ability to manipulate/mask/logit bias them), but small nit pick: Beam search isn't popular because it results in less human like

The probabilities are fun (and ability to manipulate/mask/logit bias them), but small nit pick: Beam search isn't popular because it results in less human like text and takes more processing time, as a concept it sounds great in theory, but it's generally worked worse in practise compared to the next token with temperature, and results in text that is not human like, the gap is larger with larger models too, it doesn't get better: https://blog.fastforwardlabs.com/images/2019/05/Screen_Shot_2019_05_08_at_3_06_36_PM-1557342561886.png https://blog.fastforwardlabs.com/images/2019/05/Screen_Shot_... The way most GPT models are set up, they don't edit, but there was a lot of work (made redundant by OpenAI) on using gap filling tasks (masking) to iteratively improve or construct or otherwise edit and formulate logic using the encoder-decoder models of yesteryear - in many ways, I think it would have been more exciting if that approach powered the likes of 'think step-by-step' results.