> Sample many times and vote is a highly effective (but slow) strategy. Beam search[1] has long been a great way to sample from language models, even before tr

> Sample many times and vote is a highly effective (but slow) strategy. Beam search[1] has long been a great way to sample from language models, even before transformers. Essentially you keep track of the top N most promising threads and sample randomly from those. OpenAI doesn't offer beam search yet, just temperature and top_k, but I hope they add support for it because it's far more efficient than just starting over each time. [1]: https://www.width.ai/post/what-is-beam-search https://www.width.ai/post/what-is-beam-search