4 ms·

Related to this, our min_p paper was ranked #18 out of 12000 submission at ICLR and got an oral: https://iclr.cc/virtual/2025/oral/31888 https://iclr.cc/virtua

Related to this, our min_p paper was ranked #18 out of 12000 submission at ICLR and got an oral: https://iclr.cc/virtual/2025/oral/31888 https://iclr.cc/virtual/2025/oral/31888 Our poster was popular: poster: https://iclr.cc/media/PosterPDFs/ICLR%202025/30358.png?t=1745327604.27015 https://iclr.cc/media/PosterPDFs/ICLR%202025/30358.png?t=174... oral presentation (watch me roast yoshua bengio on this topic and then have him be the first questioner, 2nd speaker starting around 19:30 min mark. My slides for the presentation are there too and really funny.): https://iclr.cc/virtual/2025/session/31936 https://iclr.cc/virtual/2025/session/31936 paper: https://arxiv.org/abs/2407.01082 https://arxiv.org/abs/2407.01082 As one of the min_p authors, I can confirm that Top N sigma is currently the best general purpose sampler by far. Also, temperature can and should be scaled far higher than it is today. Temps of 100 are totally fine with techniques like min_p and top N sigma. Also, the special case of top_k = 2 with ultra high temperature (one thing authors recommend against near the end) is very interesting in its own right. Doing it leads to spelling errors every ~10th word - but also seems to have a certain creativity to it that's quite interesting.

toxik 1y ago
Are there any samplers that aren't basically greedy? I.e. actually searches the tree. I realize it's an absolutely insane branching factor and quite expensive to expand nodes at that, but it always seemed odd to me that we don't actually search.
- Kubuxu 1y ago
  Beam Search sampling is sometimes getting used
- Der_Einzige 1y ago
  Besides beam search and it's variants? (there are many including the little known but awesomely powerful constrained beam search: https://huggingface.co/blog/constrained-beam-search https://huggingface.co/blog/constrained-beam-search) Does MBR (minimal bayes risk) sampling count? Also there was this paper at ICLR which is relevant to this question: https://arxiv.org/abs/2410.03968 https://arxiv.org/abs/2410.03968 This paper basically claims that non-heuristic methods (like beam search) are harmful compared to the heuristic ones.

toxik 1y ago

Are there any samplers that aren't basically greedy? I.e. actually searches the tree. I realize it's an absolutely insane branching factor and quite expensive to expand nodes at that, but it always seemed odd to me that we don't actually search.

Kubuxu 1y ago
Beam Search sampling is sometimes getting used
Der_Einzige 1y ago
Besides beam search and it's variants? (there are many including the little known but awesomely powerful constrained beam search: https://huggingface.co/blog/constrained-beam-search https://huggingface.co/blog/constrained-beam-search) Does MBR (minimal bayes risk) sampling count? Also there was this paper at ICLR which is relevant to this question: https://arxiv.org/abs/2410.03968 https://arxiv.org/abs/2410.03968 This paper basically claims that non-heuristic methods (like beam search) are harmful compared to the heuristic ones.