3 ms·
I'm seeing this as well. I don't quite understand how it's doing that in the context of LLMs to date being a "next token predictor". It is writing code, then ad
by numlocked 2y ago
I'm seeing this as well. I don't quite understand how it's doing that in the context of LLMs to date being a "next token predictor". It is writing code, then adding more code in the middle.
- sebzim4500 2y agoAre you seeing a `<span>...` suffix when the model is in the middle of writing a code block? I'm pretty sure that's just a display bug in lmsys.
- concurrentsquar 2y agoIs it something similar to beam search (https://huggingface.co/blog/how-to-generate#beam-search https://huggingface.co/blog/how-to-generate#beam-search) or completely different (probably is not beam search if it's changing code in the middle of a block)? (I can't try right now because of API rate limits)