4 ms·

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

by toxik 3y ago

dchest 3y ago
Try running llama.cpp with 0 temperature and without repetition penalty and you'll sometimes get caught in a loop. Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2 https://transformer.huggingface.co/doc/distil-gpt2 https://imgur.com/a/hF1Ewk4 https://imgur.com/a/hF1Ewk4
avianlyric 3y ago
That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops. That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to select any output token that already in the input, to more complex stochastic process that explore the tree of possible outputs, to find branches whose overall result is improved by choosing less optimal immediate steps. The vast majority of what make Chat GPT different to simpler GPT-3 models is this complex post-processing phase that allows designers to push and pull on the behaviour of the pre-baked static model underlying the chat interface.
- cztomsik 3y ago
  Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for. My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough. Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.
cztomsik 3y ago
It is very common for small models, and sometimes even 7B model gets stuck. There was an article/paper showing that GPTs (whole family) quickly get confident in looping if there is anything loop-like in the window. So basically, as soon as it loops once, it will never get out of it. Note that sometimes loops are desired, like with docstring in the code, always starting before the function definition, and other structural things.

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

by toxik 3y ago

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

dchest 3y ago

Try running llama.cpp with 0 temperature and without repetition penalty and you'll sometimes get caught in a loop. Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2 https://transformer.huggingface.co/doc/distil-gpt2 https://imgur.com/a/hF1Ewk4 https://imgur.com/a/hF1Ewk4

avianlyric 3y ago

That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops. That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to select any output token that already in the input, to more complex stochastic process that explore the tree of possible outputs, to find branches whose overall result is improved by choosing less optimal immediate steps. The vast majority of what make Chat GPT different to simpler GPT-3 models is this complex post-processing phase that allows designers to push and pull on the behaviour of the pre-baked static model underlying the chat interface.

cztomsik 3y ago
Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for. My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough. Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.

cztomsik 3y ago

It is very common for small models, and sometimes even 7B model gets stuck. There was an article/paper showing that GPTs (whole family) quickly get confident in looping if there is anything loop-like in the window. So basically, as soon as it loops once, it will never get out of it. Note that sometimes loops are desired, like with docstring in the code, always starting before the function definition, and other structural things.