3 ms·

Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks appl

by dchest 3y ago

Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks applied, such as repetition/frequency penalty.

cubefox 3y ago
Such penalties would only exist for fine-tuned models. Base models have only the temperature setting. As the example at the end shows, even GPT-2 seems "smarter" than the Apple model, probably because of the number of parameters.
- dchest 3y ago
  There's no such thing as "base models have only the temperature setting". Models do not have sampling settings (temperature, repetition penalty, etc), the sampling code does, which obviously you can use on any model. For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp#L4893-L4951 https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp... Here's the one from transformers: https://github.com/huggingface/transformers/blob/0a55d9f7376f72ad3ff296d4249840021b03bcc4/src/transformers/generation/logits_process.py#L270 https://github.com/huggingface/transformers/blob/0a55d9f7376... To summarize how they work: you keep some number of previously generated tokens, and once you get logits that you want to sample a new token from, you find the logits for existing tokens and multiply them by a penalty, thus lowering the probability of the corresponding tokens.
  - cubefox 3y ago
    I don't think such penalties were applied to GPT-2 or even GPT-3, yet they weren't repetitive like that.
    - dchest 3y ago
      Yes, they are applied. Here's OpenAI doc which describes how to set various sampling parameters for GPT-3: https://platform.openai.com/docs/api-reference/completions/create https://platform.openai.com/docs/api-reference/completions/c... See presence_penalty and frequency_penalty. Sampling techniques is one of important arts of LLMs, you'll can find a lot of papers on them. In general, smaller are more prone to repetition, but you can get caught in it even with larger models.
      - dchest 3y ago
        To clarify: I meant that in general they are commonly applied, but in this case they weren't as the author confirmed. The repetition, of course, doesn't happen all the time.
    - jackcook 3y ago
      Yes, you're right, I should have mentioned it in the post, but I used pure greedy sampling for the GPT-2 outputs since I couldn't do anything but that for the Apple model. So temperature was set to zero, and there was no repetition penalty.
toxik 3y ago
I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.
- dchest 3y ago
  Try running llama.cpp with 0 temperature and without repetition penalty and you'll sometimes get caught in a loop. Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2 https://transformer.huggingface.co/doc/distil-gpt2 https://imgur.com/a/hF1Ewk4 https://imgur.com/a/hF1Ewk4
- avianlyric 3y ago
  That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops. That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to select any output token that already in the input, to more complex stochastic process that explore the tree of possible outputs, to find branches whose overall result is improved by choosing less optimal immediate steps. The vast majority of what make Chat GPT different to simpler GPT-3 models is this complex post-processing phase that allows designers to push and pull on the behaviour of the pre-baked static model underlying the chat interface.
  - cztomsik 3y ago
    Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for. My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough. Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.

by dchest 3y ago

cubefox 3y ago

Such penalties would only exist for fine-tuned models. Base models have only the temperature setting. As the example at the end shows, even GPT-2 seems "smarter" than the Apple model, probably because of the number of parameters.

dchest 3y ago
There's no such thing as "base models have only the temperature setting". Models do not have sampling settings (temperature, repetition penalty, etc), the sampling code does, which obviously you can use on any model. For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp#L4893-L4951 https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp... Here's the one from transformers: https://github.com/huggingface/transformers/blob/0a55d9f7376f72ad3ff296d4249840021b03bcc4/src/transformers/generation/logits_process.py#L270 https://github.com/huggingface/transformers/blob/0a55d9f7376... To summarize how they work: you keep some number of previously generated tokens, and once you get logits that you want to sample a new token from, you find the logits for existing tokens and multiply them by a penalty, thus lowering the probability of the corresponding tokens.
- cubefox 3y ago
  I don't think such penalties were applied to GPT-2 or even GPT-3, yet they weren't repetitive like that.
  - dchest 3y ago
    Yes, they are applied. Here's OpenAI doc which describes how to set various sampling parameters for GPT-3: https://platform.openai.com/docs/api-reference/completions/create https://platform.openai.com/docs/api-reference/completions/c... See presence_penalty and frequency_penalty. Sampling techniques is one of important arts of LLMs, you'll can find a lot of papers on them. In general, smaller are more prone to repetition, but you can get caught in it even with larger models.
    - dchest 3y ago
      To clarify: I meant that in general they are commonly applied, but in this case they weren't as the author confirmed. The repetition, of course, doesn't happen all the time.
  - jackcook 3y ago
    Yes, you're right, I should have mentioned it in the post, but I used pure greedy sampling for the GPT-2 outputs since I couldn't do anything but that for the Apple model. So temperature was set to zero, and there was no repetition penalty.

toxik 3y ago

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

dchest 3y ago
Try running llama.cpp with 0 temperature and without repetition penalty and you'll sometimes get caught in a loop. Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2 https://transformer.huggingface.co/doc/distil-gpt2 https://imgur.com/a/hF1Ewk4 https://imgur.com/a/hF1Ewk4
avianlyric 3y ago
That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops. That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to select any output token that already in the input, to more complex stochastic process that explore the tree of possible outputs, to find branches whose overall result is improved by choosing less optimal immediate steps. The vast majority of what make Chat GPT different to simpler GPT-3 models is this complex post-processing phase that allows designers to push and pull on the behaviour of the pre-baked static model underlying the chat interface.
- cztomsik 3y ago
  Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for. My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough. Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.