3 ms·

Will this be better than LoRA? 32k seems like a lot.

• deleted 4y ago
  [deleted]
• rolisz 4y ago
  This is completely separate from LoRA. This is how much stuff you can give it in the prompt. You can give it now whole chapters of books to summarize for example. LoRA is for adapting the model to a certain model. It usually means you need to give it shorter prompts, but for book summarization, it wouldn't help.
• sp332 4y ago
  LoRA doesn’t change the context length. Also you can’t run LoRAs on GPT-4. So those are not relevant to each other.
  • jurdjir 4y ago
    The distinction becomes less meaningful if you can hold large numbers of tokens in attention, doesn’t it?
    • sp332 4y ago
      I don't... think so. One of us is very confused.
• heliophobicdude 4y ago
  LoRA probably does not affect the models biggest bottle neck. The attention mechanism. Original transformer was O(n^2d) where n is the query length and d was the cardinal of all the tokens