3 ms·

Has anyone found the context length for these models yet? So far I haven't seen it mentioned in their write-up or the model card

• kathleenfromgdm 3y ago
  The context length for these models is 8192 tokens.
• minimaxir 3y ago
  For posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows the 8192 mentioned in another comment. (although in this case you need to sign the agreement first)
  • brucethemoose2 3y ago
    There are some exceptions, like Mistral 0.1 (which is technically 32K according to the config but practically 8K because the sliding window is awful) and InternLM (which (at least initially) used auto rope scaling to extend the context as part of the model's architecture).
    • minimaxir 3y ago
      Yes, RoPE has thrown a wrench into things a bit.