4 ms·
For posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows
by minimaxir 3y ago
For posterity, an easy way to find the context length of a LLM hosted on Hugging Face is to look at the max_position_embeddings in the config.json, which shows the 8192 mentioned in another comment. (although in this case you need to sign the agreement first)
- brucethemoose2 3y agoThere are some exceptions, like Mistral 0.1 (which is technically 32K according to the config but practically 8K because the sliding window is awful) and InternLM (which (at least initially) used auto rope scaling to extend the context as part of the model's architecture).
- minimaxir 3y agoYes, RoPE has thrown a wrench into things a bit.