3 ms·
> The 8k context window is new Hasn’t Claude had this for many months (before they bumped to 100k)? Edit: ah, you mean new for OSS maybe?
by theptip 3y ago
> The 8k context window is new
Hasn’t Claude had this for many months (before they bumped to 100k)?
Edit: ah, you mean new for OSS maybe?
- simonw 3y agoClaude is a large language model, which is a different thing from an embedding model.
- theptip 3y agoAha, that’s what I missed, thanks!
- Der_Einzige 3y agoAny large language model generates embedding representations at every layer of the model, and these can be trivially extracted. So, large language models are indeed embedding models. This leaderboard doesn't compare these custom tailored embedding models vs the obvious thing of average pooling layered with any traditional LLM, which is easily implemented using sentence transformers.
- anothernewdude 3y agoBecause 4K+ dimensional embeddings are functionally useless.