3 ms·
Take your document, cut it (cleanly!) into pieces small enough to fit into your sentence embedder's context window, and generate several embeddings that all poi
by nestorD 3y ago
Take your document, cut it (cleanly!) into pieces small enough to fit into your sentence embedder's context window, and generate several embeddings that all point to the same document.
I would recommend against merging (averaging, etc.) the embeddings (unless you want a blurry idea of what your document contains), as well as feeding very large pieces of text to the embedder (some models have massive context lengths, but the result is similarly vague).
- gregw134 3y ago> [recommend against] feeding very large pieces of text to the embedder Sounds right, I've heard this from multiple sources. That's why I'm leaning towards just embedding the keywords.
- Havoc 3y agoWhen trying to find similarity between whole docs one would feed the entire doc though, right?