3 ms·
Sure, the difference is that the first endpoint would give you back only one single embedding vector for the entire paragraph, while the second endpoint would g
by eigenvalue 3y ago
Sure, the difference is that the first endpoint would give you back only one single embedding vector for the entire paragraph, while the second endpoint would give you a separate embedding vector for each sentence in the paragraph.
And yes, everything in this code is designed to run on the CPU well on a modest machine and is 100% self-hosted, no API keys needed at all. But if you do have a GPU installed and configured it will automatically use that since it’s powered by llama-cpp which now supports CUDA.
- lgvld 3y agoThank you very much! And when is it useful to compute an embedding for each sentence of a document/paragraph then?
- eigenvalue 3y agoHonestly, I'm not exactly sure about that myself. Someone asked for that feature on reddit and I realized that it wouldn't be too hard to add it, so I did. Another poster here mentioned that it might be useful for doing word-level highlighting of what was most relevant to a semantic query string within a sentence. I do think it is able to capture a lot more nuance simply because it's giving you so much more data. The question is how best to leverage that incremental data and take advantage of it, and I'm still trying to figure that out.