3 ms·
I’d love to see some examples of how the token-in-context embeddings stack up against sentence-level. What new use-cases are unlocked? Perhaps semantic search
by clbrmbr 3y ago
I’d love to see some examples of how the token-in-context embeddings stack up against sentence-level. What new use-cases are unlocked?
Perhaps semantic search with word-highlighting?
Any advantage to using the full context window to maximize context around the embedded token?
- eigenvalue 3y agoGreat idea, I’ll see if I can make a new endpoint that adds word level highlighting annotations that could be parsed and used to control the brightness or color of each word based on semantic relevance to a query term. To be honest, I hadn’t even thought about token-level embeddings until someone on Reddit asked about it and I realized it was possible to do with llama-cpp, so I just quickly added the functionality without closely examining the best use cases. It’s a LOT more data and compute than using the normal sentence-level embeddings, so it would really have to unlock some useful new functionality to be worth it. But I do think the “combined feature vector” concept that at least makes them fixed length is helpful.