5 ms·

Could you elaborate on “asymmetric embeddings”? That’s the first time I’ve heard that term used in this context.

• rolisz 3y ago
  It's embeddings that are generated differently for queries and for documents. The idea of that queries are usually short, while documents are longer, so if you embed them in the same way, the most relevant docs will be far from the query. Instruct from HKU is an example of such asymmetric embeddings
  • fudged71 3y ago
    I've heard the concept of applying a linear transformation to an embedding model output, is that a similar idea?
    • rolisz 3y ago
      That's a sort of adapting an embedding to a certain task/domain. But I wouldn't call it asymmetric embeddings.
• computerex 3y ago
  It's actually not asymmetric embeddings, but rather asymmetric searches. This is referring to the case where the query and documents being search against are not interchangeable. If the query and documents are interchangeable, then that's symmetric search. It has implications for retrieval. There are embeddings models that are optimized for the symmetric search use case, and then there are models optimized for asymmetric search. You have to use the appropriate model for the task. Furthermore, you can use a LLM to transform your query into the same class as the docs being retrieved, to turn asymmetric search into symmetric search.