4 ms·

I think these have common solutions that don't require building more systems, running more queries and jacking up GPU bills, which is the direction we should be

I think these have common solutions that don't require building more systems, running more queries and jacking up GPU bills, which is the direction we should be moving in at this point. e.g. asymmetric embeddings, instruct-based embeddings, and retrieval-rerank all address parts of the problems the author is presenting, all while keeping things generally light on infra.

darkteflon 3y ago
Could you elaborate on “asymmetric embeddings”? That’s the first time I’ve heard that term used in this context.
- rolisz 3y ago
  It's embeddings that are generated differently for queries and for documents. The idea of that queries are usually short, while documents are longer, so if you embed them in the same way, the most relevant docs will be far from the query. Instruct from HKU is an example of such asymmetric embeddings
  - fudged71 3y ago
    I've heard the concept of applying a linear transformation to an embedding model output, is that a similar idea?
    - rolisz 3y ago
      That's a sort of adapting an embedding to a certain task/domain. But I wouldn't call it asymmetric embeddings.
- computerex 3y ago
  It's actually not asymmetric embeddings, but rather asymmetric searches. This is referring to the case where the query and documents being search against are not interchangeable. If the query and documents are interchangeable, then that's symmetric search. It has implications for retrieval. There are embeddings models that are optimized for the symmetric search use case, and then there are models optimized for asymmetric search. You have to use the appropriate model for the task. Furthermore, you can use a LLM to transform your query into the same class as the docs being retrieved, to turn asymmetric search into symmetric search.
jxnlco 3y ago
> asymmetric embeddings, instruct-based embeddings, and retrieval-rerank how would you handle a relative time range? what if you're provided a search client that does not support embeddings (maybe say, google calendar api)

darkteflon 3y ago

Could you elaborate on “asymmetric embeddings”? That’s the first time I’ve heard that term used in this context.

rolisz 3y ago
It's embeddings that are generated differently for queries and for documents. The idea of that queries are usually short, while documents are longer, so if you embed them in the same way, the most relevant docs will be far from the query. Instruct from HKU is an example of such asymmetric embeddings
- fudged71 3y ago
  I've heard the concept of applying a linear transformation to an embedding model output, is that a similar idea?
  - rolisz 3y ago
    That's a sort of adapting an embedding to a certain task/domain. But I wouldn't call it asymmetric embeddings.
computerex 3y ago
It's actually not asymmetric embeddings, but rather asymmetric searches. This is referring to the case where the query and documents being search against are not interchangeable. If the query and documents are interchangeable, then that's symmetric search. It has implications for retrieval. There are embeddings models that are optimized for the symmetric search use case, and then there are models optimized for asymmetric search. You have to use the appropriate model for the task. Furthermore, you can use a LLM to transform your query into the same class as the docs being retrieved, to turn asymmetric search into symmetric search.

jxnlco 3y ago

> asymmetric embeddings, instruct-based embeddings, and retrieval-rerank how would you handle a relative time range? what if you're provided a search client that does not support embeddings (maybe say, google calendar api)