39 ms·

Okay, so how does the average person search through vector embedding? I would like to try this out.

ForceBru 1y ago
I don't think this is _literally_ a search through vector embeddings. LLMs store embeddings of individual tokens (usually parts of words), so a result of an actual search will be top-k embeddings and the corresponding tokens, similar to the output of a Google search. You could extract the initial matrix of embeddings from some open-weights model and find tokens closest to your query. However, it's not clear why do this. OP got coherent text, so that's not search. It's _similar_, though, because attention in LLMs basically looks for most similar tokens. So to answer the question about the term, the LLM had to create a stream of tokens that's semantically closest to the given description. Well, this is somewhat like a search, but it's not exactly the same.
exe34 1y ago
I didn't say the average person should use it, I was thinking the search functionality could be implemented that way and save on burning up the planet to tame Moloc.
- brokencode 1y ago
  If it could be implemented that way and would be helpful where Google fails, then why has nobody done this? I’d love to try this out if you can point to a product that does this. You can criticize LLMs all you want, but the fact is that they provide value to people in ways that alternatives simply don’t. The energy consumption is a concern, but don’t pretend there are viable alternatives when there aren’t.
  - exe34 1y ago
    I don't have insight on what goes on inside Google - they may well be doing this at some level, but their business isn't finding stuff, it's selling ads, so getting the search right is a very low priority. The LLM people are heavily invested in ever bigger models to keep the research money flowing in, it wouldn't make sense to release a service that undercuts that. that leaves independent actors - presumably building and maintaining an up to date database is difficult, so only the big search engines do.
rcdwealth 1y ago
[dead]

I don't think this is _literally_ a search through vector embeddings. LLMs store embeddings of individual tokens (usually parts of words), so a result of an actual search will be top-k embeddings and the corresponding tokens, similar to the output of a Google search. You could extract the initial matrix of embeddings from some open-weights model and find tokens closest to your query. However, it's not clear why do this. OP got coherent text, so that's not search. It's _similar_, though, because attention in LLMs basically looks for most similar tokens. So to answer the question about the term, the LLM had to create a stream of tokens that's semantically closest to the given description. Well, this is somewhat like a search, but it's not exactly the same.

exe34 1y ago

I didn't say the average person should use it, I was thinking the search functionality could be implemented that way and save on burning up the planet to tame Moloc.

brokencode 1y ago
If it could be implemented that way and would be helpful where Google fails, then why has nobody done this? I’d love to try this out if you can point to a product that does this. You can criticize LLMs all you want, but the fact is that they provide value to people in ways that alternatives simply don’t. The energy consumption is a concern, but don’t pretend there are viable alternatives when there aren’t.
- exe34 1y ago
  I don't have insight on what goes on inside Google - they may well be doing this at some level, but their business isn't finding stuff, it's selling ads, so getting the search right is a very low priority. The LLM people are heavily invested in ever bigger models to keep the research money flowing in, it wouldn't make sense to release a service that undercuts that. that leaves independent actors - presumably building and maintaining an up to date database is difficult, so only the big search engines do.

rcdwealth 1y ago

[dead]