3 ms·

Personally I think the limitation of keyword search it's not in the model per se but in the human langue: we have synonymous witch are relatively easy to handle

by kkfx 2y ago

Personally I think the limitation of keyword search it's not in the model per se but in the human langue: we have synonymous witch are relatively easy to handle but we also have gazillion of different way to express the very same concept that simply can't be squeezed in some "nearby keyword list". Personally I notes news, importing articles in org-mode, so I have a "trail" of the news I think are relevant in a timeline, sometimes I remember I've noted something but I can't find it immediately in my own notes with local full-text search on a very little base compared to the entire web, simply because a title does express something with very different words than another and at the moment of a search I do not think about such possible expression. For casual searches we do not notice, but for some specific searches emerge very clear as a big limitation, however so far LLMs does not solve it, they are even LESS able to extract relevant information, and "semantic" classifications does not seems to be effective either, a thing even easier to spot if you use Zotero and tags and really try to use tags to look for something, in the end you'll resort on mere keyword search for anything. That's why IMVHO it's an unsolved so far problem.

DrScientist 2y ago
For me the search problem isn't so much about making sure I get back all potentially relevant hits ( more than I could ever read ) , it's how I get the specific ones I want... So effective search more about excluding than including. Exact phrases or particular keywords are great tools here. Note there is also a difference between finding an answer to a particular question and finding web pages around a particular topic. Perhaps LLM's are more useful for the former - where there is a need to both map the question to an embedding, and summarize the answer - but for the latter I'm not interested in a summary/quick answer, I'm interested in the source material. Sometimes you can combine the two - LLM's for a quick route into the common jargon, which can then be used as keywords.
- kkfx 2y ago
  > there is also a difference between finding an answer to a particular question and finding web pages around a particular topic. That's exactly the Gessner's problem and unimplementable so far solution: we have information packed in various form, when we want specific bits it's hard do narrow enough available information packages (books, articles, post etc). For common stuff in the vast see of the web we tend to end up very frequently on exactly or nearly exactly what we want, but for less common stuff there is a big problem. Just knowing the historical temperature of a small place where many have recorded temperatures, but still no one have create a table and a graph or daily minima and maxima for let's say the 10 years period we want it's hard to find an answer. For some kind of data we have wikidata, for some others we have public datasets from various sources, but still no easy way to locate and narrow them. LLM are in theory an answer, unfortunately not in practice first for freshness problem (you have to train a model, ingesting new information is still training, far longer than a search engine crawler) and training itself witch is like the opposite of semantic web: instead of having the author prepare the information for machines we have third party who do it en masse, unfortunately they tend to be even less precise than semantic web mean authors PLUS models hallucinations and essentially no ability to tell where their answer came from. That's why we still miss semantic search...

by kkfx 2y ago

DrScientist 2y ago

For me the search problem isn't so much about making sure I get back all potentially relevant hits ( more than I could ever read ) , it's how I get the specific ones I want... So effective search more about excluding than including. Exact phrases or particular keywords are great tools here. Note there is also a difference between finding an answer to a particular question and finding web pages around a particular topic. Perhaps LLM's are more useful for the former - where there is a need to both map the question to an embedding, and summarize the answer - but for the latter I'm not interested in a summary/quick answer, I'm interested in the source material. Sometimes you can combine the two - LLM's for a quick route into the common jargon, which can then be used as keywords.

kkfx 2y ago
> there is also a difference between finding an answer to a particular question and finding web pages around a particular topic. That's exactly the Gessner's problem and unimplementable so far solution: we have information packed in various form, when we want specific bits it's hard do narrow enough available information packages (books, articles, post etc). For common stuff in the vast see of the web we tend to end up very frequently on exactly or nearly exactly what we want, but for less common stuff there is a big problem. Just knowing the historical temperature of a small place where many have recorded temperatures, but still no one have create a table and a graph or daily minima and maxima for let's say the 10 years period we want it's hard to find an answer. For some kind of data we have wikidata, for some others we have public datasets from various sources, but still no easy way to locate and narrow them. LLM are in theory an answer, unfortunately not in practice first for freshness problem (you have to train a model, ingesting new information is still training, far longer than a search engine crawler) and training itself witch is like the opposite of semantic web: instead of having the author prepare the information for machines we have third party who do it en masse, unfortunately they tend to be even less precise than semantic web mean authors PLUS models hallucinations and essentially no ability to tell where their answer came from. That's why we still miss semantic search...