7 ms·

The High Cost of Keyword Search

56 points by svcrunch 5y ago

teddyh 5y ago
> A great deal of research has established the importance of relevance in engaging and satisfying users. Too bad then that this is not the goal of web search engines. Their goal is instead to sell ad space.
skinkestek 5y ago
> Whether your objective is to improve user engagement, or you simply want to help your users quickly find the most relevant information, you pay the high cost of keyword search every day in the form of missing and irrelevant search results. At the moment my biggest frustration is that both Google and DuckDuckGo keeps ignoring my keywords and keep jamming my results with all kinds of obviously irrelevant data. Google having a "verbatim" option that they ignore on top of already ignoring doublequotes after having butchered the plus operator adds insult to injury..! There's probably some UX research that indicates that in complex scenarios it is better to give the user the impression of control while silently doing tje right thing. Well, guess what: that depends on you doing the right thing, and the right thing is not to stuff my results with pages that doesn't contain the error messages I was looking for!
- marcosdumay 5y ago
  At least on DDG, if you use quotes it will give you a "no results" page before it includes anything that doesn't have it. It used to also place the results that match all of your keywords first. They stopped doing this a few years ago, but they didn't mess with the quotes (yet?).
  - skinkestek 5y ago
    > At least on DDG, if you use quotes it will give you a "no results" page before it includes anything that doesn't have it. For some reason I don't get that except for extremely obvious cases. Example: https://duckduckgo.com/?q=%22lego+sauce%22&t=fpas&ia=web https://duckduckgo.com/?q=%22lego+sauce%22&t=fpas&ia=web
WalterBright 5y ago
Amazon search is rather lame. Last week I typed in the exact title of a book. The book way way down in the search results. Before it was what Amazon thought were "better" results. Please, Amazon, verbatim matches should go first. Then do the AI thing. Google, too. This is really important when looking for something obscure.
- FriedrichN 5y ago
  I've found that this new wave of 'smart' search systems are actually making search results progressively worse. People complain that they can't find anything on Google alternatives, my experience is that search results are equally bad. These new systems start making things up "I bet he meant to search THIS" and there's no way to escape it. All you get is ads and generic results any time you try to find anything out of the ordinary. I remember that if you were good at formulating your search query you could almost always find what you want. Instead they assume the user is stupid or just want to serve up an ad.
  - WalterBright 5y ago
    I like the imdb.com search. If I type in the exact movie title, there it is. If I only have a vague idea of what the title is, it'll often suggest it to me.
    - rawoke083600 5y ago
      Ahh but they are also very lucky in that they MOSTLY only search movies,actors and tv-series. 3 data-types.
      - WalterBright 5y ago
        Yeah, well, when I select "Books" in Amazon's search selector, I darn well expect it to give me results that are "Books". Is this too much to ask? :-/
- rawoke083600 5y ago
WalterBright 5y ago
The algorithm reminds me of Soundex. https://en.wikipedia.org/wiki/Soundex https://en.wikipedia.org/wiki/Soundex
jillesvangurp 5y ago
Elasticsearch, and probably Solr too, has some support for vector searches. https://www.elastic.co/blog/text-similarity-search-with-vectors-in-elasticsearch https://www.elastic.co/blog/text-similarity-search-with-vect... I looked into using that last year. In the end we went with a simple more like this query, which proved good enough relative to the model our machine learning people came up with (not so great because real world data is just hard). The nice thing with this is that you can combine vector search clauses with additional clauses and other features such as aggregations, highlighting, etc. Another way to use machine learning with Elasticsearch would be the more like this plugin which re-ranks results matching a particular query. It's a simple linear regression (usually) based model that you train using features in the form of elasticsearch query clauses. Basically it learns the boost factors for each of the features. E.g. Soundcloud uses this in their search. Either way, the hard part of stuff like this is testing your system. Going from a "this looks ballpark right" to "our users click on more results by 0.5% after change X", is a long journey. Not a lot of companies are able to do this and I've seen a few teams fall into the trap of having a not so great machine learned model with a flawed tests suite proving not much at all about its alleged properties. A well crafted query can do a lot of the same things. A benefit is that you can fix things easily whereas most machine learning is a bit more like a black box (either works or does not).
- gk1 5y ago
  Vector search is where the next leap in search and recommender system improvements will come from. * Is already coming from. Google, Bing, Spotify, Pinterest, Facebook, etc, already use it in production. All flavors of ES have some kind of vector search available, albeit with some limitations. There are also stand-alone open-source and managed solutions popping up left and right (I work for one, Pinecone.io). From what I’m seeing, the hard parts people encounter are: - Finding or developing good embedding models. I consider KPI testing a part of this process, since it tells you whether or not your models are actually “good.” - Finding and tuning vector search libraries that can meet speed and accuracy requirements. (Easy if you use ES although you’ll run into latency issues with >10M docs. Even easier with a managed vector search solution.) - Scaling the above to handle millions (or billions!) of items and high query throughput. Again, less of an issue if you’re using managed ES on AWS or Elastic.co, or a managed vector search solution like Pinecone, or you have a team of distributed systems engineers sitting idle.
JackPoach 5y ago
This is only to get worse if more people switch to voice-based interfaces (Alexa, anyone?). The way people search when they type vs the way it's done with voice commands can be quite different.
- dstick 5y ago
  Good point. If the actual result you’re after is the 5th, you’d have to _listen_ to 4 bad ones which costs a lot of time and will quickly lead to frustration - a bad UX. The beauty of text is that you can quickly scan. The beauty of voice is that you can provide input faster. You’d need a voice / visual (glasses?) combo to overcome that and combine the best of both worlds.
  - JackPoach 5y ago
    Yep.
- mschuster91 5y ago
  On the other side, voice based interface makes it easier to supply context - for example, am I searching for a book called "Apple", the fruit "apple" or the company "Apple"? A (future) AGI-based search engine can go even further than that, and generate the answer for any problem based on the knowledge it has ingested.
Fragoel2 5y ago
Suppose I would want to run something like this on corpus of documents I have and I'm not really a machine learning expert, any recommendation for a library/boilerplate/starting point?
- gk1 5y ago
  This tutorial uses an off-the-shelf ML model (Average Word Embeddings) and a managed vector index for document search: https://www.pinecone.io/docs/examples/semantic-text-search/ https://www.pinecone.io/docs/examples/semantic-text-search/ Even if you choose another model or method for storing and searching through the vector embeddings, the order of operations will be approximately the same.
- lacksconfidence 5y ago
  I haven't used it yet, but https://github.com/semi-technologies/weaviate https://github.com/semi-technologies/weaviate is a vector search engine that does embeddings and search. I don't know if they have succeeded, but the goal seems to be to make embedding and searching vectors easy enough to not need to be an ML expert.
blackcat201 5y ago
I recently make a vector search engine demo[0] that uses neural networks (CLIP) to encode text and image into vector. It runs surprisingly fast consider its hosted on a 2 core virtual cloud running both the text encoder and doing the vector similarity search over a 500k index results for both image and website links. [0] https://unisearch.cloud https://unisearch.cloud
deleted 5y ago
[deleted]

56 points by svcrunch 5y ago

> A great deal of research has established the importance of relevance in engaging and satisfying users. Too bad then that this is not the goal of web search engines. Their goal is instead to sell ad space.

skinkestek 5y ago

> Whether your objective is to improve user engagement, or you simply want to help your users quickly find the most relevant information, you pay the high cost of keyword search every day in the form of missing and irrelevant search results. At the moment my biggest frustration is that both Google and DuckDuckGo keeps ignoring my keywords and keep jamming my results with all kinds of obviously irrelevant data. Google having a "verbatim" option that they ignore on top of already ignoring doublequotes after having butchered the plus operator adds insult to injury..! There's probably some UX research that indicates that in complex scenarios it is better to give the user the impression of control while silently doing tje right thing. Well, guess what: that depends on you doing the right thing, and the right thing is not to stuff my results with pages that doesn't contain the error messages I was looking for!

marcosdumay 5y ago
At least on DDG, if you use quotes it will give you a "no results" page before it includes anything that doesn't have it. It used to also place the results that match all of your keywords first. They stopped doing this a few years ago, but they didn't mess with the quotes (yet?).
- skinkestek 5y ago
  > At least on DDG, if you use quotes it will give you a "no results" page before it includes anything that doesn't have it. For some reason I don't get that except for extremely obvious cases. Example: https://duckduckgo.com/?q=%22lego+sauce%22&t=fpas&ia=web https://duckduckgo.com/?q=%22lego+sauce%22&t=fpas&ia=web

WalterBright 5y ago

Amazon search is rather lame. Last week I typed in the exact title of a book. The book way way down in the search results. Before it was what Amazon thought were "better" results. Please, Amazon, verbatim matches should go first. Then do the AI thing. Google, too. This is really important when looking for something obscure.

FriedrichN 5y ago
I've found that this new wave of 'smart' search systems are actually making search results progressively worse. People complain that they can't find anything on Google alternatives, my experience is that search results are equally bad. These new systems start making things up "I bet he meant to search THIS" and there's no way to escape it. All you get is ads and generic results any time you try to find anything out of the ordinary. I remember that if you were good at formulating your search query you could almost always find what you want. Instead they assume the user is stupid or just want to serve up an ad.
- WalterBright 5y ago
  I like the imdb.com search. If I type in the exact movie title, there it is. If I only have a vague idea of what the title is, it'll often suggest it to me.
  - rawoke083600 5y ago
    Ahh but they are also very lucky in that they MOSTLY only search movies,actors and tv-series. 3 data-types.
    - WalterBright 5y ago
      Yeah, well, when I select "Books" in Amazon's search selector, I darn well expect it to give me results that are "Books". Is this too much to ask? :-/
rawoke083600 5y ago

WalterBright 5y ago

The algorithm reminds me of Soundex. https://en.wikipedia.org/wiki/Soundex https://en.wikipedia.org/wiki/Soundex

jillesvangurp 5y ago

Elasticsearch, and probably Solr too, has some support for vector searches. https://www.elastic.co/blog/text-similarity-search-with-vectors-in-elasticsearch https://www.elastic.co/blog/text-similarity-search-with-vect... I looked into using that last year. In the end we went with a simple more like this query, which proved good enough relative to the model our machine learning people came up with (not so great because real world data is just hard). The nice thing with this is that you can combine vector search clauses with additional clauses and other features such as aggregations, highlighting, etc. Another way to use machine learning with Elasticsearch would be the more like this plugin which re-ranks results matching a particular query. It's a simple linear regression (usually) based model that you train using features in the form of elasticsearch query clauses. Basically it learns the boost factors for each of the features. E.g. Soundcloud uses this in their search. Either way, the hard part of stuff like this is testing your system. Going from a "this looks ballpark right" to "our users click on more results by 0.5% after change X", is a long journey. Not a lot of companies are able to do this and I've seen a few teams fall into the trap of having a not so great machine learned model with a flawed tests suite proving not much at all about its alleged properties. A well crafted query can do a lot of the same things. A benefit is that you can fix things easily whereas most machine learning is a bit more like a black box (either works or does not).

gk1 5y ago
Vector search is where the next leap in search and recommender system improvements will come from. * Is already coming from. Google, Bing, Spotify, Pinterest, Facebook, etc, already use it in production. All flavors of ES have some kind of vector search available, albeit with some limitations. There are also stand-alone open-source and managed solutions popping up left and right (I work for one, Pinecone.io). From what I’m seeing, the hard parts people encounter are: - Finding or developing good embedding models. I consider KPI testing a part of this process, since it tells you whether or not your models are actually “good.” - Finding and tuning vector search libraries that can meet speed and accuracy requirements. (Easy if you use ES although you’ll run into latency issues with >10M docs. Even easier with a managed vector search solution.) - Scaling the above to handle millions (or billions!) of items and high query throughput. Again, less of an issue if you’re using managed ES on AWS or Elastic.co, or a managed vector search solution like Pinecone, or you have a team of distributed systems engineers sitting idle.

JackPoach 5y ago

This is only to get worse if more people switch to voice-based interfaces (Alexa, anyone?). The way people search when they type vs the way it's done with voice commands can be quite different.

dstick 5y ago
Good point. If the actual result you’re after is the 5th, you’d have to _listen_ to 4 bad ones which costs a lot of time and will quickly lead to frustration - a bad UX. The beauty of text is that you can quickly scan. The beauty of voice is that you can provide input faster. You’d need a voice / visual (glasses?) combo to overcome that and combine the best of both worlds.
- JackPoach 5y ago
  Yep.
mschuster91 5y ago
On the other side, voice based interface makes it easier to supply context - for example, am I searching for a book called "Apple", the fruit "apple" or the company "Apple"? A (future) AGI-based search engine can go even further than that, and generate the answer for any problem based on the knowledge it has ingested.

Fragoel2 5y ago

Suppose I would want to run something like this on corpus of documents I have and I'm not really a machine learning expert, any recommendation for a library/boilerplate/starting point?

gk1 5y ago
This tutorial uses an off-the-shelf ML model (Average Word Embeddings) and a managed vector index for document search: https://www.pinecone.io/docs/examples/semantic-text-search/ https://www.pinecone.io/docs/examples/semantic-text-search/ Even if you choose another model or method for storing and searching through the vector embeddings, the order of operations will be approximately the same.
lacksconfidence 5y ago
I haven't used it yet, but https://github.com/semi-technologies/weaviate https://github.com/semi-technologies/weaviate is a vector search engine that does embeddings and search. I don't know if they have succeeded, but the goal seems to be to make embedding and searching vectors easy enough to not need to be an ML expert.

blackcat201 5y ago

I recently make a vector search engine demo[0] that uses neural networks (CLIP) to encode text and image into vector. It runs surprisingly fast consider its hosted on a 2 core virtual cloud running both the text encoder and doing the vector similarity search over a 500k index results for both image and website links. [0] https://unisearch.cloud https://unisearch.cloud

deleted 5y ago

[deleted]