5 ms·

> if you're searching for [camping sites near silicon valley] and [rust http libraries] and then try to compute a similarity metric that includes both of those

by gregmac 10y ago

> if you're searching for [camping sites near silicon valley] and [rust http libraries] and then try to compute a similarity metric that includes both of those lines of querying, you'll get wildly different results than if you separate them out and then compute similar sites. Wow, never thought about how complex of a problem that can really be. To hijack your example: searching for [rust http library] and [clean rust from metal bookshelf] should be completely separate lines of querying but without understanding the full context, appear to have many false similarities (rust, library/bookshelf, 'clean' is a word associated with compiling software, and 'metal' can cause confusing with projects/articles on [bare metal programming with rust])

nostrademons 10y ago
Yeah, disambiguation is a huge problem in search as well. I remember that when I joined (2009), I kept saying we need better disambiguation algorithms, because I was working on UI stuff and getting nonsensical results when I hooked up other teams' backends to the search result page. A bunch of teams were formed and I think the problem was largely solved by 2012 or so - there're some clever tricks you can pull when you have the amount of data and computing power that Google does.
- Declanomous 10y ago
  Google drives me absolutely nuts because of it's 'intelligence', if you will. With dumber search engines it is trivial to ban words you would only expect to see on pages unrelated to your search phrase, using '-' or something similar. Similarly, if there is a word I know is associated with another topic, but is unlikely to appear in another topic, it's easy enough to force it to appear using '+'. Google's methods for detecting clustering, or whatever they are using, are so 'clever' I feel like I'm unable to communicate what I want. I feel like I'm arguing with a particularly intelligent and malicious toddler sometimes. Say I was to ask a toddler to bring me a beer, and they brought me a root beer. I'd be like, "no, bring me a beer, -root." A normal child would bring me a normal beer, cause that's the only thing in the fridge that has beer in the title without root. The Google child would bring me some sassafras, because sassafras is a root beer, which is a beer, but it doesn't have root in the name. Annoyed, I turn off safe search, and I say "Bring me a beer, -root -sassafras -sarsaparilla -ginger -cider." The Google child comes back and says "I couldn't find what you were looking for." I get up off the couch, look in the fridge, and lo and behold it's completely full of beer.
  - nostrademons 10y ago
    You're looking for Verbatim Mode: https://www.google.com/search?q=beer&tbs=li:1 https://www.google.com/search?q=beer&tbs=li:1
    - rburhum 10y ago
      this. Thank you!
    - daurnimator 10y ago
      Awesome! Is there a way I can make this the default?
      - nostrademons 10y ago
        You can't do it for all Google, but if you use Chrome, you can set it as the Omnibox default. Go to Menu -> Settings -> Search -> Manage Search Engines, then add a new row at the bottom that says "Google Verbatim", "verbatim", "https://www.google.com/search?tbs=li:1&q=%s" https://www.google.com/search?tbs=li:1&q=%s", hit Done and click "Manage Search Engines" again, then find that row in the list above (alphabetical) and click "Make default". Now everything in the address bar uses verbatim search.
    - mholmes680 10y ago
      wow, you just fixed everything that i thought was wrong with google. thank you.
  - taneq 10y ago
    I've found that part of the solution is to give it more to work with, and in a 'dumber' way. Remember, Google search isn't really a query language any more, it's more of an NLP find-what-I-mean interface. So sometimes instead of giving it keywords and logical operations ('beer +fridge -root -sassafras'), you can get better results by just asking a question ('get one of my beers from the fridge').
- LunaSea 10y ago
  Are there any that you are at liberty of talking about? I find machine learning especially applied to search and NLP a really interesting problem.
  - nostrademons 10y ago
    Probably not - when it comes to search algorithms it starts to get dangerously close to things Google considers very confidential. Machine learning is definitely a big part of it, but you can also get surprisingly far with simple bag-of-words models when applied to enough data.

by gregmac 10y ago

nostrademons 10y ago

Yeah, disambiguation is a huge problem in search as well. I remember that when I joined (2009), I kept saying we need better disambiguation algorithms, because I was working on UI stuff and getting nonsensical results when I hooked up other teams' backends to the search result page. A bunch of teams were formed and I think the problem was largely solved by 2012 or so - there're some clever tricks you can pull when you have the amount of data and computing power that Google does.

Declanomous 10y ago
Google drives me absolutely nuts because of it's 'intelligence', if you will. With dumber search engines it is trivial to ban words you would only expect to see on pages unrelated to your search phrase, using '-' or something similar. Similarly, if there is a word I know is associated with another topic, but is unlikely to appear in another topic, it's easy enough to force it to appear using '+'. Google's methods for detecting clustering, or whatever they are using, are so 'clever' I feel like I'm unable to communicate what I want. I feel like I'm arguing with a particularly intelligent and malicious toddler sometimes. Say I was to ask a toddler to bring me a beer, and they brought me a root beer. I'd be like, "no, bring me a beer, -root." A normal child would bring me a normal beer, cause that's the only thing in the fridge that has beer in the title without root. The Google child would bring me some sassafras, because sassafras is a root beer, which is a beer, but it doesn't have root in the name. Annoyed, I turn off safe search, and I say "Bring me a beer, -root -sassafras -sarsaparilla -ginger -cider." The Google child comes back and says "I couldn't find what you were looking for." I get up off the couch, look in the fridge, and lo and behold it's completely full of beer.
- nostrademons 10y ago
  You're looking for Verbatim Mode: https://www.google.com/search?q=beer&tbs=li:1 https://www.google.com/search?q=beer&tbs=li:1
  - rburhum 10y ago
    this. Thank you!
  - daurnimator 10y ago
    Awesome! Is there a way I can make this the default?
    - nostrademons 10y ago
      You can't do it for all Google, but if you use Chrome, you can set it as the Omnibox default. Go to Menu -> Settings -> Search -> Manage Search Engines, then add a new row at the bottom that says "Google Verbatim", "verbatim", "https://www.google.com/search?tbs=li:1&q=%s" https://www.google.com/search?tbs=li:1&q=%s", hit Done and click "Manage Search Engines" again, then find that row in the list above (alphabetical) and click "Make default". Now everything in the address bar uses verbatim search.
  - mholmes680 10y ago
    wow, you just fixed everything that i thought was wrong with google. thank you.
- taneq 10y ago
  I've found that part of the solution is to give it more to work with, and in a 'dumber' way. Remember, Google search isn't really a query language any more, it's more of an NLP find-what-I-mean interface. So sometimes instead of giving it keywords and logical operations ('beer +fridge -root -sassafras'), you can get better results by just asking a question ('get one of my beers from the fridge').
LunaSea 10y ago
Are there any that you are at liberty of talking about? I find machine learning especially applied to search and NLP a really interesting problem.
- nostrademons 10y ago
  Probably not - when it comes to search algorithms it starts to get dangerously close to things Google considers very confidential. Machine learning is definitely a big part of it, but you can also get surprisingly far with simple bag-of-words models when applied to enough data.