5 ms·

Better Search Doesn't Mean Beating Google

22 points by limeade 18y ago

mixmax 18y ago
The article argues that one thing is having a successful technology, and another is having a successful business. This is, of course, true - but there's a significant correlation between the two. Correlational databases, Google's search algorithm and the light bulb were all influential technologies that founded hugely successful companies. There's a rule of thumb saying that your solution to a problem has to be 2 - 300% better than the existing state of the art for it to be adapted successfully without artificial help (marketing $, monopolies, etc.) and Google certainly lived up to that when it went live. It remains to be seen whether Wolfram Alpha will.
- vaksel 18y ago
  It won't. It'll probably be another case of Powerset or Cuil, lots of hype by the company in question that is impossible to live up to.
augustiner 18y ago
The most fundamental problem with natural language search engines is that the "natural language" part is more a limitation than a feature to me. Natural language is meant for people to communicate with other people and not with computers. I believe that a well designed keyword/tag based search combined with factual auto suggestions extracted from formal/semantic sources (similar to wikipedia) could be far more efficient for people to use and computers to run.
- gord 18y ago
  I think I just said what you just said.. but then I read your post. Is this something you'd enjoy hacking on?
- micks56 18y ago
  I am in law school and as a law student I spend lots of time searching through past cases. My searching is done almost exclusively through Westlaw, the online database of Thompson West products. They have many different types of searches, but the two applicable to this discussion are (as they are written on the site) both "terms & connectors" and "natural language." T&C works well using your standard OR/AND/etc. However, natural language works so much better even though you type in the exact same words. The natural language search returns cases more on point and has one awesome feature: the most relevant text is in red type, set apart from the rest of the case. From the natural language search West is better able to determine what the legal researcher wants and shows it to him. I spent almost 2 years of law school searching using terms and connectors because I thought the same thing as you do. But I recently converted when I realized West returns better results from their natural language search.
  - ajju 18y ago
    Hi micks56, I would like to ask someone who has a grounding in both law and technology some questions not directly related to this discussion but to software for lawyers in general. Your profile doesn't have an email ID. Care to email me at heuristix at gmail or reply back with your email id?
    - micks56 18y ago
      I just sent you an email. I am happy to answer any questions that you or anyone else has.
      - ajju 18y ago
        Thank you! Got the mail. I'll write up my question(s) today evening and email you.
  - gord 18y ago
    @micks56 - re your legal search using natural language - Are you able to compare, say, Wests results to those of a pure google text search on the keyword terms? [ To do that you'd need some example of large legal texts fully online and thus indexed by google - I dont know if that exists ] Its sometimes hard to discern the value of the tech versus the quality of the implementation + usability factors - but your observations are interesting. I wonder how search on medical information compares... gord.
gord 18y ago
No. Better search does mean beating google. But you don't have to play within Googles rules.. theres lots of territory between text search [=cool] and Natural Language [=sucks, to a first order approximation]. For example, just treat data as a graph of tagged pieces of text.. and give a good web interface to that. Bypass all the RDF, semantic web hype and just make something workable, usable. A wiki for data. anyone looking for a co-founder? Im working on this.
kailashbadu 18y ago
You can’t beat Google only by developing a superior technology. You’ll still be left with the Herculean task of drilling your search engine deep into the minds of hundreds of millions of users around the world. Google the brand is far mightier than Google the technology. It’s the Google’s redoubtable omnipresence and visibility that makes it tick.
- sketerpot 18y ago
  Microsoft tried, and is still trying, with their Live Search. It's the default search in Internet Explorer, and the Microsoft brand is very well known. But you know what the problem is? The search results just aren't as good! I tried out MS Live Search, thinking that a company with as much money to throw at things as Microsoft would probably be of similar quality to Google. I quickly got frustrated when most of the search results were nothing like what I was looking for. Meanwhile Google consistently gave exactly what I wanted near the top of their results page. What I'm saying is that technology does matter. There are other brand-name titans in the world.

22 points by limeade 18y ago

The article argues that one thing is having a successful technology, and another is having a successful business. This is, of course, true - but there's a significant correlation between the two. Correlational databases, Google's search algorithm and the light bulb were all influential technologies that founded hugely successful companies. There's a rule of thumb saying that your solution to a problem has to be 2 - 300% better than the existing state of the art for it to be adapted successfully without artificial help (marketing $, monopolies, etc.) and Google certainly lived up to that when it went live. It remains to be seen whether Wolfram Alpha will.

vaksel 18y ago
It won't. It'll probably be another case of Powerset or Cuil, lots of hype by the company in question that is impossible to live up to.

augustiner 18y ago

The most fundamental problem with natural language search engines is that the "natural language" part is more a limitation than a feature to me. Natural language is meant for people to communicate with other people and not with computers. I believe that a well designed keyword/tag based search combined with factual auto suggestions extracted from formal/semantic sources (similar to wikipedia) could be far more efficient for people to use and computers to run.

gord 18y ago
I think I just said what you just said.. but then I read your post. Is this something you'd enjoy hacking on?
micks56 18y ago
I am in law school and as a law student I spend lots of time searching through past cases. My searching is done almost exclusively through Westlaw, the online database of Thompson West products. They have many different types of searches, but the two applicable to this discussion are (as they are written on the site) both "terms & connectors" and "natural language." T&C works well using your standard OR/AND/etc. However, natural language works so much better even though you type in the exact same words. The natural language search returns cases more on point and has one awesome feature: the most relevant text is in red type, set apart from the rest of the case. From the natural language search West is better able to determine what the legal researcher wants and shows it to him. I spent almost 2 years of law school searching using terms and connectors because I thought the same thing as you do. But I recently converted when I realized West returns better results from their natural language search.
- ajju 18y ago
  Hi micks56, I would like to ask someone who has a grounding in both law and technology some questions not directly related to this discussion but to software for lawyers in general. Your profile doesn't have an email ID. Care to email me at heuristix at gmail or reply back with your email id?
  - micks56 18y ago
    I just sent you an email. I am happy to answer any questions that you or anyone else has.
    - ajju 18y ago
      Thank you! Got the mail. I'll write up my question(s) today evening and email you.
- gord 18y ago
  @micks56 - re your legal search using natural language - Are you able to compare, say, Wests results to those of a pure google text search on the keyword terms? [ To do that you'd need some example of large legal texts fully online and thus indexed by google - I dont know if that exists ] Its sometimes hard to discern the value of the tech versus the quality of the implementation + usability factors - but your observations are interesting. I wonder how search on medical information compares... gord.

gord 18y ago

No. Better search does mean beating google. But you don't have to play within Googles rules.. theres lots of territory between text search [=cool] and Natural Language [=sucks, to a first order approximation]. For example, just treat data as a graph of tagged pieces of text.. and give a good web interface to that. Bypass all the RDF, semantic web hype and just make something workable, usable. A wiki for data. anyone looking for a co-founder? Im working on this.

kailashbadu 18y ago

You can’t beat Google only by developing a superior technology. You’ll still be left with the Herculean task of drilling your search engine deep into the minds of hundreds of millions of users around the world. Google the brand is far mightier than Google the technology. It’s the Google’s redoubtable omnipresence and visibility that makes it tick.

sketerpot 18y ago
Microsoft tried, and is still trying, with their Live Search. It's the default search in Internet Explorer, and the Microsoft brand is very well known. But you know what the problem is? The search results just aren't as good! I tried out MS Live Search, thinking that a company with as much money to throw at things as Microsoft would probably be of similar quality to Google. I quickly got frustrated when most of the search results were nothing like what I was looking for. Meanwhile Google consistently gave exactly what I wanted near the top of their results page. What I'm saying is that technology does matter. There are other brand-name titans in the world.