3 ms·

Amazon's search is pretty darn impressive

37 points by brandnewlow 18y ago

kqr2 18y ago
Here's google's answer as a comparison: http://www.google.com/search?hl=en&q=%22The+Triumph+of+Roberto+Bola%C3%B1o+Thursday%2C+December+18%2C+2008+11%3A00+AM+By+Sarah+Kerr+2666+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Natasha+Wimmer+The+Romantic+Dogs+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Laura+Healy+Well+beyond+his+sometimes+nomadic+life%2C+Roberto+Bola%C3%B1o+was+an+exemplary+literary+rebel.+To+drag+fiction+toward+the+unknown+he+had+to+go+there+himself%2C+and+then+invent+a+method+with+which+to+represent+it.+Since+the+unknown+place+was+reality%2C+the+results+of+his+work+are+multi-dimensional%2C+in+a+way+that+runs+ahead+of+a+critic%27s+one-at-a-time+powers+of+description.+Highlight+Bola%C3%B1o%27s+conceptual+play+and+you+risk+missing+the+sex+and+viscera+in+his+work.+Stress+his+ambition+and+his+many+references+and+you+conjure+up+threats+of+exclusive+high-modernist+obscurity%2C+or+literature+as+a+sterile+game%2C+when+the+truth+is+it%27s+hard+to+think+of+a+writer+who+is+less+of+a+snob%2C+or--in+the+double+sense+of+exposing+us+to+unsavory+things+and+carrying+seeds+for+the+future--less+sterile.%22&btnG=Google+Search&aq=f&oq= http://www.google.com/search?hl=en&q=%22The+Triumph+of+R...
- brandnewlow 18y ago
  My implicit point, exactly. I'm working on a memetracker system that will pull in related data on the fly to spruce up the memes. Google proper's been all but useless to me. Youtube's search api has actually been a great last-resort way to get related images. Meanwhile Amazon looks like a pretty good tool too.
  - diN0bot 18y ago
    by using quotes you prevent google from not searching on the complete phrase. this is a better comparison: http://www.google.com/search?hl=en&q=The+Triumph+of+Roberto+Bola%C3%B1o+Thursday%2C+December+18%2C+2008+11%3A00+AM+By+Sarah+Kerr+2666+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Natasha+Wimmer+The+Romantic+Dogs+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Laura+Healy+Well+beyond+his+sometimes+nomadic+life%2C+Roberto+Bola%C3%B1o+was+an+exemplary+literary+rebel.+To+drag+fiction+toward+the+unknown+he+had+to+go+there+himself%2C+and+then+invent+a+method+with+which+to+represent+it.+Since+the+unknown+place+was+reality%2C+the+results+of+his+work+are+multi-dimensional%2C+in+a+way+that+runs+ahead+of+a+critic%27s+one-at-a-time+powers+of+description.+Highlight+Bola%C3%B1o%27s+conceptual+play+and+you+risk+missing+the+sex+and+viscera+in+his+work.+Stress+his+ambition+and+his+many+references+and+you+conjure+up+threats+of+exclusive+high-modernist+obscurity%2C+or+literature+as+a+sterile+game%2C+when+the+truth+is+it%27s+hard+to+think+of+a+writer+who+is+less+of+a+snob%2C+or--in+the+double+sense+of+exposing+us+to+unsavory+things+and+carrying+seeds+for+the+future--less+sterile.&btnG=Search http://www.google.com/search?hl=en&q=The+Triumph+of+Robe...
    - tsally 18y ago
      " 'life' (and any subsequent words) was ignored because we limit queries to 32 words. " That's not a very Googley thing to do...
snewe 18y ago
Ok, how do they do that? Parse every word and pick the few that get the most search hits individually? Or Mechanical Turk?
- aston 18y ago
  I'm thinking it might start with their algorithm for finding improbable phrases. They serve as a kind of a word-based fingerprint for a book and probably make searching on over-specific queries like this one a lot easier. http://www.amazon.com/gp/search-inside/sipshelp.html http://www.amazon.com/gp/search-inside/sipshelp.html
- lneves 18y ago
  An easy way do it would be to extract the term vectors from the query and then find related documents computing the cosine similarity measure.
- dangoldin 18y ago
  Well my guess would be that they have a structured data back end of titles, authors, and so forth. They can then look at the query string and extract everything that looks like a proper noun (or doesn't exist in a dictionary) and and compare that against their structured data database. I'm sure it's more advanced than that but that would get you pretty far.
- zeke 18y ago
  Not too long ago Amazon did not return anything if one word was wrong. They had "The Chocolate Bible : The Difinitive Sourcebook" and would not find it in a search with "definitive" spelled correctly. They have a better search now.
thorax 18y ago
Much better than Amazon used to have, definitely. Though I'm not so sure it's really as impressive as I first thought. It seems to heavily weigh the beginning of the search text. Here's the same search you used, but with the beginning part moved to the end: http://ri.ms/vl7 http://ri.ms/vl7 We do a lot of this full-text search optimization at bug.gd-- we focus on parts of the text that are the rarest in the corpus/index and provide a higher weight to matches on those keywords (also factoring in phrase/proximity). I thought Amazon was doing something similar, but the change in moving around the words either means they have a maximum number of keywords they actually pass to their engine, or they're using a bias towards words at the front of the search (which is interesting itself).

37 points by brandnewlow 18y ago

kqr2 18y ago

Here's google's answer as a comparison: http://www.google.com/search?hl=en&q=%22The+Triumph+of+Roberto+Bola%C3%B1o+Thursday%2C+December+18%2C+2008+11%3A00+AM+By+Sarah+Kerr+2666+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Natasha+Wimmer+The+Romantic+Dogs+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Laura+Healy+Well+beyond+his+sometimes+nomadic+life%2C+Roberto+Bola%C3%B1o+was+an+exemplary+literary+rebel.+To+drag+fiction+toward+the+unknown+he+had+to+go+there+himself%2C+and+then+invent+a+method+with+which+to+represent+it.+Since+the+unknown+place+was+reality%2C+the+results+of+his+work+are+multi-dimensional%2C+in+a+way+that+runs+ahead+of+a+critic%27s+one-at-a-time+powers+of+description.+Highlight+Bola%C3%B1o%27s+conceptual+play+and+you+risk+missing+the+sex+and+viscera+in+his+work.+Stress+his+ambition+and+his+many+references+and+you+conjure+up+threats+of+exclusive+high-modernist+obscurity%2C+or+literature+as+a+sterile+game%2C+when+the+truth+is+it%27s+hard+to+think+of+a+writer+who+is+less+of+a+snob%2C+or--in+the+double+sense+of+exposing+us+to+unsavory+things+and+carrying+seeds+for+the+future--less+sterile.%22&btnG=Google+Search&aq=f&oq= http://www.google.com/search?hl=en&q=%22The+Triumph+of+R...

brandnewlow 18y ago
My implicit point, exactly. I'm working on a memetracker system that will pull in related data on the fly to spruce up the memes. Google proper's been all but useless to me. Youtube's search api has actually been a great last-resort way to get related images. Meanwhile Amazon looks like a pretty good tool too.
- diN0bot 18y ago
  by using quotes you prevent google from not searching on the complete phrase. this is a better comparison: http://www.google.com/search?hl=en&q=The+Triumph+of+Roberto+Bola%C3%B1o+Thursday%2C+December+18%2C+2008+11%3A00+AM+By+Sarah+Kerr+2666+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Natasha+Wimmer+The+Romantic+Dogs+by+Roberto+Bola%C3%B1o%2C+translated+from+the+Spanish+by+Laura+Healy+Well+beyond+his+sometimes+nomadic+life%2C+Roberto+Bola%C3%B1o+was+an+exemplary+literary+rebel.+To+drag+fiction+toward+the+unknown+he+had+to+go+there+himself%2C+and+then+invent+a+method+with+which+to+represent+it.+Since+the+unknown+place+was+reality%2C+the+results+of+his+work+are+multi-dimensional%2C+in+a+way+that+runs+ahead+of+a+critic%27s+one-at-a-time+powers+of+description.+Highlight+Bola%C3%B1o%27s+conceptual+play+and+you+risk+missing+the+sex+and+viscera+in+his+work.+Stress+his+ambition+and+his+many+references+and+you+conjure+up+threats+of+exclusive+high-modernist+obscurity%2C+or+literature+as+a+sterile+game%2C+when+the+truth+is+it%27s+hard+to+think+of+a+writer+who+is+less+of+a+snob%2C+or--in+the+double+sense+of+exposing+us+to+unsavory+things+and+carrying+seeds+for+the+future--less+sterile.&btnG=Search http://www.google.com/search?hl=en&q=The+Triumph+of+Robe...
  - tsally 18y ago
    " 'life' (and any subsequent words) was ignored because we limit queries to 32 words. " That's not a very Googley thing to do...

snewe 18y ago

Ok, how do they do that? Parse every word and pick the few that get the most search hits individually? Or Mechanical Turk?

aston 18y ago
I'm thinking it might start with their algorithm for finding improbable phrases. They serve as a kind of a word-based fingerprint for a book and probably make searching on over-specific queries like this one a lot easier. http://www.amazon.com/gp/search-inside/sipshelp.html http://www.amazon.com/gp/search-inside/sipshelp.html
lneves 18y ago
An easy way do it would be to extract the term vectors from the query and then find related documents computing the cosine similarity measure.
dangoldin 18y ago
Well my guess would be that they have a structured data back end of titles, authors, and so forth. They can then look at the query string and extract everything that looks like a proper noun (or doesn't exist in a dictionary) and and compare that against their structured data database. I'm sure it's more advanced than that but that would get you pretty far.
zeke 18y ago
Not too long ago Amazon did not return anything if one word was wrong. They had "The Chocolate Bible : The Difinitive Sourcebook" and would not find it in a search with "definitive" spelled correctly. They have a better search now.

thorax 18y ago

Much better than Amazon used to have, definitely. Though I'm not so sure it's really as impressive as I first thought. It seems to heavily weigh the beginning of the search text. Here's the same search you used, but with the beginning part moved to the end: http://ri.ms/vl7 http://ri.ms/vl7 We do a lot of this full-text search optimization at bug.gd-- we focus on parts of the text that are the rarest in the corpus/index and provide a higher weight to matches on those keywords (also factoring in phrase/proximity). I thought Amazon was doing something similar, but the change in moving around the words either means they have a maximum number of keywords they actually pass to their engine, or they're using a bias towards words at the front of the search (which is interesting itself).