4 ms·

Yeah, this sounds all well and good in theory, but after visiting thousands of sites over the years, it might be a better idea to help engineers build a search

Yeah, this sounds all well and good in theory, but after visiting thousands of sites over the years, it might be a better idea to help engineers build a search engine for their own site/data first. I can't recall many websites that have amazing search. It's a problem when I have to use google to find what I want on xyz.com because if I go search for what I am looking for on xyz.com I cannot find it even if I know its on that site. It would be so nice to go to xyz.com and actually find what I am looking for in under 1 second.

arthurcolle 12y ago
I'm pretty sure Elastic is on the right direction in this regard, don't you think?
- andrewhillman 12y ago
  I don't think so. I just went to their site and the first case study i opened was from theguardian.com. I went to theguardian.com and went to do a search. Guess who they are using to power their search function? Google. In my opinion which means nothing, sites need to figure out how to power their own search. Using a third party isn't going to work for most. Maybe people need to focus on building custom architecture that indexes the data in a more structured way rather than cobbling systems together that ultimately hinder search efforts when its time to get the user what they want. I don't know the answer but somebody eventually will. Maybe wordpress will create a powerful search for all those wordpress sites.
  - arthurcolle 12y ago
    Maybe I'm beating a dead horse but I feel like you could create a pretty compelling search engine using ES. I used it this summer at Goldman and it seems like it will really change the landscape... It's just so fast. I mean full-text-indexing seems to be a pretty integral part of search. Maybe it's just the first step, with the second step being a well written ranking function, but that's just my thought.
- nemothekid 12y ago
  I think "full-text indexing" is a very different problem than "search." Full-text indexing (what ES provides) has been around for almost forever, ES just does a way better job of productizing/delivering it. However, Google is far more than a text index. Ranking, currently is still very difficult and requires messing around with facet and weighting parameters.
- Igglyboo 12y ago
  Elastic is more like a quick way to search through huge amounts of text, not a way to rank which hits are more relevant.
  - arthurcolle 12y ago
    It lets you define your own ranking /scoring function I think
wslh 12y ago
I precisely ranted about this issue as a way to challenge Google: http://blog.databigbang.com/letters-from-the-future-challenging-googles-search-engine/ http://blog.databigbang.com/letters-from-the-future-challeng... I think we need to optimize local search engines and aggregate (in some ways) all this into a global search engine.
dugmartin 12y ago
As a site owner you can pay Google for this: https://www.google.com/work/search/products/gss.html https://www.google.com/work/search/products/gss.html I used it several years ago to power this site search: http://www.poetryfoundation.org/search/articles#qs=ginsberg http://www.poetryfoundation.org/search/articles#qs=ginsberg It works pretty well (disclaimer: I'm not sure this is still being used on that site)
- andrewhillman 12y ago
  Yes. It's been around for years. I have used it but this misses my point or proves it. Either way, sites need to figure out how to power own search.
  - dugmartin 12y ago
    This is customized for the site to 'power their own search'. The query sent to Google is based on the keyword but also on the article type (in this example) and then the xml that is returned is post-processed to have a custom render per article type. Since we had all the data the results from Google were compared to a local search I setup in Sphinx (http://sphinxsearch.com/ http://sphinxsearch.com/) and the Google results were more relevant and it was a lot cheaper to deploy.

arthurcolle 12y ago

I'm pretty sure Elastic is on the right direction in this regard, don't you think?

andrewhillman 12y ago
I don't think so. I just went to their site and the first case study i opened was from theguardian.com. I went to theguardian.com and went to do a search. Guess who they are using to power their search function? Google. In my opinion which means nothing, sites need to figure out how to power their own search. Using a third party isn't going to work for most. Maybe people need to focus on building custom architecture that indexes the data in a more structured way rather than cobbling systems together that ultimately hinder search efforts when its time to get the user what they want. I don't know the answer but somebody eventually will. Maybe wordpress will create a powerful search for all those wordpress sites.
- arthurcolle 12y ago
  Maybe I'm beating a dead horse but I feel like you could create a pretty compelling search engine using ES. I used it this summer at Goldman and it seems like it will really change the landscape... It's just so fast. I mean full-text-indexing seems to be a pretty integral part of search. Maybe it's just the first step, with the second step being a well written ranking function, but that's just my thought.
nemothekid 12y ago
I think "full-text indexing" is a very different problem than "search." Full-text indexing (what ES provides) has been around for almost forever, ES just does a way better job of productizing/delivering it. However, Google is far more than a text index. Ranking, currently is still very difficult and requires messing around with facet and weighting parameters.
Igglyboo 12y ago
Elastic is more like a quick way to search through huge amounts of text, not a way to rank which hits are more relevant.
- arthurcolle 12y ago
  It lets you define your own ranking /scoring function I think

wslh 12y ago

I precisely ranted about this issue as a way to challenge Google: http://blog.databigbang.com/letters-from-the-future-challenging-googles-search-engine/ http://blog.databigbang.com/letters-from-the-future-challeng... I think we need to optimize local search engines and aggregate (in some ways) all this into a global search engine.

dugmartin 12y ago

As a site owner you can pay Google for this: https://www.google.com/work/search/products/gss.html https://www.google.com/work/search/products/gss.html I used it several years ago to power this site search: http://www.poetryfoundation.org/search/articles#qs=ginsberg http://www.poetryfoundation.org/search/articles#qs=ginsberg It works pretty well (disclaimer: I'm not sure this is still being used on that site)

andrewhillman 12y ago
Yes. It's been around for years. I have used it but this misses my point or proves it. Either way, sites need to figure out how to power own search.
- dugmartin 12y ago
  This is customized for the site to 'power their own search'. The query sent to Google is based on the keyword but also on the article type (in this example) and then the xml that is returned is post-processed to have a custom render per article type. Since we had all the data the results from Google were compared to a local search I setup in Sphinx (http://sphinxsearch.com/ http://sphinxsearch.com/) and the Google results were more relevant and it was a lot cheaper to deploy.