6 ms·

Actually google has become ( for me, at least) lately some kind of proxy to stackoverflow and wikipedia, since both this two services have atrocious own search

by ccozan 10y ago

Actually google has become ( for me, at least) lately some kind of proxy to stackoverflow and wikipedia, since both this two services have atrocious own search solutions and I have to rely on google to get interpret my keywords the right way.

cdubzzz 10y ago
This really does seem like a big problem. What are some sites that do site search well? Is there currently a good way to implement something even half as good as Google locally without a major lift?
- ccozan 10y ago
  I was senior engineer with Autonomy ( before being bought by HP ) and we did really good projects for some big companies, it worked well. Lucene comes a long way, with enough time, some cool stuff would emerge. But if you need to replicate google, you need bootloads of user data and in generally data to operate on ( i.e. to be able to extract the equivalence of certain terms and expressions, and so on ). And generally some kind of feedback system ( like user clicking a link on a SERP, is adding that question/answer combo a certain boost for the future ). In short: it's complicated and after 10 years of experience I couldn't tell if a solution is so unversal as google.
  - cdubzzz 10y ago
    Pardon my ignorance, but would such a service need to be specific to a site? I'm wondering if there is a business (or perhaps better yet, non-profit) opportunity to provide data to drive something that can be tacked on to individual sites. Would an organization like Mozilla have an opportunity here?
    - ccozan 10y ago
      It depends. Organizations have different data, so you need a lot of abstractions in order to fit into a certain search pattern (meaning, what are the users looking for). For example, current solutions implies there is a "document" with some metadata. But what is a document in wikipedia or SO context? So is quite hard to get a an answer which satisfies both cases and you end up with cramming all kind of data together just to make a fit. So in the end, you still have to use the google model, of using the URL as a key for (data, metadata). So in this case, just let google/bing/yandex do it's job. On the other side, I saw Google Search Appliance fail spectacularly in enterprise, because the google model couldn't fit and you end using real custom search solutions. There are quite a lot of companies filling this niche at this moment and so I think Mozilla wouldn't make a dent, if decides to enter this market.
      - cdubzzz 10y ago
        Interesting. Thanks for your insight. My line of thinking was on working with "what is the user asking" vs. "what is the user looking for" in a way that could be applied regardless of the information being searched (by, for example, a non-profit with some existing insight - hence the Mozilla thought). But I know almost nil about the space so I suspect that is somewhat naive.
    - fixermark 10y ago
      You've just described Google search. If your business pitch is "Like Google search, but Google doesn't control it," good luck getting investors.
- fixermark 10y ago
  Why bother when Google will index your site for you? It's one fewer engineering task for a busy content provider to take on.
  - cdubzzz 10y ago
    I can think of many reasons to bother. In the context of the article being discussed, it can be argued that Google is presenting sites' content without clear attribution. Preventing the need for the user to click-through also limits ad revenue (where applicable). Either would be a perfectly good reason to consider alternatives. It's a massive trade off, of course, given Google's position but I don't think decentralized search is a bad goal.

by ccozan 10y ago

cdubzzz 10y ago

This really does seem like a big problem. What are some sites that do site search well? Is there currently a good way to implement something even half as good as Google locally without a major lift?

ccozan 10y ago
I was senior engineer with Autonomy ( before being bought by HP ) and we did really good projects for some big companies, it worked well. Lucene comes a long way, with enough time, some cool stuff would emerge. But if you need to replicate google, you need bootloads of user data and in generally data to operate on ( i.e. to be able to extract the equivalence of certain terms and expressions, and so on ). And generally some kind of feedback system ( like user clicking a link on a SERP, is adding that question/answer combo a certain boost for the future ). In short: it's complicated and after 10 years of experience I couldn't tell if a solution is so unversal as google.
- cdubzzz 10y ago
  Pardon my ignorance, but would such a service need to be specific to a site? I'm wondering if there is a business (or perhaps better yet, non-profit) opportunity to provide data to drive something that can be tacked on to individual sites. Would an organization like Mozilla have an opportunity here?
  - ccozan 10y ago
    It depends. Organizations have different data, so you need a lot of abstractions in order to fit into a certain search pattern (meaning, what are the users looking for). For example, current solutions implies there is a "document" with some metadata. But what is a document in wikipedia or SO context? So is quite hard to get a an answer which satisfies both cases and you end up with cramming all kind of data together just to make a fit. So in the end, you still have to use the google model, of using the URL as a key for (data, metadata). So in this case, just let google/bing/yandex do it's job. On the other side, I saw Google Search Appliance fail spectacularly in enterprise, because the google model couldn't fit and you end using real custom search solutions. There are quite a lot of companies filling this niche at this moment and so I think Mozilla wouldn't make a dent, if decides to enter this market.
    - cdubzzz 10y ago
      Interesting. Thanks for your insight. My line of thinking was on working with "what is the user asking" vs. "what is the user looking for" in a way that could be applied regardless of the information being searched (by, for example, a non-profit with some existing insight - hence the Mozilla thought). But I know almost nil about the space so I suspect that is somewhat naive.
  - fixermark 10y ago
    You've just described Google search. If your business pitch is "Like Google search, but Google doesn't control it," good luck getting investors.
fixermark 10y ago
Why bother when Google will index your site for you? It's one fewer engineering task for a busy content provider to take on.
- cdubzzz 10y ago
  I can think of many reasons to bother. In the context of the article being discussed, it can be argued that Google is presenting sites' content without clear attribution. Preventing the need for the user to click-through also limits ad revenue (where applicable). Either would be a perfectly good reason to consider alternatives. It's a massive trade off, of course, given Google's position but I don't think decentralized search is a bad goal.