5 ms·

I once spoke to someone in the know about this, and the main reason is that it's quite expensive for them to do any reasonable job with stemming, etc. I know t

I once spoke to someone in the know about this, and the main reason is that it's quite expensive for them to do any reasonable job with stemming, etc. I know that sounds weird (but it's google, they're omnipotent!), but it makes sense: It's worth their while to stem content they crawl and index off the web, cause everybody could in theory access any given page. However, with email, the only person who'll ever benefit is the recipient.

anurag 17y ago
> However, with email, the only person who'll ever benefit is the recipient. I'm not sure that makes sense - if they add stemming, all users of GMail benefit. Going by your explanation, it wouldn't make sense to add any expensive features to GMail, because the only person who would ever benefit from them is the single user.
- antipaganda 17y ago
  I think you misunderstand the nature of stemming; the point is that each and every user's inbox would have to be processed individually, and apparently Google doesn't think the overhead is worth the results.
  - anurag 17y ago
    I understand stemming. I just think the webpage contrast is not a good explanation for why they're not doing it. Building an unstemmed search index per user is also expensive, and helps only the recipient, but they do it because they think the expense is necessary. They stem webpages because they think the expense is necessary. They don't stem mails because they think the expense is not justified, not because the only person who benefits is the recipient.
    - sachinag 17y ago
      It is also expensive, but it is less expensive than doing that AND stemming. They just decided that stemming was a line where the benefit (add'l users, more use of Gmail, more AdSense revenue, whatever metric) wasn't worth the (development and ongoing processing) costs.
      - jhancock 17y ago
        This is a drawback to putting everything in the cloud: features will be weighed by the CPU cycles and storage required by the providers. Can't wait til we come full circle and get back to client/server computing ;). I'm only half joking. I actually can't wait until things mature enough so we have a hybrid of both models. Then I can decide just how much stuff I want indexed and also not worry what happens when my cable modem flakes out.
        gloob 17y ago
        Can't wait til we come full circle and get back to client/server computing ;). The next big hype cycle: OS as an OS.
bd 17y ago
They could use Gears for some more in-depth local indexing, then it would be you bearing extra computational and storage costs.
- bradgessler 17y ago
  That's a really interesting notion. Have any web applications moved indexing to the client-side before with gears and/or js?
  - bd 17y ago
    MySpace (mentioning cost savings as one of the main reasons): http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-down-server-costs/ http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-dow... http://developer.myspace.com/community/blogs/devteam/archive/2008/05/29/myspace-mail-search-with-gears-released-google-i-o-presentation-slides.aspx http://developer.myspace.com/community/blogs/devteam/archive...
- IsaacSchlueter 17y ago
  You could probably build a gadget that does this, yes?
zngtk4 17y ago
It would at least make sense to deal with plurals. I can't tell you the number of times I've searched for something using an s (or not) at the end, and failed to find what I was looking for, only to remember later that I should try to add or remove an s from my search term.
Freaky 17y ago
If it's expensive, then why not reflect that by providing it to people who pay for their accounts?
flatline 17y ago
It seems like they've worked on mitigating this on the web with suggestions for alternate spellings and by displaying related searches. While you can see stemming at work in many Google searches, I'm pretty sure they don't build extensive substring indices on the web end either. For example, I've had searches where a substring returns 0 results and the exact phrase returns a handful.
DannoHung 17y ago
Perhaps it would be an option that they could sell to people?
- mistermann 17y ago
  Exactly....I have always wondered why there is no google premium...I would gladly pay $100/year to have a collection of certain domains easily removed from my google searches, or pick from a list of my favorite sites for a "site:" search, etc etc.
  - sp332 17y ago
    Well, you can do at least that last bit fairly easily. http://www.google.com/coop/cse/ http://www.google.com/coop/cse/

anurag 17y ago

> However, with email, the only person who'll ever benefit is the recipient. I'm not sure that makes sense - if they add stemming, all users of GMail benefit. Going by your explanation, it wouldn't make sense to add any expensive features to GMail, because the only person who would ever benefit from them is the single user.

antipaganda 17y ago
I think you misunderstand the nature of stemming; the point is that each and every user's inbox would have to be processed individually, and apparently Google doesn't think the overhead is worth the results.
- anurag 17y ago
  I understand stemming. I just think the webpage contrast is not a good explanation for why they're not doing it. Building an unstemmed search index per user is also expensive, and helps only the recipient, but they do it because they think the expense is necessary. They stem webpages because they think the expense is necessary. They don't stem mails because they think the expense is not justified, not because the only person who benefits is the recipient.
  - sachinag 17y ago
    It is also expensive, but it is less expensive than doing that AND stemming. They just decided that stemming was a line where the benefit (add'l users, more use of Gmail, more AdSense revenue, whatever metric) wasn't worth the (development and ongoing processing) costs.
    - jhancock 17y ago
      This is a drawback to putting everything in the cloud: features will be weighed by the CPU cycles and storage required by the providers. Can't wait til we come full circle and get back to client/server computing ;). I'm only half joking. I actually can't wait until things mature enough so we have a hybrid of both models. Then I can decide just how much stuff I want indexed and also not worry what happens when my cable modem flakes out.
      - gloob 17y ago
        Can't wait til we come full circle and get back to client/server computing ;). The next big hype cycle: OS as an OS.

bd 17y ago

They could use Gears for some more in-depth local indexing, then it would be you bearing extra computational and storage costs.

bradgessler 17y ago
That's a really interesting notion. Have any web applications moved indexing to the client-side before with gears and/or js?
- bd 17y ago
  MySpace (mentioning cost savings as one of the main reasons): http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-down-server-costs/ http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-dow... http://developer.myspace.com/community/blogs/devteam/archive/2008/05/29/myspace-mail-search-with-gears-released-google-i-o-presentation-slides.aspx http://developer.myspace.com/community/blogs/devteam/archive...
IsaacSchlueter 17y ago
You could probably build a gadget that does this, yes?

zngtk4 17y ago

It would at least make sense to deal with plurals. I can't tell you the number of times I've searched for something using an s (or not) at the end, and failed to find what I was looking for, only to remember later that I should try to add or remove an s from my search term.

Freaky 17y ago

If it's expensive, then why not reflect that by providing it to people who pay for their accounts?

flatline 17y ago

It seems like they've worked on mitigating this on the web with suggestions for alternate spellings and by displaying related searches. While you can see stemming at work in many Google searches, I'm pretty sure they don't build extensive substring indices on the web end either. For example, I've had searches where a substring returns 0 results and the exact phrase returns a handful.

DannoHung 17y ago

Perhaps it would be an option that they could sell to people?

mistermann 17y ago
Exactly....I have always wondered why there is no google premium...I would gladly pay $100/year to have a collection of certain domains easily removed from my google searches, or pick from a list of my favorite sites for a "site:" search, etc etc.
- sp332 17y ago
  Well, you can do at least that last bit fairly easily. http://www.google.com/coop/cse/ http://www.google.com/coop/cse/