8 ms·
Why Can't Gmail Search?
- tvon 17y agoThe problem seems to be substring searching, which I guess isn't something I've ever tried to use. I tend to think of the gmail search as being fairly powerful and much faster than my usual mail client (Mail.app), it's one of the few reasons I ever use the gmail web interface. If anyone is curious, the docs are here http://mail.google.com/support/bin/answer.py?hl=en&answer=7190 http://mail.google.com/support/bin/answer.py?hl=en&answe... (I had never bothered to look them up until now).
- einarvollset 17y agoI once spoke to someone in the know about this, and the main reason is that it's quite expensive for them to do any reasonable job with stemming, etc. I know that sounds weird (but it's google, they're omnipotent!), but it makes sense: It's worth their while to stem content they crawl and index off the web, cause everybody could in theory access any given page. However, with email, the only person who'll ever benefit is the recipient.
- anurag 17y ago> However, with email, the only person who'll ever benefit is the recipient. I'm not sure that makes sense - if they add stemming, all users of GMail benefit. Going by your explanation, it wouldn't make sense to add any expensive features to GMail, because the only person who would ever benefit from them is the single user.
- antipaganda 17y agoI think you misunderstand the nature of stemming; the point is that each and every user's inbox would have to be processed individually, and apparently Google doesn't think the overhead is worth the results.
- anurag 17y agoI understand stemming. I just think the webpage contrast is not a good explanation for why they're not doing it. Building an unstemmed search index per user is also expensive, and helps only the recipient, but they do it because they think the expense is necessary. They stem webpages because they think the expense is necessary. They don't stem mails because they think the expense is not justified, not because the only person who benefits is the recipient.
- sachinag 17y agoIt is also expensive, but it is less expensive than doing that AND stemming. They just decided that stemming was a line where the benefit (add'l users, more use of Gmail, more AdSense revenue, whatever metric) wasn't worth the (development and ongoing processing) costs.
- jhancock 17y agoThis is a drawback to putting everything in the cloud: features will be weighed by the CPU cycles and storage required by the providers. Can't wait til we come full circle and get back to client/server computing ;). I'm only half joking. I actually can't wait until things mature enough so we have a hybrid of both models. Then I can decide just how much stuff I want indexed and also not worry what happens when my cable modem flakes out.
- gloob 17y agoCan't wait til we come full circle and get back to client/server computing ;). The next big hype cycle: OS as an OS.
- bd 17y agoThey could use Gears for some more in-depth local indexing, then it would be you bearing extra computational and storage costs.
- bradgessler 17y agoThat's a really interesting notion. Have any web applications moved indexing to the client-side before with gears and/or js?
- bd 17y agoMySpace (mentioning cost savings as one of the main reasons): http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-down-server-costs/ http://gigaom.com/2008/05/28/myspace-uses-gears-to-grind-dow... http://developer.myspace.com/community/blogs/devteam/archive/2008/05/29/myspace-mail-search-with-gears-released-google-i-o-presentation-slides.aspx http://developer.myspace.com/community/blogs/devteam/archive...
- IsaacSchlueter 17y agoYou could probably build a gadget that does this, yes?
- zngtk4 17y agoIt would at least make sense to deal with plurals. I can't tell you the number of times I've searched for something using an s (or not) at the end, and failed to find what I was looking for, only to remember later that I should try to add or remove an s from my search term.
- Freaky 17y agoIf it's expensive, then why not reflect that by providing it to people who pay for their accounts?
- flatline 17y agoIt seems like they've worked on mitigating this on the web with suggestions for alternate spellings and by displaying related searches. While you can see stemming at work in many Google searches, I'm pretty sure they don't build extensive substring indices on the web end either. For example, I've had searches where a substring returns 0 results and the exact phrase returns a handful.
- DannoHung 17y agoPerhaps it would be an option that they could sell to people?
- mistermann 17y agoExactly....I have always wondered why there is no google premium...I would gladly pay $100/year to have a collection of certain domains easily removed from my google searches, or pick from a list of my favorite sites for a "site:" search, etc etc.
- sp332 17y agoWell, you can do at least that last bit fairly easily. http://www.google.com/coop/cse/ http://www.google.com/coop/cse/
- _pius 17y agoFinally, someone else who feels this way! I've never understood why people praise the search in GMail so much.
- jsonscripter 17y agoGmail search is fast. On outlook, a desktop mail client, it can take up to an hour to search for a simple term, in my experience.
- shiranaihito 17y agoIt may be fast, but what you want is results.
- jsonscripter 17y agoI think some people also want timely results. I certainly do.
- endtime 17y agoIdeally, yes, but that doesn't appear to be an option. Given the choice between instant and incomplete results, and slightly slower but complete results, I'll take the latter.
- TallGuyShort 17y agoConsidering the difference in speed, it's faster for me to find what I was looking for by sifting through GMail's results, than to wait for Outlook's results to show up at all. Outlook's search essentially renders my whole machine useless until it's done searching.
- timcederman 17y agoHave you used Outlook 2007 with the built-in search indexer? Incredibly quick.
- 17y ago
- imp 17y agoI think I hit the same problem occasionally, but I can usually keep trying different keywords until I find one that works. If he only tried 4 keywords like he writes, then there was probably some word in the email that would have been at least semi-unique. Maybe "earbud" or "order" or something. Then browsing or date-based filtering can usually do the rest. Kind of a pain, but not enough for me to ditch GMail completely.
- mildweed 17y agoI hope they build more robust searching into their Wave client.
- euroclydon 17y agoThat's really clever how he auto-forwards all his gmail messages to yahoo. I wish I had done something like that form the get go. Too late now. Here is a Lifehacker article explaining how to use automate gmail backups with sendmail and cygwin: http://lifehacker.com/235207/geek-to-live--back-up-gmail-with-fetchmail http://lifehacker.com/235207/geek-to-live--back-up-gmail-wit...
- ramoq 17y agoI actually heard that it's quite costly to index emails for gmail (not from a gmail source, just random web chatter). It makese sense. Most emails are not important, they are just huge amounts of random chatter. I'd imagine indexing emails (full-text) properly would require some effort. The gmail team is probably on a budget :) But then again, if they can index the web. Why not email.
- plusbryan 17y agoI'm no expert, but reMail does it on my freakin iPhone - I'm guessing they could do even better on a cluster...
- ramoq 17y agoAgreed, Google could probably do it better. But imagine maintaining separate indexes for each and every gmail acct and constantly updating those indexes. reMail used to do something similar to that.
- metachor 17y agoWell, they already do (and have to) maintain separate indexes for each and every gmail account (or else how would you search on the metadata fields like To: and From: in your inbox at all?). Supposedly the issue is that they don't perform more computationally-expensive linguistic analysis during the indexing phase. If they tokenize each word but don't perform any stemming or lemmatization, for example, the result would be similar: only full-word non-substring matching. As others have pointed out, its probably a cost-benefit decision by Google to not spare grid cycles on full-fledged linguistic analysis for individual's email accounts. Google CAN do better at it, as is evidenced by their web search index.
- ajb 17y agoI have hit this problem too. If you subscribe to the git email list, but only want to look at discussion, not patches, it ought to be possible to filter out the patches with 'subject:patch'. But this doesn't completely work, because quite often the patches have patchv2 patchv3 etc, which doesn't match. The curious thing is that google groups is even more painful; where you would think it would be more worth having the indexes, because more people are going to search the same data.
- sfk 17y agoI think Google found that they can make more money by directing Usenet searches to crappy archive sites full of Google ads. Which is a shame, since 5 years ago Usenet search was absolutely wonderful.
- mblakele 17y agoI find it easier to let an external service index my list subscriptions: http://markmail.org/search/?q=list%3Aorg.kernel.vger.git#query:type%3Adevelopment http://markmail.org/search/?q=list%3Aorg.kernel.vger.git#que... (or http://git.markmail.org/search/?q=#query:type%3Adevelopment http://git.markmail.org/search/?q=#query:type%3Adevelopment if you want to use the list-specific site).
- sundae79 17y agoNot only that, I also found that if you search by sender name and the sender name is never in any of the subject of any of your emails, it doesnt find it.
- tvon 17y agouse "from:example.com" or similar... important part being the "from:" bit
- gecko 17y agoSpeaking of which: you and I, as developers, understand search axes, and use them intuitively. But they're utterly opaque for someone who hasn't encountered that interface before--and given that Google itself doesn't support search axes in its main product, I don't think they're an interface someone's likely to learn elsewhere, either. If you're unaware of search axes, you have to click, "Show search options," which is in an extremely tiny font, next to a button that very notably says "Search the Web"--not search my inbox. Furthermore, once you actually discover that, and use it, you still don't know about the axes, because that search box doesn't use them. The only place where you can discover search axes is by clicking on a label, which results in "label:foo" being shown in the search box--but that's actually a lie, because the output if you dump that into the search field, versus if you click on a label name, don't match if you have more than about 30 messages in that label. Go try it. So, yeah. Axes are great. And they're completely unintuitive and impossible to discover in Gmail unless you read the help, which no user ever does. So you've done a great job finding a part of Gmail's search interface that's at least as broken as the underlying implementation.
- tvon 17y ago> The only place where you can discover search axes is by clicking on a label, which results in "label:foo" being shown in the search box (...) That's how I found it, and I learned a little more about it by a need to get all mail to/from a specific client for a specific month and doing advanced searches. Granted, that's also how I learned about "site:", by going to advanced search on Google and specifying a single site to search, the resulting page shows the "site:" in the search box. > (...) but that's actually a lie, because the output if you dump that into the search field, versus if you click on a label name, don't match if you have more than about 30 messages in that label. Go try it. Good point, though as far as I can tell the number of results is the only difference... but still, it's a bit misleading.
- moldenke 17y agoAt least it's better than Google Groups search.
- sahaj 17y agojust an observation: lot of attacks on gmail today/recently.
- justinhj 17y agoGood points there. When I search in google I know I can be sloppy with my typing because 90% of the time it's quicker to get it wrong and click the "did you mean?" results, rather than edit my input. That feature's absence is very obvious when struggling to find data in gmail. Personally I use subject line tags for stuff I want to filter on. (Like 'music' 'todo' 'idea'), and when I store something in gmail I want to remember I make sure the key words I will search for are very obvious (and easy to spell).
- periferral 17y agoit seems to me that this problem isnt just limited to gmail. other than google search itself, I think search in other google products (android included) is lacking when it comes to what I presume to be basic functionality.
- dpcan 17y agoHis experiment fails. He didn't apply the same inputs to both systems, therefore his findings must be discarded. He changed it up, and had he STARTED at Yahoo and performed the exact same searches, he would yield the same results, only reversed, and his blog post would be about why he dumped Yahoo instead. Here's my case: 1) He searched Google for "Zags" which is NOT in "Zaggs". So, no results. 2) Then he goes to Yahoo and searches for "Zag" which IS in "Zaggs" - AH HA, he gets a result (of course he does!) 3) He RETURNS to Google and searches for "Zagg" which IS in "Zaggs", and boom, results. Surprised? Me neither. I guess my point is, I don't have enough karma to have down arrows next to this post, so I'm going to write a cranky comment about this article.
- beza1e1 17y agoHe did search for "zag" in Gmail. Right after "zbuds" and before "headset".
- dpcan 17y agoGAH! All that frustration over nothing. Thank you.
- nkurz 17y ago3) He RETURNS to Google and searches for "Zagg" which IS in "Zaggs", and boom, results. Surprised? Me neither. You might want to hold off on your cranky comment. Would you be surprised if the search for 'Zagg' did NOT turn up the results for 'Zaggs'? Because for better or worse, that's what actually happens. GMail search does not perform stemming (like removing that final 's') and also does not allow for substring searches. So in fact, a search for 'Zagg' will return nothing. While this isn't a fatal flaw, it is a drag.
- ramoq 17y agoOh yeah, the gap in gmail's search ability does hint at possible startup ideas to solve this problem. However I wonder why some great startups have passed on directly competing in this sphere (reMail being a good example)
- calcnerd256 17y agoIf lack of substring search is bad, how much worse is the fact that it sometimes doesn't even return all the results for a correct query. I use labels a lot, and I often use boolean searches to search for exactly which Venn intersection I need. A search for "in:inbox is:unread -label:Triangle" (without the quotes) in my gmail turns up messages labeled Triangle. I have similar other problems and cannot trust gmail's search.
- calcnerd256 17y agoMy solution was to use fetchmail, mb2md, and fgrep on my local server. Now I feel like I can almost trust the setup I have.
- andyf 17y agoAt least gmail filters out spam better than any other mail client and yes it includes Yahoo Mail.
- zandorg 17y agoTypical of someone to glibly say 'where's regex matching?' when it's very advanced, and it's possible Google simply don't have the cpu for making (and searching) a search tree for many megabytes of mail. I don't know how Yahoo do it, but this guy should at least present a solution to the problem. It reminds me of that Alexei Sayle sketch where he says 'I blame the council' (which in the UK is the local authority and handles all sorts of things in a town or city), and at the end, wanting someone to blame for blaming the council? He blames the council!
- Jem 17y agoI did a search for a mail I was sure I had in gmail a few weeks ago, and it returned 0 results. I came to the conclusion that I'd simply deleted the mail. Now I'm wondering if it wasn't me at all.
- cubedice 17y agoWhat's interesting is that after reading this article I've realized I always expected e-mail search to be bad. Why? Possibly because, when another entity holds a large portion of my social information, it feigning ignorance about deep technical knowledge of my personal life (which is likely better than my own understanding) makes me feel good. It's less 'big brother'y, which may explain why poor e-mail search doesn't bother me. It's as though its waiting for me to get the answer first.
- snprbob86 17y agoI'm going to keep my fingers crossed for "search as you type" across all Google properties. This has got to happen...someday...right?
- slater 17y agoI'd suggest he tries using Lotus Notes' built-in "search". If there was ever a more useless function, it's LN's pathetic attempt at searching for stuff. It'll find stuff to every search term you enter! Just nothing that a) contains that search term, and b) resembles anything you wanted to find.