7 ms·

Greplin’s Chrome Extension Now Makes Gmail Search Infinitely Better

55 points by ssclafani 16y ago

smanek 16y ago
The coolest features: "Once you’ve started searching, you’ll notice that tweaks to your query show up in real-time as you type each character, the same way they do on Google Instant. And it shows results for partial-word matches (“Tech” would match for both “Technology” and “TechCrunch”) — which Gmail doesn’t do." And "a message with 500k of attachments showed up in my search results within a second of receiving it. Not too shabby." Full disclosure: I work at Greplin :-D I'm just proud of the very cool stuff my coworkers are building!
- alooPotato 16y ago
  How exactly is the data from gmail gathered so quickly? Presumably they are using a gmail API, but does that mean they are polling very frequently? I guess another way to do it would be to create an auto forward to greplin filter in gmail when a user connects their gmail account but that seems wrong.....
  - rwalker 16y ago
    We use IMAP IDLE: http://tools.ietf.org/html/rfc2177 http://tools.ietf.org/html/rfc2177
    - a13ph 16y ago
      Is it used for indexing or at the moment query hits the search box? (I am interested in how this process works in more words than 4 and less than in that tech spec that i am illiterate about, sorry for that)
      - kevinclark 16y ago
        For GMail, our syncers hang out on an IDLE'd IMAP connection and process (and then index) whenever a new message comes in. Edit: So, to clarify, it's used to trigger an update to the index. Hopefully this happens as close to the time the message hits the inbox as possible.
    - lobster_johnson 16y ago
      So for every Greplin user you have a TCP connection open to Gmail's IMAP server? How can that possibly scale in the long run?
  - deleted 16y ago
    [deleted]
  - a13ph 16y ago
    Data is indexed to Greplin's servers
Groxx 16y ago
I hear about these long search waits in Gmail, but I have yet to encounter them. Granted, I only have about 26k emails, but every single search I've made completes within the second. Also, killer feature question: does Greplin allow search operators, like `label:`, `from:`, `has:attachment` ? And can it search within those attachments (not sure if any desktop clients do this currently, but I wants it, Precious. I needs it.)?
- danicgross 16y ago
  from:foo, has:attachment, to:foo all should work.
joelthelion 16y ago
Soon these people are going to discover thunderbird :) Why does history always seem to repeat itself?
- Groxx 16y ago
  I keep re-trying Thunderbird, hoping it has improved, but keep abandoning it because it keeps annoying me. Especially that embarrassing contacts importer. But it's still the best I've found on Windows, so I keep installing it everywhere, and helping others do so as well, especially at school (as the school's setup info is about 3 versions of their email server behind).
  - a13ph 16y ago
    Isn't obvious way to fix outdated setup guide is to write-edit new? (if it will cost less time to you rather than teach everyone in person)
    - Groxx 16y ago
      They won't let me. Despite working for over a year in what's essentially their employee-tech-support / teach-me-how department. But yes.
- shadowfox 16y ago
  Care to explain? I have been using Thunderbird for 3-4 years at this point. I have indexing on and that sort of stuff. But searches that include the body text is remarkably slow. It does not seem to search inside attachments as well. It would be awesome if I could fix these.
d_c 16y ago
How do they access the data used to build their search index?
- smanek 16y ago
  You have to sign up at https://www.greplin.com https://www.greplin.com, and grant us permission (via OAuth - so we never see your password) to index your mail for you. See http://googlecode.blogspot.com/2010/03/oauth-access-to-imapsmtp-in-gmail.html http://googlecode.blogspot.com/2010/03/oauth-access-to-imaps... for some details.
andrewcooke 16y ago
"there’s one possible issue for those of you who are concerned about your privacy" for anyone worried about this kind of thing, there's a program called mairix that will index and search your local maildirs. it's command line, but the results can be used to populate another maildir folder, so it integrates pretty nicely with clients - http://www.rpcurnow.force9.co.uk/mairix/ http://www.rpcurnow.force9.co.uk/mairix/ if you want something more C21 then there's also beagle -http://beagle-project.org/Main_Page http://beagle-project.org/Main_Page [edit: skip that; looks like it's a dead project; recoll might be a good replacement - http://www.lesbonscomptes.com/recoll/ http://www.lesbonscomptes.com/recoll/ ]
sinaiman 16y ago
I've never had a problem with Gmail search and I'm surprised the article writes about 20+ second search times as if it's a common and widespread issue.
- psykotic 16y ago
  I have around 45,000 messages in my Gmail inbox. My experience searching it has felt sluggish for a long time. But really the fatal flaw is that search terms have to be damn near exact. Gmail is a joke compared to Google when it comes to fuzzy searches. I often find myself having to do several searches to get what I want, compounding the issue of search latency. It's so bad I sometimes give up entirely.
  - greattypo 16y ago
    I'm at 264,500. And yes, it's about as terrible as you'd imagine.
    - psykotic 16y ago
      Ouch. If you're anything like me, around 90% of your traffic comes from mailing lists. Greplin could win big by inferring what mailing lists you are on (along with beginning and ending subscription dates) in order to factor out all relevant storage and indexing.
      - klochner 16y ago
        Do you need to store mailing list email? Usually there's a public archive.
        mbreese 16y ago
        Sometimes it's just easier to search for things in Gmail. But since it is just a mailing list, it's pretty easy to label it and kill it en masse if you need to. At least, that's what I do.
- 16y ago
psykotic 16y ago
It seems pretty promising so far. I guess my next Emacs hacking project will be a Greplin source for anything.el. Has anyone written plug-ins for Spotlight or Quicksilver? For someone like me whose attraction is improved Gmail searching, I can't see paying $4.99/month for this. Were my daily workflow based on Google Apps and Evernote, it'd be another story.
salsakran 16y ago
I think all the talk of being faster than gmail search is going to last until they actually get a lot of users. It's easy to make search fast for small datasets.
- psykotic 16y ago
  You're not making any sense. They can shard the hell out of their indexes since all searches are user specific, so that's a non-issue. They can afford to throw enough hardware at the problem if their business model works out and they attract enough paying users. It's presumably not cost effective for Google to offer a super snappy search experience through gigabytes of user-specific data, which is why Gmail search sucks so badly for people like me. (Does anyone know if search is way better if you're a paying customer of Google Apps?)
  - deleted 16y ago
    [deleted]
  - salsakran 16y ago
    The existence of a way to shard searches doesn't make scaling real time search on email (hint: do some back of the envelope calculation on how much data that involves) a non-issue.
    - psykotic 16y ago
      Fair enough, I too readily dismissed your drive-by objection, however non-specific and snide. Scaling real-time anything is never easy. My main point was that cost effectiveness is what keeps GMail from offering a similar level of search quality. If Greplin can get a large enough fraction of users to pay for their service, the technical challenges are surmountable.
greattypo 16y ago
I want to love Greplin. I hate gmail search, and would kill for anything better. But giving a startup a complete database of my e-mails? That is a little scary. Anyone else have the same concerns?
- wotsrovert 16y ago
  Yes. Scares me too much to even consider the service.
- kelnos 16y ago
  Yep. And above it looks like they use IMAP IDLE to watch for new mail in your account, which presumably means they need your real Google password, and don't go through the OAuth stuff (unless Google's IMAP server has an alternate auth method). Deal-breaker for me. Having access to all of my data... well... Google already has access to all my data, and they were once a startup. Of course, Greplin could get bought by a company with a poor privacy track record. So there's that to consider. I'm not sure that alone would keep me away from the service, but it might. Sigh... but it is really cool. EDIT: Neeeeevermind, just saw this: http://news.ycombinator.com/item?id=2452068 http://news.ycombinator.com/item?id=2452068
- catch23 16y ago
  Maybe it would be okay to use Greplin for company mail stuff where technically the company already owns all of your emails anyway -- of course they'd have to deploy Greplin company wide though.
- tdurden 16y ago
  Deal breaker here. Giving a company, start-up or not, access to _all_ my data just so I can search easily? Not a chance.
- mpakes 16y ago
  People said the same thing about Mint, but things worked out pretty nicely for them..
- fmavituna 16y ago
  Same here, I would pay good money for the offline version. Personally this falls into the category of "too much risk not enough benefits"
rxin 16y ago
https://www.cloudmagic.com/ https://www.cloudmagic.com/ Try this. It builds a local index of your gmail messages. Instant gmail search (and also matches partial keywords like Greplin). No worry about exposing your data to a third party service.
- wazoox 16y ago
  And CloudMagic is available for both Chrome and Firefox. This is important to me because I'm a strict FF fanatic.

55 points by ssclafani 16y ago

The coolest features: "Once you’ve started searching, you’ll notice that tweaks to your query show up in real-time as you type each character, the same way they do on Google Instant. And it shows results for partial-word matches (“Tech” would match for both “Technology” and “TechCrunch”) — which Gmail doesn’t do." And "a message with 500k of attachments showed up in my search results within a second of receiving it. Not too shabby." Full disclosure: I work at Greplin :-D I'm just proud of the very cool stuff my coworkers are building!

alooPotato 16y ago
How exactly is the data from gmail gathered so quickly? Presumably they are using a gmail API, but does that mean they are polling very frequently? I guess another way to do it would be to create an auto forward to greplin filter in gmail when a user connects their gmail account but that seems wrong.....
- rwalker 16y ago
  We use IMAP IDLE: http://tools.ietf.org/html/rfc2177 http://tools.ietf.org/html/rfc2177
  - a13ph 16y ago
    Is it used for indexing or at the moment query hits the search box? (I am interested in how this process works in more words than 4 and less than in that tech spec that i am illiterate about, sorry for that)
    - kevinclark 16y ago
      For GMail, our syncers hang out on an IDLE'd IMAP connection and process (and then index) whenever a new message comes in. Edit: So, to clarify, it's used to trigger an update to the index. Hopefully this happens as close to the time the message hits the inbox as possible.
  - lobster_johnson 16y ago
    So for every Greplin user you have a TCP connection open to Gmail's IMAP server? How can that possibly scale in the long run?
- deleted 16y ago
  [deleted]
- a13ph 16y ago
  Data is indexed to Greplin's servers

Groxx 16y ago

I hear about these long search waits in Gmail, but I have yet to encounter them. Granted, I only have about 26k emails, but every single search I've made completes within the second. Also, killer feature question: does Greplin allow search operators, like `label:`, `from:`, `has:attachment` ? And can it search within those attachments (not sure if any desktop clients do this currently, but I wants it, Precious. I needs it.)?

danicgross 16y ago
from:foo, has:attachment, to:foo all should work.

joelthelion 16y ago

Soon these people are going to discover thunderbird :) Why does history always seem to repeat itself?

Groxx 16y ago
I keep re-trying Thunderbird, hoping it has improved, but keep abandoning it because it keeps annoying me. Especially that embarrassing contacts importer. But it's still the best I've found on Windows, so I keep installing it everywhere, and helping others do so as well, especially at school (as the school's setup info is about 3 versions of their email server behind).
- a13ph 16y ago
  Isn't obvious way to fix outdated setup guide is to write-edit new? (if it will cost less time to you rather than teach everyone in person)
  - Groxx 16y ago
    They won't let me. Despite working for over a year in what's essentially their employee-tech-support / teach-me-how department. But yes.
shadowfox 16y ago
Care to explain? I have been using Thunderbird for 3-4 years at this point. I have indexing on and that sort of stuff. But searches that include the body text is remarkably slow. It does not seem to search inside attachments as well. It would be awesome if I could fix these.

d_c 16y ago

How do they access the data used to build their search index?

smanek 16y ago
You have to sign up at https://www.greplin.com https://www.greplin.com, and grant us permission (via OAuth - so we never see your password) to index your mail for you. See http://googlecode.blogspot.com/2010/03/oauth-access-to-imapsmtp-in-gmail.html http://googlecode.blogspot.com/2010/03/oauth-access-to-imaps... for some details.

andrewcooke 16y ago

"there’s one possible issue for those of you who are concerned about your privacy" for anyone worried about this kind of thing, there's a program called mairix that will index and search your local maildirs. it's command line, but the results can be used to populate another maildir folder, so it integrates pretty nicely with clients - http://www.rpcurnow.force9.co.uk/mairix/ http://www.rpcurnow.force9.co.uk/mairix/ if you want something more C21 then there's also beagle -http://beagle-project.org/Main_Page http://beagle-project.org/Main_Page [edit: skip that; looks like it's a dead project; recoll might be a good replacement - http://www.lesbonscomptes.com/recoll/ http://www.lesbonscomptes.com/recoll/ ]

sinaiman 16y ago

I've never had a problem with Gmail search and I'm surprised the article writes about 20+ second search times as if it's a common and widespread issue.

psykotic 16y ago
I have around 45,000 messages in my Gmail inbox. My experience searching it has felt sluggish for a long time. But really the fatal flaw is that search terms have to be damn near exact. Gmail is a joke compared to Google when it comes to fuzzy searches. I often find myself having to do several searches to get what I want, compounding the issue of search latency. It's so bad I sometimes give up entirely.
- greattypo 16y ago
  I'm at 264,500. And yes, it's about as terrible as you'd imagine.
  - psykotic 16y ago
    Ouch. If you're anything like me, around 90% of your traffic comes from mailing lists. Greplin could win big by inferring what mailing lists you are on (along with beginning and ending subscription dates) in order to factor out all relevant storage and indexing.
    - klochner 16y ago
      Do you need to store mailing list email? Usually there's a public archive.
      - mbreese 16y ago
        Sometimes it's just easier to search for things in Gmail. But since it is just a mailing list, it's pretty easy to label it and kill it en masse if you need to. At least, that's what I do.
16y ago

psykotic 16y ago

It seems pretty promising so far. I guess my next Emacs hacking project will be a Greplin source for anything.el. Has anyone written plug-ins for Spotlight or Quicksilver? For someone like me whose attraction is improved Gmail searching, I can't see paying $4.99/month for this. Were my daily workflow based on Google Apps and Evernote, it'd be another story.

salsakran 16y ago

I think all the talk of being faster than gmail search is going to last until they actually get a lot of users. It's easy to make search fast for small datasets.

psykotic 16y ago
You're not making any sense. They can shard the hell out of their indexes since all searches are user specific, so that's a non-issue. They can afford to throw enough hardware at the problem if their business model works out and they attract enough paying users. It's presumably not cost effective for Google to offer a super snappy search experience through gigabytes of user-specific data, which is why Gmail search sucks so badly for people like me. (Does anyone know if search is way better if you're a paying customer of Google Apps?)
- deleted 16y ago
  [deleted]
- salsakran 16y ago
  The existence of a way to shard searches doesn't make scaling real time search on email (hint: do some back of the envelope calculation on how much data that involves) a non-issue.
  - psykotic 16y ago
    Fair enough, I too readily dismissed your drive-by objection, however non-specific and snide. Scaling real-time anything is never easy. My main point was that cost effectiveness is what keeps GMail from offering a similar level of search quality. If Greplin can get a large enough fraction of users to pay for their service, the technical challenges are surmountable.

greattypo 16y ago

I want to love Greplin. I hate gmail search, and would kill for anything better. But giving a startup a complete database of my e-mails? That is a little scary. Anyone else have the same concerns?

wotsrovert 16y ago
Yes. Scares me too much to even consider the service.
kelnos 16y ago
Yep. And above it looks like they use IMAP IDLE to watch for new mail in your account, which presumably means they need your real Google password, and don't go through the OAuth stuff (unless Google's IMAP server has an alternate auth method). Deal-breaker for me. Having access to all of my data... well... Google already has access to all my data, and they were once a startup. Of course, Greplin could get bought by a company with a poor privacy track record. So there's that to consider. I'm not sure that alone would keep me away from the service, but it might. Sigh... but it is really cool. EDIT: Neeeeevermind, just saw this: http://news.ycombinator.com/item?id=2452068 http://news.ycombinator.com/item?id=2452068
catch23 16y ago
Maybe it would be okay to use Greplin for company mail stuff where technically the company already owns all of your emails anyway -- of course they'd have to deploy Greplin company wide though.
tdurden 16y ago
Deal breaker here. Giving a company, start-up or not, access to _all_ my data just so I can search easily? Not a chance.
mpakes 16y ago
People said the same thing about Mint, but things worked out pretty nicely for them..
fmavituna 16y ago
Same here, I would pay good money for the offline version. Personally this falls into the category of "too much risk not enough benefits"

rxin 16y ago

https://www.cloudmagic.com/ https://www.cloudmagic.com/ Try this. It builds a local index of your gmail messages. Instant gmail search (and also matches partial keywords like Greplin). No worry about exposing your data to a third party service.

wazoox 16y ago
And CloudMagic is available for both Chrome and Firefox. This is important to me because I'm a strict FF fanatic.