4 ms·

This looks very cool, but what I was really hoping for, is way to finally enter a regular expression in a search box and get results back. Looks like I'll need

This looks very cool, but what I was really hoping for, is way to finally enter a regular expression in a search box and get results back. Looks like I'll need to wait a bit longer.

greglindahl 15y ago
Grep is a mapjob which takes hours to run. You'll be waiting quite a while until anyone can afford to quickly run regex queries against billions of documents! And by then, there will be hundreds of billions of documents.
- _delirium 15y ago
  Google Code Search does do regexes against an impressively large set of documents nearly instantly, though it's clearly much smaller than the set of all webpages. It'd be interesting to know how much Google could scale it; could they handle 100x the number of documents in the current code search? 10,000x?
  - tikhonj 15y ago
    One thing you should note is that Google Code Search, as far as I know, supports regular expressions that are actually regular. This means you can't have an expression like /(ab..)\1/, for example. In all, re2, the regular expression engine that Google Code uses, is a very interesting project; you should read about it on its google code page: http://code.google.com/p/re2/ http://code.google.com/p/re2/.
    - greglindahl 15y ago
      The issue is not so much how much cpu time the regex evaluation takes up, it's the I/O time of loading every byte of every page we've crawled. That being said, re2 does look pretty cool... having a guarantee that nothing in an re can blow up is pretty nice, on top of the overall speed improvement.
krishna2 15y ago
We do plan to eventually support regexes but slowly introduce them with limited-support and not jump head-first with PCRE. Of course, this is only within the scope of webgrepper that I am talking about. PS: I am one of the engineers who built wegbrep
ez77 15y ago
I guess the more accurate motto "Fgrep the Web" would be off-putting even for geeks!

This looks very cool, but what I was really hoping for, is way to finally enter a regular expression in a search box and get results back. Looks like I'll need to wait a bit longer.

greglindahl 15y ago

Grep is a mapjob which takes hours to run. You'll be waiting quite a while until anyone can afford to quickly run regex queries against billions of documents! And by then, there will be hundreds of billions of documents.

_delirium 15y ago
Google Code Search does do regexes against an impressively large set of documents nearly instantly, though it's clearly much smaller than the set of all webpages. It'd be interesting to know how much Google could scale it; could they handle 100x the number of documents in the current code search? 10,000x?
- tikhonj 15y ago
  One thing you should note is that Google Code Search, as far as I know, supports regular expressions that are actually regular. This means you can't have an expression like /(ab..)\1/, for example. In all, re2, the regular expression engine that Google Code uses, is a very interesting project; you should read about it on its google code page: http://code.google.com/p/re2/ http://code.google.com/p/re2/.
  - greglindahl 15y ago
    The issue is not so much how much cpu time the regex evaluation takes up, it's the I/O time of loading every byte of every page we've crawled. That being said, re2 does look pretty cool... having a guarantee that nothing in an re can blow up is pretty nice, on top of the overall speed improvement.

krishna2 15y ago

We do plan to eventually support regexes but slowly introduce them with limited-support and not jump head-first with PCRE. Of course, this is only within the scope of webgrepper that I am talking about. PS: I am one of the engineers who built wegbrep

ez77 15y ago

I guess the more accurate motto "Fgrep the Web" would be off-putting even for geeks!