3 ms·
Awwww. I hoped for a moment it was literally searching the web with a regex. Unfortunately no, although it may be a handy regex cheat sheet. Wake me when someo
by AlyssaRowan 12y ago
Awwww. I hoped for a moment it was literally searching the web with a regex. Unfortunately no, although it may be a handy regex cheat sheet.
Wake me when someone does this properly. It's only been done on a small scale or with very limited precomputed expressions, to my knowledge. Not many people would need it, but for those people it'd be insanely useful. But it's also insanely computationally hard - which means it'd be a really interesting technical achievement! There are no general-purpose reverse indexes that I know of that accelerate that as easily as keywords, but there are some data structures that might help a bit, although I can't think of practical ways to deploy them over arbitrary regexps specified at runtime! Plus some sanity heuristics and limits, of course, as regexps can undergo combinatorial explosion and some fun unexpected worse-case performance.
- unhappyhippie 12y agoMaybe I am missing out potential applications but I can't understand why I would like to search the entire web with a regex pattern? Find all strings that could be SSNs?
- BenoitP 12y agoFor 580$, you can still do it: http://www.michaelnielsen.org/ddi/how-to-crawl-a-quarter-billion-webpages-in-40-hours/ http://www.michaelnielsen.org/ddi/how-to-crawl-a-quarter-bil...