3 ms·
actually i didn't use PHP, the code i submitted is 'ported' using /regex/ ... i haven't tested it in PHP (i don't have) ... but that what i might use if i were
by hs 18y ago
actually i didn't use PHP, the code i submitted is 'ported' using /regex/ ... i haven't tested it in PHP (i don't have) ... but that what i might use if i were forced to use PHP
the actual code is in newlisp:
<code>
(set 'p1 (regex-comp {[-\.\w]@\w+(\.\w+)+})) ;compile
(map (lambda (f)
(replace p1 (read-file f) (push $0 E) 0x10000)
) (directory "." "php|htm|html"))
(set 'E (unique E))
(save "email.db" 'E)
</code>
my newlisp code is not used for validation, it's for scrapping emails from ~30000 html pages totalling 870 MB
<pre>
~/dl $ find . | wc
29546 29548 1423727
~/dl $ du -h
870M
</pre>
I used more complex regexes (slow) but only marginal improvement (< 0.1%) and a lot of noises
This code is pretty fast, almost an order magnitude faster than more complex, complete email regex
[-\.\\w] means ... match - or \. or word (a-zA-Z0-9) character ad nauseum (can put more chars inside [] but the regex is slower)
i didn't use it for validation, so no anchoring needed ... but it's an easy modification
anyway, this is the code i use for real world - i won't pretend that it's perfect for every corner cases, but hey it works fast for me :D short code solving 90+% cases
- thwarted 18y agoI don't know about the lisp code (where you use \w), but \\w matches a backslash and a w, as the first \ escapes the second. I wasn't saying you were using PHP, but I know the PHP docs were confusing on how to escape things because backslash is overloaded for string escaping and for regular expression escaping and character classes. I can't see how a more complete (complex?) correct regular expression would produce MORE noise in your output. The very fact that you allow the empty string on the LHS means that you risk getting invalid addresses. Continue to use that busted regular expression for parsing email addresses out of these 30,000 HTML files and please remove all my email addresses from the list you generate while you're at it.
- hs 18y agodon't worry, the htmls are ads pages where people deliberately put phone and email to contact to (public info) the correct regex can indeed produce noise, consider this string in an ads where the poster uses '+' to join info: "please contact the owner, her info: 4041234567+owner@gmail.com ... thx!" the correct 'boilerplate' regex will happily take 4041234567+owner@gmail.com ... which is not what i want; thus, noise the less pedantic regex i use will just take owner@gmail.com, which is what i want similarly with "?" char (legal for local part), the correct regex will catch "sendIM?owner@gmail.com"; which again, not what i want ... the 'busted' regex will only take 'owner@gmail.com' ... which is what i want that's the main reason the omissions of +,? and other legal chars in local part of my email regex, the second reason is speed for email validation where what u get is just a string of less than 100 chars, it's fine to use complete email validation regex; however, for email scrapping of close to 1gb data, boilerplate won't cut it i'm willing to sacrifice completeness for speed my regex doesn't allow empty string, there's metachar '\s' for spaces, equivalent to [^ \t\n] and i was wrong, '\w' is [^a-zA-Z0-9_] ... i forgot '^' and '_' are included in metachar '\w' thx, your criticism forced me to reopen my regex book