3 ms·
Is this still the case if you set LANG=C on the command, so that you're lowercasing ASCII bytes instead of Unicode?
by rspeer 11y ago
Is this still the case if you set LANG=C on the command, so that you're lowercasing ASCII bytes instead of Unicode?
- lqdc13 11y agoEdit: There was a bug. No it's actually slower when the locale is set to C. I have 4434 files with about 100 words of text in them. here is python func that searches utf8: def find_sp(paths): for path in paths: with open(path,'rb') as f: if b"snow" in f.read().lower(): yield True else: yield False %timeit [o for o in find_sp(glob.glob('./*'))] 10 loops, best of 3: 35.2 ms per loop Here is the bash one: $>echo $LANG C $>time grep -Riq "snow" . real 0m0.015s user 0m0.005s sys 0m0.010s However, if you have a lot of files and you are doing the searching for them in a not very smart way, it takes a lot longer. time for f in ./*txt; do grep -iq "snow" $f;done real 0m2.307s user 0m0.337s sys 0m2.113s
- jbnicolai 11y agoIn the python example you're running a function, while in the grep example you're launching a program (grep) from your shell. To make the comparison more fair you'd store the python program and time starting and running both from a shell.