4 ms·
While we're talking about obscure speed differences, I recently needed to traverse large directories and collect files with a certain suffix and found out that
by JohnStrange 10y ago
While we're talking about obscure speed differences, I recently needed to traverse large directories and collect files with a certain suffix and found out that using "find" as a subprocess and parsing its output line-by-line was orders of magnitudes faster than any direct directory traversal I wrote and any other method I've tried, including other Unix utilities.
I don't know how it works but it's insanely fast.
- Lanzaa 10y agoThe find utility is probably smarter than the other methods. I know two things to watch out for: sorting the directory entries and calling stat on each entry. You might like to read the rationale section of PEP 471: https://www.python.org/dev/peps/pep-0471/ https://www.python.org/dev/peps/pep-0471/