3 ms·
Indeed, I've actually written such a thing (although in Rust, not C++), I'm still looking for a clean concise version. :-) Here's the public API of mine[1], wh
by burntsushi 8y ago
Indeed, I've actually written such a thing (although in Rust, not C++), I'm still looking for a clean concise version. :-)
Here's the public API of mine[1], which is pretty blah, because it takes a closure (per-thread initialization) that must return a closure (executed on each entry in the tree). The most interesting part of the implementation is here[2], along with a suspicious termination argument.
Outside of that, ucg (written in C++) also has a recursive directory iterator.[3]
I also know of two written in Go, one in sift[4] and another in the platinum searcher[5]. The ones in Go seem considerably simpler as they fall back to goroutines.
I haven't carefully benchmarked all of these, but if memory serves, they're all faster than the single threaded variants. In my experience, the hardest part about implementing this is coming up with a solid termination argument because your consumers are also your producers. (I documented this a bit in `get_work` in my Rust code.)
[1] - https://docs.rs/ignore/0.4.2/ignore/struct.WalkParallel.html https://docs.rs/ignore/0.4.2/ignore/struct.WalkParallel.html
[2] - https://github.com/BurntSushi/ripgrep/blob/b38b101c77003fb94aaaa8084fcb93b6862586eb/ignore/src/walk.rs#L940-L1378 https://github.com/BurntSushi/ripgrep/blob/b38b101c77003fb94...
[3] - https://github.com/gvansickle/ucg/blob/master/src/libext/DirTree.cpp https://github.com/gvansickle/ucg/blob/master/src/libext/Dir...
[4] - https://github.com/svent/sift/blob/2ca94717ef0bbf43068f217c8a0dfb216df52225/sift.go#L159-L376 https://github.com/svent/sift/blob/2ca94717ef0bbf43068f217c8...
[5] - https://github.com/monochromegane/the_platinum_searcher/blob/master/walk.go https://github.com/monochromegane/the_platinum_searcher/blob...
- jjnoakes 8y agoYeah, I've written such a thing many times as well, in languages like Go which make it easy and languages like C which make it not-as-easy and a few in between. I was more curious if there was a cleaner C++ way of doing it with all of the recent or pending languages changes.