Tangentially related, but I’m interested in doing file system traversal for finding some paths in Python: using a simple scandir to process takes minutes on the

Tangentially related, but I’m interested in doing file system traversal for finding some paths in Python: using a simple scandir to process takes minutes on the data I’m working with: shelling out to find is faster but still takes minutes. I know ‘fd’ exists and does parallel traversal, but I’d like to learn more about how to implement something like that: what structures do file systems use and how do you know what limitations should be considered when working with files?