3 ms·
No date on the post. Probably old and fixed now. Just checked with a random folder, it happily supported 782292 files.
by rshm 10y ago
No date on the post. Probably old and fixed now. Just checked with a random folder, it happily supported 782292 files.
- Hnrobert42 10y agoI think his point still stands. Basically, if you do something far enough outside the norm, you may hit really time consuming bugs.
- bArray 10y agoAlthough the exception would probably be rare if it used a large enough number, do you think the "copying utility" (whatever it may be) should handle those extreme edge cases? Or is it just "disk full" that's used as a fall-back?
- ryanbertrand 10y agoThere is mention of 2016. "The lesson here is that even in 2016, filesystems still are finicky with large numbers of files in a single directory."
- Xylakant 10y agoSince the root cause is a hash collision in the dir_index code you might just have been lucky and the author of the article unlucky.
- caf 10y agoThey would have had to have been exceptionally lucky - the chance of getting no collisions in a 32 bit hash among 782292 inputs is about 10^-62.
- rshm 10y agoProbably lucky. File names are md5 of the content, i am not sure if it helped on collision on 32 bit md4. The linked post mentions dir_index default and tune2fs -l lists the dir_index under Filesystem features only. Not knowing this default behavior, i was considering enabling dir_index explicitly to aid chocking rsync. EDIT: I ran the reproduction code with 200K instead of 100K just to be sure. There was no problem creating file as well as copying all the files to a different folder. The disk was formatted about 3 months ago and mounted with default options "ext4 noatime,barrier=0 0 0" . On original as well as copied folder `ls | wc -l` still gives 200000.