28 ms·
Yeah, because ext4 never cleans up the dentry table after it's been made. This is a pain. Unfortunately XFS also sucks for handling larger numbers of files - a
by jfindley 12y ago
Yeah, because ext4 never cleans up the dentry table after it's been made. This is a pain.
Unfortunately XFS also sucks for handling larger numbers of files - at the hundreds of millions or larger scale, you're largely metadata bound - and XFS is very slow at this for a number of reasons.
RH did an interesting study a while back [0] on storing 1 billion files, which came out in favor of ext4. If you're able to work around the lack of dentry cleanup, I still think ext4 is probably the best choice here (ideally with the journal on a small SSD) - assuming you can't store them in a database or something more appropriate like mogileFS [1]
[0]: http://www.redhat.com/summit/2011/presentations/summit/decoding_the_code/thursday/wheeler_t_0310_billion_files_2011.pdf http://www.redhat.com/summit/2011/presentations/summit/decod...
[1]: https://code.google.com/p/mogilefs/wiki/Start https://code.google.com/p/mogilefs/wiki/Start
- otterley 12y agoSlide 32 suggests that XFS performance took the lead again after RHEL 6.2. Am I mistaken? See also https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html-single/6.2_Release_Notes/#filesystem https://access.redhat.com/site/documentation/en-US/Red_Hat_E...
- justin66 12y agoI wonder if you know about Dave Chinner's talk, which addresses the metadata performance issue. (they fixed it) https://www.youtube.com/watch?v=FegjLbCnoBw https://www.youtube.com/watch?v=FegjLbCnoBw
- jfindley 12y agoI hadn't seen that - that's interesting. Thanks.
- rbanffy 12y ago> Yeah, because ext4 never cleans up the dentry table after it's been made. Why is that? It doesn't seem to make much sense.
- deleted 12y ago[deleted]