3 ms·
> XFS has absolutely terrible performance for lots of use cases (deletions, for instance). Depends /heavily/ on hardware. XFS evolved on high-end machines, and
by dexen 15y ago
> XFS has absolutely terrible performance for lots of use cases (deletions, for instance).
Depends /heavily/ on hardware. XFS evolved on high-end machines, and performs awesomely when you have:
* EITHER large write cache (write cache size >> journal size), think `decent RAID controller',
* OR at least put the journal is on a separate harddrive -- which is very real on average workstation.
Been there, done that, the difference is astonishing.
It boils down to specific media access patterns: XFS uses mixed physical/logical journaling; some operations cause a lot of `physical' (i.e., low-level) representation of directory and file metadata to be written to the journal.
Having harddrive's heads fly back-and-forth between different areas (journal vs. metadata) is a sure recipe for abysmal performance. On the other hand, if you have large write cache to handle journal, or at least employ separate harddrive head to serve journal, directory and file metadata has pretty good locality (thanks to very smart allocator) and the other head doesn't have to move too much.
EDIT:
it's worth noting XFS handles large (multi-gigabyte) files very well, as compared to other filesystems. Both r/w access and creation/removal is fast, on any hardware. This is XFS's original and primary use case: handling large multimedia and scientific datasets.
Want to keep countless virtual machine images? XFS is the way to go, especially thanks to smart allocator which lessens fragmentation as compared to competing filesystems.
tl;dr: XFS is optimized for handling large files. With the right setup -- possible on mid-end hardware -- it also handles numerous small files (say, linux-kernel-size projects) very well.
- 0x12 15y agoYes, large files are fine. But contrary to your claim many (10's or 100's of millions) of small files and performance is terrible, orders of magnitudes slower than vanilla ext3. I don't consider the linux kernel to be 'many' files.
- notmyname 15y agoAs we were developing openstack swift, we evaluated many different file systems and settled on xfs because our testing showed it to be faster than other file systems when storing many small files (where "many" is much more than the number of files in the linux kernel). Unfortunately, I can't find the test results that back up my claim.
- 0x12 15y agoYes, storing files is fast. We ran similar tests to yours and storing never was the problem, deletion also wasn't a problem as long as the number of files was in the low millions. The problem comes around when you start to delete files once you have 10's of millions of entries. I helped a customer set up a CDN and this was one of the most painful mistakes I ever made to correct for. It took weeks to migrate all the data to EXT3 filesystems while the system was live. The whole point was to end up with something scaleable, the 'sweep' ran 3 months behind the write so after 90 days, with the filesystems nicely filled to 80% capacity or so we found out that deleting a single file would take an impossibly long time. In the end we found a manageable workaround, to try to select candidates for deletion on a directory-by-directory basis which improved performance to the point that we could migrate the data but it still was a pretty scary operation.
- rbanffy 15y ago> Having harddrive's heads fly back-and-forth between different areas (journal vs. metadata) is a sure recipe for abysmal performance. I wonder how XFS performs on SSD, where head transit is not an issue.