3 ms·

I haven't checked in a really long time, but years ago ntfs-3g had a really suboptimal allocation pattern for spinning drives, and optimizations were only prese

I haven't checked in a really long time, but years ago ntfs-3g had a really suboptimal allocation pattern for spinning drives, and optimizations were only present in the commercial version. I don't know if the block allocation strategy was owned by ntfs-3g or if it was using something in-kernel. If I'm remembering the details right (and that's a big if), large files written sequentially would have their blocks scattered all over the place, making e.g. large uncompressed videos written from Linux too slow to play due to seek times, when if they'd been written sequentially they would have been fine.

pengaru 6y ago
ntfs-3g is entirely implemented in userspace, and is far more trivial to hack on and test if anyone wanted to improve its file allocation strategies than an in-kernel driver would be. Years ago when working on a commercial virtualization product I had to support initializing sparsely populated ntfs filesystems where only the filesystem metadata was written out, with allocated file extent ranges logged externally and not actually populated in the sparse ntfs filesystem in a file. I ended up deriving a small library for doing this from the libntfs-3g code, I don't think it took more than a weekend to do, the code was quite sane. What I recall from that experience was that the file extent allocations on an empty filesystem was perfectly efficient, since you'd have to go out of your way to make it bad. The real trouble was when the filesystem was used enough for its free space to become fragmented. At that point the allocation strategy was quite poor and you would end up with extremely fragmented large files. IIRC it was really just a naive sequential find first free blocks kind of thing. This aspect is consistent with your memory. I think if we wanted to improve that aspect of linux ntfs support we're better off working from ntfs-3g. Not to mention there are some significant down sides to in-kernel filesystem drivers, they are rather problematic when it comes to containers/namespaces and the security story is a total shit show.

pengaru 6y ago

ntfs-3g is entirely implemented in userspace, and is far more trivial to hack on and test if anyone wanted to improve its file allocation strategies than an in-kernel driver would be. Years ago when working on a commercial virtualization product I had to support initializing sparsely populated ntfs filesystems where only the filesystem metadata was written out, with allocated file extent ranges logged externally and not actually populated in the sparse ntfs filesystem in a file. I ended up deriving a small library for doing this from the libntfs-3g code, I don't think it took more than a weekend to do, the code was quite sane. What I recall from that experience was that the file extent allocations on an empty filesystem was perfectly efficient, since you'd have to go out of your way to make it bad. The real trouble was when the filesystem was used enough for its free space to become fragmented. At that point the allocation strategy was quite poor and you would end up with extremely fragmented large files. IIRC it was really just a naive sequential find first free blocks kind of thing. This aspect is consistent with your memory. I think if we wanted to improve that aspect of linux ntfs support we're better off working from ntfs-3g. Not to mention there are some significant down sides to in-kernel filesystem drivers, they are rather problematic when it comes to containers/namespaces and the security story is a total shit show.