3 ms·
> But it's equally naive to state that it's a 'fundamental issue' that writing a bunch of text to disk requires a lot of I/O, and that compression doesn't help.
by wtallis 9y ago
> But it's equally naive to state that it's a 'fundamental issue' that writing a bunch of text to disk requires a lot of I/O, and that compression doesn't help.
That wasn't stated anywhere in this thread.
> He didn't just say filesystem smarts were redundant with controller smarts, he said no data layout could avoid the problem. That's a big misstatement of the problem.
You're imagining the conversation to be about a different problem than the one that was explicitly referred to. The fundamental problem under discussion is that random writes will thrash any data structure. You might be able to convert a stream of random writes to sequential writes on the first pass of filling the disk, but once you have a data set on disk and are updating it randomly (which is what almost all real-world random writes are), nothing can prevent the fragmentation and need for garbage collection. Doing it in the filesystem layer won't magically be far more effective than what's already being done at the drive level.
- Dylan16807 9y ago> That wasn't stated anywhere in this thread. It's an extension to your "double zipping" analogy. > nothing can prevent the fragmentation and need for garbage collection Nothing prevents fragmentation entirely but these tactics can decrease the write amplification by an enormous amount. If the initial comment was only saying that filesystems can't improve on what drives already do, it was accidentally worded in a very misleading way. And are you sure filesystems can't arrange writes to make it more effective?