4 ms·

I suspect the blog author did not understand this (based on the content) - as a Linode user myself, I just had a look at one of my VMs and they install with the

I suspect the blog author did not understand this (based on the content) - as a Linode user myself, I just had a look at one of my VMs and they install with the regular 5% reserved space (ext4/Debian).

birdyrooster 6y ago
Funny because I have always tune2fs -m1 or tune2fs -m0 because the reserved space was never supposed to scale linearly with hard drive capacities and is not useful to userspace in anyway. Have never had any issues and been doing it for decades in commercial applications. In some cases, where you probably shouldn't be using ext3/4 anyways, we are talking about reclaiming TBs of reserved space. It's important to note that mkfs doesn't care if you are formatting the root partition or a data volume partition, it will still reserve space for the kernel.
- stonesweep 6y ago
  Fun trivia: you can actually set per-device settings in /etc/mke2fs.conf with all sorts of alternate defaults. https://man7.org/linux/man-pages/man5/mke2fs.conf.5.html https://man7.org/linux/man-pages/man5/mke2fs.conf.5.html
  - anonymousiam 6y ago
    More fun trivia. The original(?) man page for tunefs had an "easter egg": https://unixhistory.livejournal.com/1808.html?nojs=1 https://unixhistory.livejournal.com/1808.html?nojs=1
- tytso 6y ago
  If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. So that's why having a fixed percentage even for massively big disks still makes sense. Disk space is cheap enough that even 5% of a 14TB disk is really not that much money --- and if you see bad performance, and then have to pay $$$ to switch to an SSD, maybe it would have been much cheaper to use a HDD with a larger free space reserve....
  - axiolite 6y ago
    > If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. Not true, I've checked. I have plenty of Linux ext3 servers running for many years that routinely drop down to 1% free space for extended periods before being cleaned-up, which still have essentially zero fragmentation. You can create plenty of 10MB files on a multi-terabyte volume that has under 1% free space, as that's still tens of gigabytes to work with. Obviously at some point you'll hit a severe problem and it's best to avoid taking a chance, but a fixed percentage really isn't the best measurement to tell you where that horizon will be.
    - ericbarrett 6y ago
      It's true, but it's more true for some file systems than others. When you write a file larger than the contiguous available space after its starting point, a file system must break the file into "extents" (chunks). The less space available, the smaller the extents tend to be, and the more fragmentation you will impose for continued writes. It's just math. Different file systems have wildly different strategies and data structures behind this process, however. Some drop to their knees over 92-93%. Some can write to the last byte with reasonable efficiency—but it'll never be as fast as when it was empty. Copy-on-write systems like ZFS tend to do poorly under near-full conditions.
      - axiolite 6y ago
        > It's true No, still completely untrue. > When you write a file larger than the contiguous available space after its starting point, a file system must break the file And why would having a 10TB file system force you to write 100GByte files, whereas a 10GB file system would write 100MByte files instead? Because that's the topic you're responding to... GP said it doesn't matter whether it is a 10GB file system or a 10TB file system, it "gets badly fragmented" when you get to "99% full".
    - tytso 6y ago
      Like most things, it depends on your workload. If all of the files are written all at once when they created (e.g., no slow append workloads) and the files are all the same size, then that's a very "friendly" workload that will be much less likely to suffer fragmentation. But if you are creating and deleting files that have a large range of sizes, and some files grow gradually over time, then free space fragmentation will tend to occur much more quickly.
- boring_twenties 6y ago
  It's reserved for the superuser, not the kernel
- tryauuum 6y ago
  Had anyone succeed in looking up the motivation behind the reserved space idea in ext filesystems, e.g a commit message from the time this feature was introduced? I've tried but failed miserably, got lost in many different git repositories. One fun fact I learned though - one of the reasons is "for quota file". Since ext stores quota information in an ordinary file, there could be an issue where user fills up a disk but the information about it isn't written in quota file due to lack of space.

birdyrooster 6y ago

Funny because I have always tune2fs -m1 or tune2fs -m0 because the reserved space was never supposed to scale linearly with hard drive capacities and is not useful to userspace in anyway. Have never had any issues and been doing it for decades in commercial applications. In some cases, where you probably shouldn't be using ext3/4 anyways, we are talking about reclaiming TBs of reserved space. It's important to note that mkfs doesn't care if you are formatting the root partition or a data volume partition, it will still reserve space for the kernel.

stonesweep 6y ago
Fun trivia: you can actually set per-device settings in /etc/mke2fs.conf with all sorts of alternate defaults. https://man7.org/linux/man-pages/man5/mke2fs.conf.5.html https://man7.org/linux/man-pages/man5/mke2fs.conf.5.html
- anonymousiam 6y ago
  More fun trivia. The original(?) man page for tunefs had an "easter egg": https://unixhistory.livejournal.com/1808.html?nojs=1 https://unixhistory.livejournal.com/1808.html?nojs=1
tytso 6y ago
If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. So that's why having a fixed percentage even for massively big disks still makes sense. Disk space is cheap enough that even 5% of a 14TB disk is really not that much money --- and if you see bad performance, and then have to pay $$$ to switch to an SSD, maybe it would have been much cheaper to use a HDD with a larger free space reserve....
- axiolite 6y ago
  > If you try to use a file system to 99% full --- and it doesn't matter whether it is a 10GB file system or a 10TB file system, you will see significant performance penalties as the file system gets badly fragmented. Not true, I've checked. I have plenty of Linux ext3 servers running for many years that routinely drop down to 1% free space for extended periods before being cleaned-up, which still have essentially zero fragmentation. You can create plenty of 10MB files on a multi-terabyte volume that has under 1% free space, as that's still tens of gigabytes to work with. Obviously at some point you'll hit a severe problem and it's best to avoid taking a chance, but a fixed percentage really isn't the best measurement to tell you where that horizon will be.
  - ericbarrett 6y ago
    It's true, but it's more true for some file systems than others. When you write a file larger than the contiguous available space after its starting point, a file system must break the file into "extents" (chunks). The less space available, the smaller the extents tend to be, and the more fragmentation you will impose for continued writes. It's just math. Different file systems have wildly different strategies and data structures behind this process, however. Some drop to their knees over 92-93%. Some can write to the last byte with reasonable efficiency—but it'll never be as fast as when it was empty. Copy-on-write systems like ZFS tend to do poorly under near-full conditions.
    - axiolite 6y ago
      > It's true No, still completely untrue. > When you write a file larger than the contiguous available space after its starting point, a file system must break the file And why would having a 10TB file system force you to write 100GByte files, whereas a 10GB file system would write 100MByte files instead? Because that's the topic you're responding to... GP said it doesn't matter whether it is a 10GB file system or a 10TB file system, it "gets badly fragmented" when you get to "99% full".
  - tytso 6y ago
    Like most things, it depends on your workload. If all of the files are written all at once when they created (e.g., no slow append workloads) and the files are all the same size, then that's a very "friendly" workload that will be much less likely to suffer fragmentation. But if you are creating and deleting files that have a large range of sizes, and some files grow gradually over time, then free space fragmentation will tend to occur much more quickly.
boring_twenties 6y ago
It's reserved for the superuser, not the kernel
tryauuum 6y ago
Had anyone succeed in looking up the motivation behind the reserved space idea in ext filesystems, e.g a commit message from the time this feature was introduced? I've tried but failed miserably, got lost in many different git repositories. One fun fact I learned though - one of the reasons is "for quota file". Since ext stores quota information in an ordinary file, there could be an issue where user fills up a disk but the information about it isn't written in quota file due to lack of space.