5 ms·

Ext4 does support larger than 16tb filesystems. df -h | grep md127 ; mount | grep md127 /dev/md127 26T 14T 12T 55% /home/data /dev/md127 on /home/

by edgan 11y ago

Ext4 does support larger than 16tb filesystems. df -h | grep md127 ; mount | grep md127 /dev/md127 26T 14T 12T 55% /home/data /dev/md127 on /home/data type ext4 (rw,relatime,stripe=1792,data=ordered) Ext3 was 32k, but ext4 is 65k subfolder limit. I have run into this one before. The real solution is to do directory hashing when you get this crazy with tons of subfolders.

istvan__ 11y ago
I think there is a difference between what is supported and what is possible. https://access.redhat.com/solutions/1532 https://access.redhat.com/solutions/1532
- simoncion 11y ago
  RedHat has tested EXT4 on partitions larger than 16TB. From your linked note: "EXT4 [on RHEL7] [Cerfified max FS size:] 50TB [Theoretical max FS size:] (1EB)" Moreover, RedHat's lack of Enterprise Support for a configuration doesn't mean that that configuration is a bad idea, or is doomed to failure. :)
  - istvan__ 11y ago
    This is exactly what my comment was about. :)
    - simoncion 11y ago
      Oh, I see. You were remarking on edgan's use of the phrase "Ext4 does support". So, firstly, both Red Hat and ext4 support FSs larger than 16TB. So, yeah. Secondly, I mean, Ted Ts'o works for Google, rather than Red Hat, so it's not like Red Hat has any special insight into the inner workings of ext4. RHEL's lack of "support" for ext4 FSs large than a certain size wouldn't make me wary of using FSs larger than that size. Ts'o & co. seem to be pretty careful, so I would expect that the worst that would happen would be performance issues.
      - istvan__ 11y ago
        I am wondering what Google uses nowadays. Regarding Tso & co, I am not so optimistic. How they communicate on the mailing list is more like: "Look we wrote this, it runs on my laptop fine it might work for other workloads too". There is no extensive test coverage run by against ext4 (at least I am not aware of it). This is not the first data loss bug in the ext4 codebase. Few years from now when enough users start to use it an get rid off these serious bugs from the code, I might even consider POCing it again. http://www.phoronix.com/scan.php?page=news_item&px=MTIxNDQ http://www.phoronix.com/scan.php?page=news_item&px=MTIxNDQ
        simoncion 11y ago
        I'm somewhat certain that -despite the name- xfstests [0] is the canonical filesystem test suite these days. (At least, I've heard the btrfs devs concerned about passing its tests and adding new tests in it to check corner cases in btrfs. I also remember hearing the ext4 devs mention xfstest.) I read that mailing list tone as something more like "This software is offered without warranty and might shave your dog and weld your toilet seat down.". I mean, folks on the LKML talk in that manner about their patches, too. Also, even XFS has had severe data loss bugs in the distant past [1] and much more recently. [2] [0] http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=summary http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.gi... [1] https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.15/+bug/37435 https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.1... (Note in particular the last comment that describes this as "not a bug, but a feature!".) [2] http://thread.gmane.org/gmane.comp.file-systems.xfs.general/45648 http://thread.gmane.org/gmane.comp.file-systems.xfs.general/... (fixed in RHEL in 2013(!))

by edgan 11y ago

istvan__ 11y ago

I think there is a difference between what is supported and what is possible. https://access.redhat.com/solutions/1532 https://access.redhat.com/solutions/1532

simoncion 11y ago
RedHat has tested EXT4 on partitions larger than 16TB. From your linked note: "EXT4 [on RHEL7] [Cerfified max FS size:] 50TB [Theoretical max FS size:] (1EB)" Moreover, RedHat's lack of Enterprise Support for a configuration doesn't mean that that configuration is a bad idea, or is doomed to failure. :)
- istvan__ 11y ago
  This is exactly what my comment was about. :)
  - simoncion 11y ago
    Oh, I see. You were remarking on edgan's use of the phrase "Ext4 does support". So, firstly, both Red Hat and ext4 support FSs larger than 16TB. So, yeah. Secondly, I mean, Ted Ts'o works for Google, rather than Red Hat, so it's not like Red Hat has any special insight into the inner workings of ext4. RHEL's lack of "support" for ext4 FSs large than a certain size wouldn't make me wary of using FSs larger than that size. Ts'o & co. seem to be pretty careful, so I would expect that the worst that would happen would be performance issues.
    - istvan__ 11y ago
      I am wondering what Google uses nowadays. Regarding Tso & co, I am not so optimistic. How they communicate on the mailing list is more like: "Look we wrote this, it runs on my laptop fine it might work for other workloads too". There is no extensive test coverage run by against ext4 (at least I am not aware of it). This is not the first data loss bug in the ext4 codebase. Few years from now when enough users start to use it an get rid off these serious bugs from the code, I might even consider POCing it again. http://www.phoronix.com/scan.php?page=news_item&px=MTIxNDQ http://www.phoronix.com/scan.php?page=news_item&px=MTIxNDQ
      - simoncion 11y ago
        I'm somewhat certain that -despite the name- xfstests [0] is the canonical filesystem test suite these days. (At least, I've heard the btrfs devs concerned about passing its tests and adding new tests in it to check corner cases in btrfs. I also remember hearing the ext4 devs mention xfstest.) I read that mailing list tone as something more like "This software is offered without warranty and might shave your dog and weld your toilet seat down.". I mean, folks on the LKML talk in that manner about their patches, too. Also, even XFS has had severe data loss bugs in the distant past [1] and much more recently. [2] [0] http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.git;a=summary http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/cmds/xfstests.gi... [1] https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.15/+bug/37435 https://bugs.launchpad.net/ubuntu/+source/linux-source-2.6.1... (Note in particular the last comment that describes this as "not a bug, but a feature!".) [2] http://thread.gmane.org/gmane.comp.file-systems.xfs.general/45648 http://thread.gmane.org/gmane.comp.file-systems.xfs.general/... (fixed in RHEL in 2013(!))