9 ms·

I've quickly skimmed the discussion on fedora-devel regarding btrfs. I wondered mainly how they'd handle the various cases where btrfs does not work well, e.g.

by bkor 6y ago

I've quickly skimmed the discussion on fedora-devel regarding btrfs. I wondered mainly how they'd handle the various cases where btrfs does not work well, e.g. files that change often inline (databases, VMs, etc). Apparently an application can tell to treat those files differently. So it's basically a matter of fixing various software to work nicely with btrfs as well as any similar filesystem. As mentioned in the thread, openSUSE already uses btrfs for loads of years. I do wonder why it isn't more supported by upstream software (postgres, VM things, etc). I was also surprised how good the discussion was on fedora-devel. It seems that people didn't break into camps of "over my dead body", and "if we do not do this I'll leave", etc. It seemed more like a "this is my really thought out proposal, but is it actually feasible or did I forget anything". Then people raised problems, sometimes the proposer(s) already had a solution, sometimes it was something they didn't think of. What's nice is that despite seeing various problems everyone seemed to understand that people are trying to improve things. I didn't read fedora-devel for many years so that was a welcome change over the past.

vetinari 6y ago
> I wondered mainly how they'd handle the various cases where btrfs does not work well, e.g. files that change often inline (databases, VMs, etc). It least for default libvirt locations fedora disables CoW (chattr +C). No change in software needed.
- prussian 6y ago
  This is what I was thinking as well. It just means packagers have to be mindful of what kind of files their maintained software makes and to appropriately carry these metadata changes where they are needed. I don't really see an issue with this mindset.
nix23 6y ago
>databases, VMs, etc You should disable CoW and Caching on the filesystem/-set where VM and Databases-files reside, but that counts for ZFS as well...well for all CoW-FS's in fact.
- schoolornot 6y ago
  Can you explain the reasoning for this?
  - wtallis 6y ago
    On a copy-on-write filesystem, modification of part of a file means fragmenting it, which carries a large performance penalty. Disabling the CoW behavior means you either cannot take snapshots of the file, or that snapshots need to duplicate the file in its entirety, but it allows for quick modification of the file in-place without fragmenting.
    - kim0 6y ago
      Is that assuming HDDs not SSDs ?
- pezezin 6y ago
  I understand disabling CoW for databases, as the DB itself will manage it, but why disable caching?
  - Uberphallus 6y ago
    Depends on the RDBMS, but for big ones it's common practice to tune[0] them to allocate a big chunk of memory (or all of it) and let them manage caching by themselves. After all, they know better about the structure of the db files than the OS. [0]https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serv... Relevant parameters: effective_cache_size/shared_buffers/work_mem
    - chasil 6y ago
      Oracle offers a document for running their database on ZFS. https://www.oracle.com/technetwork/server-storage/solaris10/config-solaris-zfs-wp-167894.pdf https://www.oracle.com/technetwork/server-storage/solaris10/... "The number one rule for setting up an Oracle database on ZFS is to set ZFS recordsize equal to the database block size for the file systems that contain the Oracle data files." "Starting with Oracle Solaris 11.1, blocks that are referenced from either the snapshot or from the database image actually share ZFS cache buffers for a reduced memory footprint." "Taking a snapshot of a database can involve placing the database in hot-backup mode just for the instant necessary to capture the ZFS snapshot. The database can then be returned to normal operation." "A clone is a full read/write accessible fork of the original database. Therefore, by using either the primary or a replicated storage system, you can cheaply create hundreds of copies of your up-to-date database to run test and development workloads under real data conditions." They also allow compression of the archived (intent) logs.
throw0101a 6y ago
> So it's basically a matter of fixing various software to work nicely with btrfs as well as any similar filesystem. I do wonder why it isn't more supported by upstream software (postgres, VM things, etc). The concept of an application 'supporting a (specific) file system' sounds slightly ridiculous to me. Why is Btrfs so special that apps should be rewritten for it? The always-compared ZFS seems to work just fine without alterations being necessary, so what make Btrfs worthy of extra considerations? At most perhaps copy-on-write (COW) file systems as a general class could perhaps be considered, as that may help with things like shingled drives and SSDs (i.e., zoned drives may be aligned with COW FSes).
- ghaff 6y ago
  Because Fedora, among other distributions, will not include ZFS for licensing reasons. (Leaving aside any other discussion of the relative merits of ZFS and Btrfs.)
  - Uberphallus 6y ago
    ZFS has a good share, if not 100%, of the same problems as Btrfs, and in the end of the day it's always matter of leaving files/directories/volumes without CoW, which is hardly a software rewrite as the grandparent insinuated, and more of a packaging issue.
    - nix23 6y ago
      >ZFS has a good share, if not 100% ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3) >leaving files/directories/volumes without CoW No...just NO, just for a special Cases like a Database.
      - Uberphallus 6y ago
        > ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3) The thread was about where CoW is problematic, both ZFS and Btrfs are identically problematic in this sense. > No...just NO, just for a special Cases like a Database. Like... for what we were talking about.

by bkor 6y ago

vetinari 6y ago

> I wondered mainly how they'd handle the various cases where btrfs does not work well, e.g. files that change often inline (databases, VMs, etc). It least for default libvirt locations fedora disables CoW (chattr +C). No change in software needed.

prussian 6y ago
This is what I was thinking as well. It just means packagers have to be mindful of what kind of files their maintained software makes and to appropriately carry these metadata changes where they are needed. I don't really see an issue with this mindset.

nix23 6y ago

>databases, VMs, etc You should disable CoW and Caching on the filesystem/-set where VM and Databases-files reside, but that counts for ZFS as well...well for all CoW-FS's in fact.

schoolornot 6y ago
Can you explain the reasoning for this?
- wtallis 6y ago
  On a copy-on-write filesystem, modification of part of a file means fragmenting it, which carries a large performance penalty. Disabling the CoW behavior means you either cannot take snapshots of the file, or that snapshots need to duplicate the file in its entirety, but it allows for quick modification of the file in-place without fragmenting.
  - kim0 6y ago
    Is that assuming HDDs not SSDs ?
pezezin 6y ago
I understand disabling CoW for databases, as the DB itself will manage it, but why disable caching?
- Uberphallus 6y ago
  Depends on the RDBMS, but for big ones it's common practice to tune[0] them to allocate a big chunk of memory (or all of it) and let them manage caching by themselves. After all, they know better about the structure of the db files than the OS. [0]https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server https://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Serv... Relevant parameters: effective_cache_size/shared_buffers/work_mem
  - chasil 6y ago
    Oracle offers a document for running their database on ZFS. https://www.oracle.com/technetwork/server-storage/solaris10/config-solaris-zfs-wp-167894.pdf https://www.oracle.com/technetwork/server-storage/solaris10/... "The number one rule for setting up an Oracle database on ZFS is to set ZFS recordsize equal to the database block size for the file systems that contain the Oracle data files." "Starting with Oracle Solaris 11.1, blocks that are referenced from either the snapshot or from the database image actually share ZFS cache buffers for a reduced memory footprint." "Taking a snapshot of a database can involve placing the database in hot-backup mode just for the instant necessary to capture the ZFS snapshot. The database can then be returned to normal operation." "A clone is a full read/write accessible fork of the original database. Therefore, by using either the primary or a replicated storage system, you can cheaply create hundreds of copies of your up-to-date database to run test and development workloads under real data conditions." They also allow compression of the archived (intent) logs.

throw0101a 6y ago

> So it's basically a matter of fixing various software to work nicely with btrfs as well as any similar filesystem. I do wonder why it isn't more supported by upstream software (postgres, VM things, etc). The concept of an application 'supporting a (specific) file system' sounds slightly ridiculous to me. Why is Btrfs so special that apps should be rewritten for it? The always-compared ZFS seems to work just fine without alterations being necessary, so what make Btrfs worthy of extra considerations? At most perhaps copy-on-write (COW) file systems as a general class could perhaps be considered, as that may help with things like shingled drives and SSDs (i.e., zoned drives may be aligned with COW FSes).

ghaff 6y ago
Because Fedora, among other distributions, will not include ZFS for licensing reasons. (Leaving aside any other discussion of the relative merits of ZFS and Btrfs.)
- Uberphallus 6y ago
  ZFS has a good share, if not 100%, of the same problems as Btrfs, and in the end of the day it's always matter of leaving files/directories/volumes without CoW, which is hardly a software rewrite as the grandparent insinuated, and more of a packaging issue.
  - nix23 6y ago
    >ZFS has a good share, if not 100% ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3) >leaving files/directories/volumes without CoW No...just NO, just for a special Cases like a Database.
    - Uberphallus 6y ago
      > ZFS has no Raid5 write-hole...no traditional Raid problems at all (z1/2/3) The thread was about where CoW is problematic, both ZFS and Btrfs are identically problematic in this sense. > No...just NO, just for a special Cases like a Database. Like... for what we were talking about.