4 ms·

One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to d

One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. The default is 5 seconds, but I usually increase mine to 20. When writing a lot of data, it'll get flushed to disk more often, so this timer is only for when you're writing small amounts of data like the test you just described. > like might happen for some programs that are writing logs fairly continuously On Linux, I think journald would be aggregating your logs from multiple services so at least you wouldn't be incurring that cost on a per-program basis. On FreeBSD with syslog we're doomed to separate log files. > over a 10 year span that same program running continuously would result in about 100 TB of writes being done to the drive which is about a quarter of what my SSD is rated for I sure hope I've upgraded SSDs by the year 2065.

dizhn 2y ago
>I sure hope I've upgraded SSDs by the year 2065. My mind jumped at that too when I first read parent's comment. But presumably he's writing other files to disk too. Not just that one file. :)
- JZerf 2y ago
  > But presumably he's writing other files to disk too. Not just that one file. Yes, there will be much more going on than the simple test I was doing. The server will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps.
JZerf 2y ago
> One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. I don't believe that zfs_txg_timeout setting would make much of a difference for the test I described where I was doing synchronous writes. > On Linux, I think journald would be aggregating your logs from multiple services so at least you wouldn't be incurring that cost on a per-program basis. The server I'm setting up will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps. Some of the logging could be aggregated but there will be multiple types of I/O (various types of databases, app updates, file server, etc...) and I wanted to get an idea of how much file system overhead there might be in a worst case kind of scenario. > I sure hope I've upgraded SSDs by the year 2065. Since I'll be running a lot of stuff on the server, I'll probably have quite a bit more writing going on than the test I described so if I used ZFS I believe the SSD could reach its rated endurance in just several years.

dizhn 2y ago

>I sure hope I've upgraded SSDs by the year 2065. My mind jumped at that too when I first read parent's comment. But presumably he's writing other files to disk too. Not just that one file. :)

JZerf 2y ago
> But presumably he's writing other files to disk too. Not just that one file. Yes, there will be much more going on than the simple test I was doing. The server will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps.

JZerf 2y ago

> One knob you could change that should radically alter that is zfs_txg_timeout which is how many seconds ZFS will accumulate writes before flushing them out to disk. I don't believe that zfs_txg_timeout setting would make much of a difference for the test I described where I was doing synchronous writes. > On Linux, I think journald would be aggregating your logs from multiple services so at least you wouldn't be incurring that cost on a per-program basis. The server I'm setting up will be hosting several VMs running a mix of OSes and distros and running many types types of services and apps. Some of the logging could be aggregated but there will be multiple types of I/O (various types of databases, app updates, file server, etc...) and I wanted to get an idea of how much file system overhead there might be in a worst case kind of scenario. > I sure hope I've upgraded SSDs by the year 2065. Since I'll be running a lot of stuff on the server, I'll probably have quite a bit more writing going on than the test I described so if I used ZFS I believe the SSD could reach its rated endurance in just several years.