7 ms·

The "popular misconceptions" section seems to have a couple of the author's own misconceptions. * On precision, he notes "almost no filesystems provide that ki

by chungy 8y ago

The "popular misconceptions" section seems to have a couple of the author's own misconceptions. * On precision, he notes "almost no filesystems provide that kind of precision" (nanoseconds), but I would honestly say the exact opposite statement. ext4, xfs, btrfs, ZFS are some of the very common file systems that support this. He cites that his ext4 system only has 10ms granularity, which is most certainly not the default, but likely a result of upgrading from ext2/ext3 to ext4. As an aside, NTFS only has a granularity of 100ns. * It is unclear what he means by "If your system clock jumps from one time to another...". If this is talking about NTP, it's probably accurate. My first reading was "daylight saving" or time zone changes, in which case, everyone uses UTC internally and such changes don't affect the actual mtimes. (You might get strange cases where a file listing regards a file modified at 01:45 to be older than a file modified at 01:20, but if you display in UTC, you can see it's just DST nonsense)

mahkoh 8y ago
He cites that his ext4 system only has 10ms granularity, which is most certainly not the default, but likely a result of upgrading from ext2/ext3 to ext4. What is the granularity of your file system? Mine appears to be 3.33ms.
- chungy 8y ago
  Nanosecond granularity on all the ones I use, which includes ZFS, ext4, and tmpfs.
  - jlokier 8y ago
    Check the timestamps which are actually set on the files. I was surprised and disappointed to find Linux sets mtime to the nearest clock tick (250Hz on my laptop) on filesystems whose documentation says they provide nanosecond timestamps. It's not obvious because the numbers actually stored still have 9 random looking digits. But the chosen mtime values actually go up only on clock ticks. If you're running those filesystems on Linux, try it yourself: (n=1; while [[ $n -le 10000 ]]; do > test$n; n=$((n+1)); done) ls -ltr --full-time You should see the timestamp nanoseconds increment every few files, in batches. If they were truly nanosecond accurate mtimes, they would be different for every file. That's why some of my programs on Linux now set the mtime explicitly with a call to clock_gettime() followed by futimens(), after writing the file. To make sure the timestamps do change each times files are replaced, in case it's more than once inside a 250Hz tick.
delroth 8y ago
You're looking at the 90th percentile, he's looking at the 99th percentile. Claiming things like "everyone uses UTC internally" is obviously wrong when many people just a few years ago were still setting up systems in localtime when dual booting with Windows. Upgrading to ext4 is also not a rare edge case.
- tedunangst 8y ago
  Even if you're using an offset from CMOS time, it's still UTC internally and in the filesystem.
- chungy 8y ago
  Upgrading to ext4 is not extremely rare, but the recommended procedure involves mkfs and copying files over anyway: https://ext4.wiki.kernel.org/index.php/UpgradeToExt4 https://ext4.wiki.kernel.org/index.php/UpgradeToExt4 In-place upgrades do have the potential to leave some non-default options for the final ext4 file system, such as 128B inodes instead of the 256B ones, which is where certain features like reduced timestamp granularity comes in.
  - apenwarr 8y ago
    In any case, my system was installed from scratch quite recently using native ext4. As others have pointed out and as I linked in the article, it’s likely a kernel issue. I assume many people have the same thing.
- kbenson 8y ago
  > Upgrading to ext4 is also not a rare edge case. Ext4 has been stable for over a decade. It's been a default filesystem on many distributions. It was the default on RHEL 6[1] which was first released over 8 years ago, and the default for the ext variants after that. It's been in use in Debian since 6.0/Squeeze or later[2], which was 2011. It's been in use in Ubuntu since 9.10[3], released in late 2009. To be clear, your argument is that it's not a rare edge case to have a filesystem that was originally only in common use as the default variant 6-8 years ago or more for the vast majority of installations, which has persisted and since been upgraded? 1: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/storage_administration_guide/ch-ext4 https://access.redhat.com/documentation/en-us/red_hat_enterp... 2: https://wiki.debian.org/FileSystem https://wiki.debian.org/FileSystem 3: https://wiki.ubuntu.com/KarmicKoala/TechnicalOverview#ext4_by_default https://wiki.ubuntu.com/KarmicKoala/TechnicalOverview#ext4_b...
STRML 8y ago
> It is unclear what he means by "If your system clock jumps from one time to another...". If this is talking about NTP, it's probably accurate. I don't know what he's talking about either. Most popular NTP clients (ntpd, chrony) will try very hard to make sure this never happens by simply slowing down or speeding up time. You don't know what will break if you just gap time like that.
- cyphar 8y ago
  There isn't really a mechanism to slow down or speed up time on Linux (this has become clear with the recent time namespace discussions), you'd have constantly re-set the time to whatever slowed-down clock you are trying to emulate -- which isn't hard but it would result in lost of micro-jumps backwards or forwards which would be more chaotic than just one macro-jump. ntpd and chrony might do it (I'm not sure), but systemd's NTP implementation (which is widely used, even though it does have many other issues -- such as not implementing the spec properly IIRC) does just jump time when you enable NTP on a system where it was disabled. From memory, back when I used ntpd, it did the same thing but I could be mistaken.
  - bogomipz 8y ago
    >"There isn't really a mechanism to slow down or speed up time on Linux (this has become clear with the recent time namespace discussions)" Interesting, might you have any links to these discussions?
    - cyphar 8y ago
      There is an LWN article about it[1], and the discussions are all on LKML[2]. [1]: https://lwn.net/Articles/766089/ https://lwn.net/Articles/766089/ [2]: https://lore.kernel.org/lkml/20180919205037.9574-1-dima@arista.com/ https://lore.kernel.org/lkml/20180919205037.9574-1-dima@aris...
  - caf 8y ago
    Yes there is, and has been for a very long time, specifically to support ntpd. http://man7.org/linux/man-pages/man2/adjtimex.2.html http://man7.org/linux/man-pages/man2/adjtimex.2.html
cozzyd 8y ago
Probably it's because this python implementation uses st.st_mtime (https://github.com/apenwarr/redo/blob/master/state.py#L314 https://github.com/apenwarr/redo/blob/master/state.py#L314) which is a double and therefore is not precise to enough digits for maximum granularity.
- apenwarr 8y ago
  doubles have ~15 decimal digits of granularity. $ python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 >>> 4e9 + 0.000001 4000000000.000001 In any case, I did my testing for the article using the C program linked from the article. The first timestamp in its output gives you a lower bound on your system's granularity: https://apenwarr.ca/log/mmap_test.c https://apenwarr.ca/log/mmap_test.c
  - jlokier 8y ago
    15 decimal digits isn't enough to encode a file's st_mtime seconds and nanoseconds value. I use Perl and found this to be a problem. Like Python, it uses a double for st_mtime, and the nanoseconds value is truncated, so it fails equality tests with nanoseconds recorded by other programs (e.g. in a cache). It even fails equality tests against itself, when timestamp values are serialised to JSON or strings with (say) 6 or 9 digits of precision and back again. Timestamps serialised that way don't round trip reliably due to double truncation.
  - cozzyd 8y ago
    Well, this limits you to ~1 us granularity, but you are right that it's not the limiting factor. Apparently, according to https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond https://stackoverflow.com/questions/14392975/timestamp-accur..., the ext4 driver just uses the cached kernel clock value without the counter correction that gives you a ns-precise value. One could perhaps have an LD_PRELOADED fsync (or whatever) that updates the mtime with clock_gettime() to store it in its full nanosecond precision glory but it's probably not worth the performance penalty. That wouldn't address the mmap issue of course...
BeeOnRope 8y ago
The author links to a post which indicates that while ext4 with 256b inodes might technically store mtime with nanosecond granularity - the call that ext4 to get that time is limited by the scheduler clocktick granularity, aka HZ which is 100, 250 or 300 for most people, hence reported granularities of 10ms, 4ms, 3.33ms, etc.

by chungy 8y ago

mahkoh 8y ago

He cites that his ext4 system only has 10ms granularity, which is most certainly not the default, but likely a result of upgrading from ext2/ext3 to ext4. What is the granularity of your file system? Mine appears to be 3.33ms.

chungy 8y ago
Nanosecond granularity on all the ones I use, which includes ZFS, ext4, and tmpfs.
- jlokier 8y ago
  Check the timestamps which are actually set on the files. I was surprised and disappointed to find Linux sets mtime to the nearest clock tick (250Hz on my laptop) on filesystems whose documentation says they provide nanosecond timestamps. It's not obvious because the numbers actually stored still have 9 random looking digits. But the chosen mtime values actually go up only on clock ticks. If you're running those filesystems on Linux, try it yourself: (n=1; while [[ $n -le 10000 ]]; do > test$n; n=$((n+1)); done) ls -ltr --full-time You should see the timestamp nanoseconds increment every few files, in batches. If they were truly nanosecond accurate mtimes, they would be different for every file. That's why some of my programs on Linux now set the mtime explicitly with a call to clock_gettime() followed by futimens(), after writing the file. To make sure the timestamps do change each times files are replaced, in case it's more than once inside a 250Hz tick.

delroth 8y ago

You're looking at the 90th percentile, he's looking at the 99th percentile. Claiming things like "everyone uses UTC internally" is obviously wrong when many people just a few years ago were still setting up systems in localtime when dual booting with Windows. Upgrading to ext4 is also not a rare edge case.

tedunangst 8y ago
Even if you're using an offset from CMOS time, it's still UTC internally and in the filesystem.
chungy 8y ago
Upgrading to ext4 is not extremely rare, but the recommended procedure involves mkfs and copying files over anyway: https://ext4.wiki.kernel.org/index.php/UpgradeToExt4 https://ext4.wiki.kernel.org/index.php/UpgradeToExt4 In-place upgrades do have the potential to leave some non-default options for the final ext4 file system, such as 128B inodes instead of the 256B ones, which is where certain features like reduced timestamp granularity comes in.
- apenwarr 8y ago
  In any case, my system was installed from scratch quite recently using native ext4. As others have pointed out and as I linked in the article, it’s likely a kernel issue. I assume many people have the same thing.
kbenson 8y ago
> Upgrading to ext4 is also not a rare edge case. Ext4 has been stable for over a decade. It's been a default filesystem on many distributions. It was the default on RHEL 6[1] which was first released over 8 years ago, and the default for the ext variants after that. It's been in use in Debian since 6.0/Squeeze or later[2], which was 2011. It's been in use in Ubuntu since 9.10[3], released in late 2009. To be clear, your argument is that it's not a rare edge case to have a filesystem that was originally only in common use as the default variant 6-8 years ago or more for the vast majority of installations, which has persisted and since been upgraded? 1: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/storage_administration_guide/ch-ext4 https://access.redhat.com/documentation/en-us/red_hat_enterp... 2: https://wiki.debian.org/FileSystem https://wiki.debian.org/FileSystem 3: https://wiki.ubuntu.com/KarmicKoala/TechnicalOverview#ext4_by_default https://wiki.ubuntu.com/KarmicKoala/TechnicalOverview#ext4_b...

STRML 8y ago

> It is unclear what he means by "If your system clock jumps from one time to another...". If this is talking about NTP, it's probably accurate. I don't know what he's talking about either. Most popular NTP clients (ntpd, chrony) will try very hard to make sure this never happens by simply slowing down or speeding up time. You don't know what will break if you just gap time like that.

cyphar 8y ago
There isn't really a mechanism to slow down or speed up time on Linux (this has become clear with the recent time namespace discussions), you'd have constantly re-set the time to whatever slowed-down clock you are trying to emulate -- which isn't hard but it would result in lost of micro-jumps backwards or forwards which would be more chaotic than just one macro-jump. ntpd and chrony might do it (I'm not sure), but systemd's NTP implementation (which is widely used, even though it does have many other issues -- such as not implementing the spec properly IIRC) does just jump time when you enable NTP on a system where it was disabled. From memory, back when I used ntpd, it did the same thing but I could be mistaken.
- bogomipz 8y ago
  >"There isn't really a mechanism to slow down or speed up time on Linux (this has become clear with the recent time namespace discussions)" Interesting, might you have any links to these discussions?
  - cyphar 8y ago
    There is an LWN article about it[1], and the discussions are all on LKML[2]. [1]: https://lwn.net/Articles/766089/ https://lwn.net/Articles/766089/ [2]: https://lore.kernel.org/lkml/20180919205037.9574-1-dima@arista.com/ https://lore.kernel.org/lkml/20180919205037.9574-1-dima@aris...
- caf 8y ago
  Yes there is, and has been for a very long time, specifically to support ntpd. http://man7.org/linux/man-pages/man2/adjtimex.2.html http://man7.org/linux/man-pages/man2/adjtimex.2.html

cozzyd 8y ago

Probably it's because this python implementation uses st.st_mtime (https://github.com/apenwarr/redo/blob/master/state.py#L314 https://github.com/apenwarr/redo/blob/master/state.py#L314) which is a double and therefore is not precise to enough digits for maximum granularity.

apenwarr 8y ago
doubles have ~15 decimal digits of granularity. $ python Python 2.7.13 (default, Sep 26 2018, 18:42:22) [GCC 6.3.0 20170516] on linux2 >>> 4e9 + 0.000001 4000000000.000001 In any case, I did my testing for the article using the C program linked from the article. The first timestamp in its output gives you a lower bound on your system's granularity: https://apenwarr.ca/log/mmap_test.c https://apenwarr.ca/log/mmap_test.c
- jlokier 8y ago
  15 decimal digits isn't enough to encode a file's st_mtime seconds and nanoseconds value. I use Perl and found this to be a problem. Like Python, it uses a double for st_mtime, and the nanoseconds value is truncated, so it fails equality tests with nanoseconds recorded by other programs (e.g. in a cache). It even fails equality tests against itself, when timestamp values are serialised to JSON or strings with (say) 6 or 9 digits of precision and back again. Timestamps serialised that way don't round trip reliably due to double truncation.
- cozzyd 8y ago
  Well, this limits you to ~1 us granularity, but you are right that it's not the limiting factor. Apparently, according to https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond https://stackoverflow.com/questions/14392975/timestamp-accur..., the ext4 driver just uses the cached kernel clock value without the counter correction that gives you a ns-precise value. One could perhaps have an LD_PRELOADED fsync (or whatever) that updates the mtime with clock_gettime() to store it in its full nanosecond precision glory but it's probably not worth the performance penalty. That wouldn't address the mmap issue of course...

BeeOnRope 8y ago

The author links to a post which indicates that while ext4 with 256b inodes might technically store mtime with nanosecond granularity - the call that ext4 to get that time is limited by the scheduler clocktick granularity, aka HZ which is 100, 250 or 300 for most people, hence reported granularities of 10ms, 4ms, 3.33ms, etc.