4 ms·

Tonight's rabbit hole: time math and 32 bit longs

74 points by zdw 4y ago

4pkjai 4y ago
I’ll admit I couldn’t really follow the blog post. I’ve always thought it’d be fun to do some research and understand a post like this, then blog about what I had to learn to understand the post. Not sure if that’s “bad form” or not
- h2odragon 4y ago
  Sounds like a great idea, not "bad form" at all. I think it would be a great help to others in the position you're in, as well as worth doing just for your own self edification, without publishing it. This may help get you started: https://en.wikipedia.org/wiki/Year_2038_problem https://en.wikipedia.org/wiki/Year_2038_problem
- jay-barronville 4y ago
  Ask ChatGPT to explain it to you.
  - cratermoon 4y ago
    There's no denying that ChatGPT would do better than some HN commenters.
- 0x0 4y ago
  This blog post probably makes more sense if you read the earlier blog post first, about the 32bit time_t doomsday coming in year 2038. http://rachelbythebay.com/w/2023/01/19/time/ http://rachelbythebay.com/w/2023/01/19/time/
  - jwilk 4y ago
    Discussed on HN 4 days ago: https://news.ycombinator.com/item?id=34496287 https://news.ycombinator.com/item?id=34496287 (>130 comments)
- everybodyknows 4y ago
  This may be enough to make the Perl-style regexes comprehensible: https://perldoc.perl.org/perlre#Character-Classes-and-other-Special-Escapes https://perldoc.perl.org/perlre#Character-Classes-and-other-...
raldi 4y ago
$domain =~ s/\.\s+$//; This will chop off a trailing dot and any white space that follows but not a trailing dot without trailing whitespace. Is that really the intent? Also, it will turn `domain.com.. ` into `domain.com.` $domain =~ s/[^A-Za-z0-9.]+/-/g; Won’t this change my—-domain.com into my-domain.com ? Double dashes are particularly common in internationalized domain names (https://en.wikipedia.org/wiki/Internationalized_domain_name#Example_of_IDNA_encoding https://en.wikipedia.org/wiki/Internationalized_domain_name#...), and this mangles perfectly-valid ones.
- arjvik 4y ago
  I've got no idea why either regex would be useful, can anyone come up with a reason other than a bad attempt to sanitize input?
raldi 4y ago
When I run that program on my Mac, I don't encounter the bug: 2147440447 -> 24855 2147440448 -> 24855
- nebulous1 4y ago
  Make sure you're compiling with -m32
  - astrange 4y ago
    No such thing on Macs anymore. You'd need qemu.
nebulous1 4y ago
> return (long) (t + DAY / 2) / DAY; should have been > return (long) ((t + DAY / 2) / DAY); ?
- hermitdev 4y ago
  No, the problem is using the wrong data type: "long". On 32-bit systems, Linux defines long to be 32-bits,vs 64 bits on 64-bit. As another example of how this could be broken: use this code on Windows, either 32 or 64 bit and the code is completely broken because longs are 32-bits regardless of whether the OS is 32 or 64 bit capable. Use time_t as that's the correct type and will do the right thing assuming your C standard lib is up to date.
  - nebulous1 4y ago
    Well, the problem is that it is casting (t + DAY/2) to a long before it divides by DAY, not after. The whole expression ((t + DAY/2) / DAY) will not overflow 32 bits in any reasonable timeframe, but (t + DAY / 2) will.
    - Arnavion 4y ago
      https://github.com/shadow-maint/shadow/commit/815ffb7d3e0a8e1d78886dded65a5c6afd22c94e#diff-9ade509d0a1eeb8cd0b2638ddf8ddb2bd65a8dd3a901c829fcac29c12cd42588R81 https://github.com/shadow-maint/shadow/commit/815ffb7d3e0a8e... Amusing that it was broken by an attempt to make it clearer.
zokier 4y ago
Tangentially related, but I find it somewhat baffling that at some point people thought that using data types with unknown ranges is viable way of producing robust portable programs. It took until C99 to get basic int32_t etc, before that you were mostly relying on bunch of ifdefs to get portable results. The whole mindset for things like short/int/long must have been just so very different than what I'm used to.
- jodoherty 4y ago
  It doesn't make sense in the context of today's relatively uniform hardware, but I think the goal wasn't so much portably correct code, but portably efficient code that avoided the need to use assembly language for everything. The first version of UNIX was written in assembly language for an 18-bit machine. C was invented as part of rewriting UNIX to work on a 16-bit machine and who knows what to expect after that. Machine architectures were a lot less uniform back in the day, and most of those minicomputers are easily beat by some of today's 20 cent microcontrollers. Given that context, being able to loosely define relative data type sizes based on how efficiently they might be represented on the machine might make more sense. But you're right, that's quite alien to where we are today. By the 1980s, Ada was already making the right call of allowing the developer to specify the behavior and exact range of modular integer types for correctness and letting the compiler optimize things for efficiency or performance as needed. It'd be nice to see that approach become more popular.
johnklos 4y ago
This is why people shouldn't write their own time functions (in the same vein as not writing their own encryption code). If it's not used in a public library by many, many other things, issues like these might not be discovered until it's too late, or until Rachel discovers them (but Rachel can't be expected to check all the things).
- aftbit 4y ago
  By extension of that logic, engineers basically shouldn't write anything except very high level application glue code, and only that as a last resort. Rely on libraries made by people who are domain experts or at least have a lot of eyeballs and users, because _every_ problem domain has a lot of ugly warts and major issues. People warn about time and encryption specifically because these issues can be subtle and take years to be exposed, but the same is true for lots of others as well. Don't write your own parsers, network libraries, locking primitives, data structures, filesystems, databases, etc. Except of course, sometimes you have to.
  - xboxnolifes 4y ago
    The insight is to not write critical, bug prone code if you don't have to. Encryption, time, primitives, etc., are things that many people need and use and already exist. If the solution you need doesn't already exist, of course you need to write your own.
astrange 4y ago
> 2038-01-18 15:14:08Z is when that code will start returning negative numbers. That'll be fun and interesting. Ah but it won't, because signed integer overflow is undefined in C. It will make demons fly out your nose. (gcc may make it return negative numbers, but clang may do something else.)
RickHull 4y ago
Somewhat related, a small ruby project of mine to read input events from the linux kernel. https://github.com/rickhull/device_input https://github.com/rickhull/device_input The README goes in some depth about kernel events and timestamps and time implementations on 32-bit vs 64-bit systems.

74 points by zdw 4y ago

I’ll admit I couldn’t really follow the blog post. I’ve always thought it’d be fun to do some research and understand a post like this, then blog about what I had to learn to understand the post. Not sure if that’s “bad form” or not

h2odragon 4y ago
Sounds like a great idea, not "bad form" at all. I think it would be a great help to others in the position you're in, as well as worth doing just for your own self edification, without publishing it. This may help get you started: https://en.wikipedia.org/wiki/Year_2038_problem https://en.wikipedia.org/wiki/Year_2038_problem
jay-barronville 4y ago
Ask ChatGPT to explain it to you.
- cratermoon 4y ago
  There's no denying that ChatGPT would do better than some HN commenters.
0x0 4y ago
This blog post probably makes more sense if you read the earlier blog post first, about the 32bit time_t doomsday coming in year 2038. http://rachelbythebay.com/w/2023/01/19/time/ http://rachelbythebay.com/w/2023/01/19/time/
- jwilk 4y ago
  Discussed on HN 4 days ago: https://news.ycombinator.com/item?id=34496287 https://news.ycombinator.com/item?id=34496287 (>130 comments)
everybodyknows 4y ago
This may be enough to make the Perl-style regexes comprehensible: https://perldoc.perl.org/perlre#Character-Classes-and-other-Special-Escapes https://perldoc.perl.org/perlre#Character-Classes-and-other-...

raldi 4y ago

$domain =~ s/\.\s+$//; This will chop off a trailing dot and any white space that follows but not a trailing dot without trailing whitespace. Is that really the intent? Also, it will turn `domain.com.. ` into `domain.com.` $domain =~ s/[^A-Za-z0-9.]+/-/g; Won’t this change my—-domain.com into my-domain.com ? Double dashes are particularly common in internationalized domain names (https://en.wikipedia.org/wiki/Internationalized_domain_name#Example_of_IDNA_encoding https://en.wikipedia.org/wiki/Internationalized_domain_name#...), and this mangles perfectly-valid ones.

arjvik 4y ago
I've got no idea why either regex would be useful, can anyone come up with a reason other than a bad attempt to sanitize input?

raldi 4y ago

When I run that program on my Mac, I don't encounter the bug: 2147440447 -> 24855 2147440448 -> 24855

nebulous1 4y ago
Make sure you're compiling with -m32
- astrange 4y ago
  No such thing on Macs anymore. You'd need qemu.

nebulous1 4y ago

> return (long) (t + DAY / 2) / DAY; should have been > return (long) ((t + DAY / 2) / DAY); ?

hermitdev 4y ago
No, the problem is using the wrong data type: "long". On 32-bit systems, Linux defines long to be 32-bits,vs 64 bits on 64-bit. As another example of how this could be broken: use this code on Windows, either 32 or 64 bit and the code is completely broken because longs are 32-bits regardless of whether the OS is 32 or 64 bit capable. Use time_t as that's the correct type and will do the right thing assuming your C standard lib is up to date.
- nebulous1 4y ago
  Well, the problem is that it is casting (t + DAY/2) to a long before it divides by DAY, not after. The whole expression ((t + DAY/2) / DAY) will not overflow 32 bits in any reasonable timeframe, but (t + DAY / 2) will.
  - Arnavion 4y ago
    https://github.com/shadow-maint/shadow/commit/815ffb7d3e0a8e1d78886dded65a5c6afd22c94e#diff-9ade509d0a1eeb8cd0b2638ddf8ddb2bd65a8dd3a901c829fcac29c12cd42588R81 https://github.com/shadow-maint/shadow/commit/815ffb7d3e0a8e... Amusing that it was broken by an attempt to make it clearer.

zokier 4y ago

Tangentially related, but I find it somewhat baffling that at some point people thought that using data types with unknown ranges is viable way of producing robust portable programs. It took until C99 to get basic int32_t etc, before that you were mostly relying on bunch of ifdefs to get portable results. The whole mindset for things like short/int/long must have been just so very different than what I'm used to.

jodoherty 4y ago
It doesn't make sense in the context of today's relatively uniform hardware, but I think the goal wasn't so much portably correct code, but portably efficient code that avoided the need to use assembly language for everything. The first version of UNIX was written in assembly language for an 18-bit machine. C was invented as part of rewriting UNIX to work on a 16-bit machine and who knows what to expect after that. Machine architectures were a lot less uniform back in the day, and most of those minicomputers are easily beat by some of today's 20 cent microcontrollers. Given that context, being able to loosely define relative data type sizes based on how efficiently they might be represented on the machine might make more sense. But you're right, that's quite alien to where we are today. By the 1980s, Ada was already making the right call of allowing the developer to specify the behavior and exact range of modular integer types for correctness and letting the compiler optimize things for efficiency or performance as needed. It'd be nice to see that approach become more popular.

johnklos 4y ago

This is why people shouldn't write their own time functions (in the same vein as not writing their own encryption code). If it's not used in a public library by many, many other things, issues like these might not be discovered until it's too late, or until Rachel discovers them (but Rachel can't be expected to check all the things).

aftbit 4y ago
By extension of that logic, engineers basically shouldn't write anything except very high level application glue code, and only that as a last resort. Rely on libraries made by people who are domain experts or at least have a lot of eyeballs and users, because _every_ problem domain has a lot of ugly warts and major issues. People warn about time and encryption specifically because these issues can be subtle and take years to be exposed, but the same is true for lots of others as well. Don't write your own parsers, network libraries, locking primitives, data structures, filesystems, databases, etc. Except of course, sometimes you have to.
- xboxnolifes 4y ago
  The insight is to not write critical, bug prone code if you don't have to. Encryption, time, primitives, etc., are things that many people need and use and already exist. If the solution you need doesn't already exist, of course you need to write your own.

astrange 4y ago

> 2038-01-18 15:14:08Z is when that code will start returning negative numbers. That'll be fun and interesting. Ah but it won't, because signed integer overflow is undefined in C. It will make demons fly out your nose. (gcc may make it return negative numbers, but clang may do something else.)

RickHull 4y ago

Somewhat related, a small ruby project of mine to read input events from the linux kernel. https://github.com/rickhull/device_input https://github.com/rickhull/device_input The README goes in some depth about kernel events and timestamps and time implementations on 32-bit vs 64-bit systems.