4 ms·

You make my point exactly. It is possible to get deterministic floating point out of existing toolchains. It requires work. Sure, people who can't do that wo

by yarvin9 11y ago

You make my point exactly. It is possible to get deterministic floating point out of existing toolchains. It requires work. Sure, people who can't do that work suck, for certain values of the word "suck." I'm just saying, like OP, that this work shouldn't be needed. You should get this result for free and not even have to think about it. The instruction sets certainly aren't irreproducible and I shouldn't have implied that, although there are plenty of IEEE extensions not well-standardized (eg, transcendentals). The problem is that their specifications grow weaker as you move up the programming hierarchy. Optimizers rearranging algebra, etc. And the result that's presented to the normal programmer is that floating-point is analog computing.

varelse 11y ago
To get this for free (for various values of the word "free"), you'd have to encapsulate reduction across arbitrary distributions of processes and threads. Good luck with that(1)... As for the second part, one needs a great deal more thought into the definitions of transcendentals before proceeding. The 22-bit mantissa approximations of transcendentals on GPUs are fantastic for most scientific computing and machine learning applications. Any error they introduce is more than offset by errors introduced by the assumptions that led to those algorithms in the first place. (1)We agree that the farther you move up the programmer hierarchy, the harder it is to make a reproducible algorithm. My solution is to defy the trend and implement the bowels of the hierarchy, but I'd be happy for a computer scientist to design something from the ground up that was bitwise-reproducible (and please please strongly typed so it's L1-friendly without doing functional programming backflips to make it so). I really don't think it should be that hard given how easily and repeatedly I have done so from the guts outward.
- yarvin9 11y ago
  You can do a lot with single-threaded actors. You're right - it's not that hard, for certain values of "hard." Working on it... :-)

by yarvin9 11y ago

varelse 11y ago

To get this for free (for various values of the word "free"), you'd have to encapsulate reduction across arbitrary distributions of processes and threads. Good luck with that(1)... As for the second part, one needs a great deal more thought into the definitions of transcendentals before proceeding. The 22-bit mantissa approximations of transcendentals on GPUs are fantastic for most scientific computing and machine learning applications. Any error they introduce is more than offset by errors introduced by the assumptions that led to those algorithms in the first place. (1)We agree that the farther you move up the programmer hierarchy, the harder it is to make a reproducible algorithm. My solution is to defy the trend and implement the bowels of the hierarchy, but I'd be happy for a computer scientist to design something from the ground up that was bitwise-reproducible (and please please strongly typed so it's L1-friendly without doing functional programming backflips to make it so). I really don't think it should be that hard given how easily and repeatedly I have done so from the guts outward.

yarvin9 11y ago
You can do a lot with single-threaded actors. You're right - it's not that hard, for certain values of "hard." Working on it... :-)