6 ms·

In Java there's just Math.random(), and it's been broken, with WONTFIX, since 1998. Java's random number generator returns exceptionally non-random values, but

In Java there's just Math.random(), and it's been broken, with WONTFIX, since 1998. Java's random number generator returns exceptionally non-random values, but Sun, er, Oracle, won't fix it because of the most insane of reasons: unlike in any reasonable language, Java's PRNG essentially has a contract to be deterministic. There's seemingly a worry that someone, somewhere, is actually relying on java.util.Random to always produce the same random number sequence for a given seed from Java version to version.

alex_duf 3y ago
Why is it unreasonable that given the same seed the same sequence will be generated? It's useful for generative art and video games at least. Sure it shouldn't be used for security...
- SeanLuke 3y ago
  Because it permanently prevents all bugfixes for a major library class. And java.util.Random has some whopping bugs. If you need determinism, then the proper thing to do is use your own RNG. There's been a good Mersenne Twister implementation on Java since 1999. But a system-wide generator should never be beholden to determinism.
  - jibe 3y ago
    If you need determinism, and the standard library is deterministic, it seems reasonable to use the standard library.
    - SeanLuke 3y ago
      Even if its high bits go 1 0 1 0 1 0 1 0 ... ?
      - _a_a_a_ 3y ago
        I would say that's a bug in the RNG, nothing to do with any determinism issue
        adgjlsfhk1 3y ago
        The point is that if you document your rng as deterministic, you can't fix any bugs.
        _a_a_a_ 3y ago
        why do you believe that? Can you explain why not?
        adgjlsfhk1 3y ago
        because those bugfixes will change the random numbers generated
        _a_a_a_ 3y ago
        That's fine (if you document it). Repeatability may mean 'repeatability over the same program instance/minor version/major instance'. It's fine if you're clear. But ISWYM too.
  - _a_a_a_ 3y ago
    Links to these whopping bugs pls? I also don't think you've justified why determinism from a seed is a bad thing.
    - SeanLuke 3y ago
      Name another language where random() is deterministic. Ask yourself why that is. As to bugs. Let's start with the famous one: http://alife.co.uk/nonrandom/ http://alife.co.uk/nonrandom/ This is due to boneheaded mistakes in Sun's choice of constants for its LCG and errors in its bit-handling. These are massive mistakes, which it can no longer fix. Next, there's an outstanding bug in nextBytes(), which generates ints and then cuts them into bytes (a big no-no for this particular LCG). Some unfortunate omissions: nextChar(), nextShort(), nextByte() are missing, and there is no save-state procedure. There is no nextDouble() method that is inclusive for one or exclusive for zero. There was a notorious bug in nextGaussian() which would take the log of 0 and then divide it by 0, but that has long been fixed. :-) [And some other bugs which were fixed early on despite Sun's claim that it couldn't fix bugs due to its stupid nondeterminism promise. For an RNG!]
      - _a_a_a_ 3y ago
        > Name another language where random() is deterministic pretty well all of them, surely? eg. https://learn.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql?view=sql-server-ver16 https://learn.microsoft.com/en-us/sql/t-sql/functions/rand-t... "For a specified seed value, the result returned is always the same." Pretty well every language I'm aware of does it this way. Are we even talking about the same thing? As for the others you point out, I'm afraid I can't speak for that. I'll just have to accept what you say.
        adgjlsfhk1 3y ago
        Julia, for example promises that seeded random numbers will only be the same within the minor version and there's a 3rd party package if you need reproducible random numbers. Guaranteeing cross version random number compatibility comes at a pretty major cost for performance (as new better algorithms are discovered) and removes your ability to bugfix the random number generator.
- somat 3y ago
  Because random(seed) has no specification for what the output will be. random() should only be used if you specifically want a non-deterministic value. The function you want to use for uniform distribution in a deterministic manner is hash(value)
stiff 3y ago
I think this is no longer true: https://www.baeldung.com/java-17-random-number-generators https://www.baeldung.com/java-17-random-number-generators https://openjdk.org/jeps/356 https://openjdk.org/jeps/356
- SeanLuke 3y ago
  I guess sort of with the new proposal.
somat 3y ago
Not java but a company I worked for had a perl application where old sales records were encrypted then put on dvd's for long term cold storage. I was a lowly tape monkey at the time so the actual details of the application were far above my pay grade. but long nights between shuffling tapes and swapping dvd I would try and figure out how the system worked. One thing I found out is that they were generating the encryption keys by combining a couple of values(something like secret + date I think) then hashing. They would then regenerate the key if the data ever needed to be retrieved. On it's own this is... fine I guess. cold records in a secure(ish) facility. The worrisome part was that they were using perl's rand(seed) as a hash function. Young me was like "well I don't know much about perl but I don't think there is any documentation guarantee about how exactly the rand(seed) function works." a few quick tests later and I found out that yes rand(seed) will return different values from different operating systems. and sometime even changing the version of perl was enough. They ended up having to make sure they went back to the same system the key was generated on then regenerate and store all keys used. The lesson I learned, don't use random() when you want hash(). random is for non-deterministic output, hash is for deterministic output. random(seed) is an artifact of implementation and should never be used for deterministic output. The real wtf was that this was perl on win98, probably due to when it was implemented, they wanted dvd burning capability and someone sort of knew perl.
nitwit005 3y ago
When they changed the iteration order of HashMap keys (1.7 I think?), tons of code broke, although fortunately mostly tests. It was easy to depend on it by doing something like creating a gold file test with expected results. They don't want code to break like that, which means they need to preserve this sort of thing.
jjkeddo199 3y ago
Maybe its Minecraft?
iudqnolq 3y ago
Isn't it normal to set a fixed seed in testing and then expect deterministic results so you can write simpler assertions?

alex_duf 3y ago

Why is it unreasonable that given the same seed the same sequence will be generated? It's useful for generative art and video games at least. Sure it shouldn't be used for security...

SeanLuke 3y ago
Because it permanently prevents all bugfixes for a major library class. And java.util.Random has some whopping bugs. If you need determinism, then the proper thing to do is use your own RNG. There's been a good Mersenne Twister implementation on Java since 1999. But a system-wide generator should never be beholden to determinism.
- jibe 3y ago
  If you need determinism, and the standard library is deterministic, it seems reasonable to use the standard library.
  - SeanLuke 3y ago
    Even if its high bits go 1 0 1 0 1 0 1 0 ... ?
    - _a_a_a_ 3y ago
      I would say that's a bug in the RNG, nothing to do with any determinism issue
      - adgjlsfhk1 3y ago
        The point is that if you document your rng as deterministic, you can't fix any bugs.
        _a_a_a_ 3y ago
        why do you believe that? Can you explain why not?
        adgjlsfhk1 3y ago
        because those bugfixes will change the random numbers generated
        _a_a_a_ 3y ago
        That's fine (if you document it). Repeatability may mean 'repeatability over the same program instance/minor version/major instance'. It's fine if you're clear. But ISWYM too.
- _a_a_a_ 3y ago
  Links to these whopping bugs pls? I also don't think you've justified why determinism from a seed is a bad thing.
  - SeanLuke 3y ago
    Name another language where random() is deterministic. Ask yourself why that is. As to bugs. Let's start with the famous one: http://alife.co.uk/nonrandom/ http://alife.co.uk/nonrandom/ This is due to boneheaded mistakes in Sun's choice of constants for its LCG and errors in its bit-handling. These are massive mistakes, which it can no longer fix. Next, there's an outstanding bug in nextBytes(), which generates ints and then cuts them into bytes (a big no-no for this particular LCG). Some unfortunate omissions: nextChar(), nextShort(), nextByte() are missing, and there is no save-state procedure. There is no nextDouble() method that is inclusive for one or exclusive for zero. There was a notorious bug in nextGaussian() which would take the log of 0 and then divide it by 0, but that has long been fixed. :-) [And some other bugs which were fixed early on despite Sun's claim that it couldn't fix bugs due to its stupid nondeterminism promise. For an RNG!]
    - _a_a_a_ 3y ago
      > Name another language where random() is deterministic pretty well all of them, surely? eg. https://learn.microsoft.com/en-us/sql/t-sql/functions/rand-transact-sql?view=sql-server-ver16 https://learn.microsoft.com/en-us/sql/t-sql/functions/rand-t... "For a specified seed value, the result returned is always the same." Pretty well every language I'm aware of does it this way. Are we even talking about the same thing? As for the others you point out, I'm afraid I can't speak for that. I'll just have to accept what you say.
      - adgjlsfhk1 3y ago
        Julia, for example promises that seeded random numbers will only be the same within the minor version and there's a 3rd party package if you need reproducible random numbers. Guaranteeing cross version random number compatibility comes at a pretty major cost for performance (as new better algorithms are discovered) and removes your ability to bugfix the random number generator.
somat 3y ago
Because random(seed) has no specification for what the output will be. random() should only be used if you specifically want a non-deterministic value. The function you want to use for uniform distribution in a deterministic manner is hash(value)

stiff 3y ago

I think this is no longer true: https://www.baeldung.com/java-17-random-number-generators https://www.baeldung.com/java-17-random-number-generators https://openjdk.org/jeps/356 https://openjdk.org/jeps/356

SeanLuke 3y ago
I guess sort of with the new proposal.

somat 3y ago

Not java but a company I worked for had a perl application where old sales records were encrypted then put on dvd's for long term cold storage. I was a lowly tape monkey at the time so the actual details of the application were far above my pay grade. but long nights between shuffling tapes and swapping dvd I would try and figure out how the system worked. One thing I found out is that they were generating the encryption keys by combining a couple of values(something like secret + date I think) then hashing. They would then regenerate the key if the data ever needed to be retrieved. On it's own this is... fine I guess. cold records in a secure(ish) facility. The worrisome part was that they were using perl's rand(seed) as a hash function. Young me was like "well I don't know much about perl but I don't think there is any documentation guarantee about how exactly the rand(seed) function works." a few quick tests later and I found out that yes rand(seed) will return different values from different operating systems. and sometime even changing the version of perl was enough. They ended up having to make sure they went back to the same system the key was generated on then regenerate and store all keys used. The lesson I learned, don't use random() when you want hash(). random is for non-deterministic output, hash is for deterministic output. random(seed) is an artifact of implementation and should never be used for deterministic output. The real wtf was that this was perl on win98, probably due to when it was implemented, they wanted dvd burning capability and someone sort of knew perl.

nitwit005 3y ago

When they changed the iteration order of HashMap keys (1.7 I think?), tons of code broke, although fortunately mostly tests. It was easy to depend on it by doing something like creating a gold file test with expected results. They don't want code to break like that, which means they need to preserve this sort of thing.

jjkeddo199 3y ago

Maybe its Minecraft?

iudqnolq 3y ago

Isn't it normal to set a fixed seed in testing and then expect deterministic results so you can write simpler assertions?