4 ms·

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with t

Random sampling issues due to problems with the inference kernels on certain GPU configurations. This seems like a clear root cause and has nothing to do with the magic of NNs. I don’t understand what the fuss is about.

throwuwu 3y ago
They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token instead. Instant aphasia. The report said it was a GPU configuration problem so my guess is the wrong precision was used in some number of GPUs but I have no idea how they configure their cluster so take that with a giant grain of salt.
- cqqxo4zV46cp 3y ago
  Yep. Some obviously made their mind up before the page even rendered.
- meowface 3y ago
  Agreed, but I think they raise a fair point that some kind of automated testing probably should've existed to look for sudden major changes in prompt -> output production between versions, with lots of prompt test cases. Maybe this testing did exist but this problem just didn't surface, for some reason?
- FridgeSeal 3y ago
  > so my guess is the wrong precision was used in some number of GPUs Well we wouldn’t have to guess if their PM wasn’t so pointlessly vague would we?
FridgeSeal 3y ago
If that’s what it was, they’ve done a fabulously bad job of conveying that, and then made no attempt to dig into why _that_ happened. Which, again, is like going “well it broke because it broke”, not even so much as a “this can happen because some bit of hardware had a cosmic bit flip and freaked out, generally happens with a probability of 1:x”.

throwuwu 3y ago

They just want to reinforce their own bias that OpenAI BAD and DUMB, rationalism GOOD! When it’s their own fault for not understanding enough theory to know what happens if there were to be a loss of precision in the predicted embedding vector that maps to a token. If enough decimal places are lopped off or nudged then that moves the predicted vector slightly away from where it should be and you get a nearby token instead. Instant aphasia. The report said it was a GPU configuration problem so my guess is the wrong precision was used in some number of GPUs but I have no idea how they configure their cluster so take that with a giant grain of salt.

cqqxo4zV46cp 3y ago
Yep. Some obviously made their mind up before the page even rendered.
meowface 3y ago
Agreed, but I think they raise a fair point that some kind of automated testing probably should've existed to look for sudden major changes in prompt -> output production between versions, with lots of prompt test cases. Maybe this testing did exist but this problem just didn't surface, for some reason?
FridgeSeal 3y ago
> so my guess is the wrong precision was used in some number of GPUs Well we wouldn’t have to guess if their PM wasn’t so pointlessly vague would we?

FridgeSeal 3y ago

If that’s what it was, they’ve done a fabulously bad job of conveying that, and then made no attempt to dig into why _that_ happened. Which, again, is like going “well it broke because it broke”, not even so much as a “this can happen because some bit of hardware had a cosmic bit flip and freaked out, generally happens with a probability of 1:x”.