3 ms·
I don't think the integer weights are neccessary for sparsity; they are just faster because they allow for low precision. Of course floats aren't strictly asso
by eutectic 6y ago
I don't think the integer weights are neccessary for sparsity; they are just faster because they allow for low precision.
Of course floats aren't strictly associative so you wouldn't get bitwise equivalence between the incremental and non-incremental updates, but I don't see how that would matter in this context.
- dan-robertson 6y agoThe point is that integer weights allow for incremental updates to be correct. If you used floats then they would drift away from correct as you applied more incremental updates. And neural networks can be quite sensitive to small errors
- eutectic 6y agoWith 32 bits of precision I don't see it being a big problem over a maximum of say 50 moves. Neural networks which are not overfit should be tolerant of a small amount of random (not crafted) noise.
- deleted 6y ago[deleted]
- recursive 6y ago64-bit floats have 53 bits of precision. 32-bit floats have 24. The rest goes into sign and exponent.
- dan-robertson 6y agoMaybe the network uses integers for other reasons. Perhaps due to the history of the program. I don’t know how the search works but I would imagine it would consider more than 50 moves with lots of backtracking and trying different possible moves.