3 ms·
What would be the advantage of binarized gradient descent?
by karmacondon 11y ago
What would be the advantage of binarized gradient descent?
- jimfleming 11y agoFor one, more compression during parameter transfer in data parallelism scenarios.
- tachyonbeam 11y agoAnd maybe the possibility of building more efficient hardware. Floating point units take up a lot of transistors (in the ~100K transistor range AFAIK), whereas boolean logic is tiny.
- Scaevolus 11y agoGoing to 1-bit precision is probably overkill, but papers have shown how neural networks with 8 bits of precision and simple ALUs can give results equivalent to full FPUs.