2 ms·
GPU's aren't really designed for 1 bit math... They don't perform much faster than floating point math. Whereas a custom ASIC or updated design of GPU could
by londons_explore 3y ago
GPU's aren't really designed for 1 bit math... They don't perform much faster than floating point math.
Whereas a custom ASIC or updated design of GPU could give massive speedups with 1 bit math.
- bee_rider 3y agoFor 1 bit math, at least it should be possible to populate every other bit of an integer type, right? Surely one could do better with a dedicated type for this, but at least we could pack 16 single-bit weights into a 32 bit int for addition, right?
- UncleOxidant 3y agoYes, exactly. Neither GPUs nor CPUs are setup for 1 bit math. Pulling 1 or 2 bits out of a word isn't all that straightforward on CPU or GPU - lots of shifting and masking. I wonder how long it's going to be before we see custom hardware for bitnets? I suspect we'll see it on FPGAs first.
- imtringued 3y agoYou're telling me GPUs aren't designed for additions and subtractions? Where did you hear that?
- bick_nyers 3y agoI think they are moreso saying that GPUs are not optimized for those operations. CPU aren't "designed" for matrix multiplies yet we can still run them, albeit at a slower rate than on a GPU.
- shaklee3 3y agoA100 (> 5yo GPU) has a 1-bit tensor core engine