4 ms·
Sounds like these guys didn't use custom kernels, but BitNet did.
by thatguysaguy 3y ago
Sounds like these guys didn't use custom kernels, but BitNet did.
- mobicham 3y agoThat's correct. Only the dequantization is done on CUDA, the matmul is done with Pytorch. If they put their kernels open-source we could re-use them!