Neural nets usually use 32-bit floats for weights. There is a rapid trend of using quantization. There are already papers describing 2-bit quantization strategi

Neural nets usually use 32-bit floats for weights. There is a rapid trend of using quantization. There are already papers describing 2-bit quantization strategies – either quantization-aware training or post-training quantization. Less precision means larger & faster models (current utility), buuuut more importantly you can start to think about different substrates for implementation – analog or optical seems to be viable paths.