3 ms·
I am big proponent of balanced ternary for neural nets and general computation (Setun), but this repo does not make sense to me at all.
by mlajtos 3y ago
I am big proponent of balanced ternary for neural nets and general computation (Setun), but this repo does not make sense to me at all.
- pyinstallwoes 3y agoSETUN fascinates me as well. What are your thoughts on ternary computation and neural nets, or general use of it? I myself want to explore some more use cases with utility rather than yak/shaving.
- mlajtos 3y agoNeural nets usually use 32-bit floats for weights. There is a rapid trend of using quantization. There are already papers describing 2-bit quantization strategies – either quantization-aware training or post-training quantization. Less precision means larger & faster models (current utility), buuuut more importantly you can start to think about different substrates for implementation – analog or optical seems to be viable paths.