3 ms·
What stopped you from scaling this method to billions of neurons and training a LLM SNN?
by jackblemming 3y ago
What stopped you from scaling this method to billions of neurons and training a LLM SNN?
- orbifold 3y agoThere is nothing theoretical that stops anyone from trying that. It is just a matter of engineering and architecture. It is (in principle) possible to define "spiking transformers" and there is more than one way to do so, but this will not mean necessarily that you will immediately get good performance. Also current generation hardware doesn't as efficiently simulate SNN, so this is another constraint.