10 ms·
The time and space complexity of inference is constant wrt. context size. You will probably need more parameters to match the performance of a transformer thoug
by sebzim4500 4y ago
The time and space complexity of inference is constant wrt. context size. You will probably need more parameters to match the performance of a transformer though, so whether it scales better in practice is an open question.