2 ms·
I agree - unfortunately we have gone so far down the rabbit hole optimizing transformer models. Alternative models like S4 have been able to get transformer le
by whimsicalism 4y ago
I agree - unfortunately we have gone so far down the rabbit hole optimizing transformer models.
Alternative models like S4 have been able to get transformer level performance with O(N) sequence length scaling.