4 ms·
Compute scales quadratically with increasing context window sizes. It's not reasonable to achieve this with current network architectures.
by reaperman 3y ago
Compute scales quadratically with increasing context window sizes. It's not reasonable to achieve this with current network architectures.