3 ms·
An architecture where compute grows linearly with context length seems dangerous. It can get very expensive as context grows and performance degrades
by noashavit 5mo ago
An architecture where compute grows linearly with context length seems dangerous. It can get very expensive as context grows and performance degrades