4 ms·
Is there a limit on how big the context size can be?
Is there a limit on how big the context size can be?
There’s no theoretical limit
There is a soft limit due to the computation required; the currently used model architectures are quadratic with respect to context size, so if you want ten times larger context size, that's going to need a hundred times more effort.