3 ms·
Increasing the token limit is a solvable problem
by WithinReason 4y ago
Increasing the token limit is a solvable problem
- jhoelzel 4y agoSure we just need next level super computers for these large models and the patience of multiple days to wait for output
- WithinReason 4y agoIncreasing the token limit without needing more resources to run the network is a solvable problem
- jhoelzel 4y agobut you do sure see the problem with a codebase right?
- WithinReason 4y agoThe current token limit comes from a O(N^2) memory requirements for N tokens, there is research that's trying to reduce this towards O(N), for example as the (downvoted) sibling comment suggests. This is not exactly straightforward but not impossible either. It's not a fundamental limitation of language models going forward.
- jhoelzel 4y agothat still will only be enough to hold an extended cli application
- throwaway3672 4y agoNot necessarily - you just need hierarchical abstraction memory. I reckon my "token" limit when analysing code is around 7.