3 ms·
That is in fact the elephant in the room: you will most likely never be able to actually use it for complex stuff because of the token limits required for ai.
by jhoelzel 4y ago
That is in fact the elephant in the room: you will most likely never be able to actually use it for complex stuff because of the token limits required for ai.
Imagine analyzing a massive code base, sure it can tell you how you where solving function ex by translating it to natural language, but it still does not understand any of it.
As far as i know, training it on your dataset will not improve this.
- WithinReason 4y agoIncreasing the token limit is a solvable problem
- jhoelzel 4y agoSure we just need next level super computers for these large models and the patience of multiple days to wait for output
- WithinReason 4y agoIncreasing the token limit without needing more resources to run the network is a solvable problem
- jhoelzel 4y agobut you do sure see the problem with a codebase right?
- WithinReason 4y agoThe current token limit comes from a O(N^2) memory requirements for N tokens, there is research that's trying to reduce this towards O(N), for example as the (downvoted) sibling comment suggests. This is not exactly straightforward but not impossible either. It's not a fundamental limitation of language models going forward.
- jhoelzel 4y agothat still will only be enough to hold an extended cli application
- throwaway3672 4y agoNot necessarily - you just need hierarchical abstraction memory. I reckon my "token" limit when analysing code is around 7.