5 ms·
No, it's not. I've not seen any local models that can handle 1m+ tokens. I haven't actually done many experiments with long context local models - I tend to hi
by simonw 2y ago
No, it's not. I've not seen any local models that can handle 1m+ tokens.
I haven't actually done many experiments with long context local models - I tend to hit the hosted API models for that kind of thing.