3 ms·
The contexts are pretty large now
by ThrowawayTestr 2y ago
The contexts are pretty large now
- stickfigure 2y agoYour nefarious plan for enslaving humanity is still unlikely to fit into 128k tokens.
- HeatrayEnjoyer 2y agoOperational success does not hinge on persisting the entire plan in working memory, that's what notebooks and word docs are for. 128k is table stakes now, regardless. Google's models support 1 million tokens and 10 million for approved clients. That is 13x War and Peace, or 1x the entire source code for 3D modeling application Blender.
- xvector 2y agoYeah, but most LLMs are barely functional after 16k tokens, even if it says 128k on the tin. Sure, they will have recall, but the in-context reasoning ability drops dramatically. LLMs just aren't smart enough to take over the world. They suck at backtracking, they're pretty bad at world models, they struggle to learn new information, etc. o1, QwQ, and CoT models marginally improve this but if you play with them they still kinda suck
- sam_1m 2y ago[dead]