2 ms·
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investig
by CSMastermind 3mo ago
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
- ifwinterco 3mo agoIt’s got worse though right? Older models from before everyone went off the deep end with CoT don’t do this and just write the code with 1/10 the token usage. The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine. CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
- NuclearPM 3mo agoCoT?
- NatKarmios 3mo agoChain of Thought, according to a quick search