3 ms·
Okay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's
by vatsachak 3mo ago
Okay here's something LLMs can't. They can't solve problems that are longer than ~10 pages of math. They also can't maintain codebases without supervision. It's because they are have no memory and use various tricks to supplant that fact.
- ACCount37 3mo agoNow, how long before someone rolls out some sort of 10M context hybrid attention active context management monstrosity and ruins this guy's "can't"? Start the clock. My opinion of claims like "LLMs need memory to manage codebases" has also hit the dumpster bin a while ago. Why would knowing how to make a maintainable change to a codebase require any more "memory" than knowing how to play an optimal chess move? The codebase is the memory. A sufficiently capable LLM can ingest it, figure out what changes to make, and make them.
- kskdkwkdjw 3mo ago> The codebase is the memory. Were that the case LLMs would’ve been phenomenal code monkeys from the get go. They were not. They still are not.
- vatsachak 3mo agoOkay then why do software engineers still exist. And if companies are making an error keeping people on instead of LLMs, why isn't there an all agent company making money off of it?
- dandaka 3mo ago1/ LLMs are doing increasingly bigger share of work of SWE with an increasing success rate 2/ SWE are doing way more than "writing code", and since those areas are less "computational verifiable" (what is a good architecture that will stand in 5y?) and more "connected to real world" (what are the requirements?), LLMs struggle with them