6 ms·
> {In late July of 2025} all LLMs hit a ceiling of complexity beyond which they cease to understand the code base Fixed that for you
by drcode 1y ago
> {In late July of 2025} all LLMs hit a ceiling of complexity beyond which they cease to understand the code base
Fixed that for you
- yifanl 1y agoIf you're from the future, please let us know and we can alert the relevant authorities to ~~disect~~ help you.
- drcode 1y agobecause I predicted that AI will get better in future months?
- showerst 1y agoThe fundamental question is "will the LLM get better before your vibecoded codebase becomes unmaintainable, or you need a feature that is beyond the LLM's ceiling". It's an interesting race.
- 20k 1y agoEvery 6 months since chatgpt launched, everyone keeps telling me that LLMs are going to be amazing in a year from now and they'll replace programmers, just you wait They're getting better, but a lot of the improvement was driven by increases in the training data. These models have now consumed literally all available information on the planet - where do they go from here?
- drcode 1y agoAs far as I understand, coding ability of AIs is now driven mostly entirely by RL, as well synthetic data generated by inference time compute combined with code execution tool use. Coding is arguably the single thing least affected by a shortage of training data. We're still in the very early steps of this new cycle of AI coding advancements.
- AlwaysRock 1y agoYeah... There are improvements to be made by increasing the context window and having agents reference documentation more. Half the issues I see are with agents just doing their own thing instead of following established best practices they could/should be referencing in a codebase or looking up documentation. Which, believe it or not, is the same issue I see in my own code.
- fragmede 1y agoGive the LLM access to a VM with a compiler and have it generate code for it to train on. They're great at next.js but not as good with swift. So have it generate a million swift programs, along with tests to verify they actually work, and add that to the private training data set.
- marcosdumay 1y agoThe "time to amazingness" is falling quickly, though. It used to be "just a few years" a few years ago, and has been steady around 6 months for the last year or so. I'm waiting for the day when every comment session on the internet will be full of people predicting AGI tomorrow.
- spacemadness 1y agoAs you can see from the other commenters on here, any perceived limitation is no longer the fault of the LLM. So where we go from here is gaslighting. Never mind that the LLM should be good at refactoring, you need to keep doing that for it until it works you see. Or the classic you’re prompting it wrong, etc.
- drcode 1y agoLet's hope you are right, and that the limitations of AI will never improve, and that all of us get to live out our full natural lifespans without AI xrisk