4 ms·
Au contraire - LLMs are quite bad at large scale pattern fidelity. They'll even forget key details and constraints unless told over and over again. That's why A
by akssri 3mo ago
Au contraire - LLMs are quite bad at large scale pattern fidelity. They'll even forget key details and constraints unless told over and over again. That's why AI-written code has the quality of a patch-on-patch-on-patch.
- gofreddygo 3mo ago[dead]
- karlkloss 3mo agoThat's also true for humans.
- dnikolovv 3mo agoReductio ad absurdum.
- fxtentacle 3mo agoHumans will typically learn after you have forced them to apologise for the same mistake for 20 times in a row. AI won’t.
- AndrewThrowaway 3mo agoIf you gave some junior dev a large codebase and just told to "refactor it" you would get a terrible result. If you gave junior dev exact tasks what to do where you will get better results. Just like with LLM.
- al_borland 3mo agoThat’s why junior devs generally aren’t give the responsibility of architecting a large scale refactor. Yet people seem to be trying to had these types of tasks over to LLMs.
- AndrewThrowaway 3mo agoExactly. Is it LLMs fault or yours to believing in it so much.
- LtWorf 3mo ago"Coding is solved"
- AndrewThrowaway 3mo agoCoding, programming and engineering are all a bit different concepts. Coding as in typing in the code might be solved. Engineering? Doubt. What is definitely not solved is knowing what you want and what user wants and what the end result is supposed to be. To write the code you need the specs and to write the specs you need to know what you want. And that can only be answered by years of therapy.
- al_borland 3mo agoIt is the fault of the salesmen and evangelists. I’ve been hearing for quite some time now that I should be using an LLM to plan before the build. This is treating the LLM as the architect, not as a junior being handed small tasks here and there. I haven’t bought into it, so don’t use it this way, but an army of people online and in the media are pushing the fomo hard and telling people this is how it should be used. If the LLM isn’t doing what we want, it simply means we need to use the LLM even more. That’s the prevailing message from the industry.
- vogelke 3mo agoI love this. Straight into my quotes file.
- deleted 3mo ago[deleted]
- fxtentacle 3mo agoFully agree. I tried to refactor parts of a large code base with Fable+ultracode and it just keeps accidentally merging distinct concepts and making up explanations/reasonings that the code base did not contain. For example, the code base contains a physical controller. It’s closed loop in that it can react in realtime to changes. But it’s a slightly untypical implementation because this one can even look into the future through simulations. But Fable does not understand that. Instead, I need to remind it every 30 minutes that this is closed loop. It keeps wrongly claiming that the controller was open loop and then based upon that it will make up constraints that don’t actually exist.
- rapidaneurism 3mo agoWould putting that in black and white in the comments around then controller help?
- bulbar 3mo agoI feel like there are a lot "you are holding it wrong" arguments flying around. Like, when somebody says that AI wasn't able to accomplish something, people tend to assume it's an User problem. Meanwhile, I have a hard time to believe people don't encounter problems with AI solutions on a regular basis (I do).
- cyanydeez 3mo agomostly the problem with coding is the semantic ambiguation; coders like to reuse similar methods, variable names, copy/paste etc; so large code bases have so much out of context simularities that the LLM, regardless of size, isn't
- AndrewThrowaway 3mo ago"It is not LLMs fault but you not knowing how to write a prompt". I know I know. But just giving all codebase and saying "rewrite it" is a no go. If e.g. going one class after class LLM will be exceptionally good at keeping the patterns and logics. I mean it is a tool and you need to understand how the tool works. When there is too little context, where there is so much context so that you are poisoning it, when you are allowing the tool to do patch-on-patch and etc.
- ed_elliott_asc 3mo agoThe patch-on-patch-on-patch is exactly right, nice way to describe it. It feels like, and I think is, optimized to find the quickest answer not necessarily the right answer.