4 ms·
> We don't expect human developers to be perfect, why should we expect AI assistants. What absolute nonsense. What an absurd false equivalence. It's not that w
by mvdtnz 2y ago
> We don't expect human developers to be perfect, why should we expect AI assistants.
What absolute nonsense. What an absurd false equivalence. It's not that we expect perfection or even human level performance from "AI". It's that the crap that comes out of LLMs is not even at the level of a first year student. I've never in my entire life reviewed the code of a junior engineer and seen them invent third party APIs from whole cloth. I've never had a junior send me code that generates a payload that doesn't validate at the first layer of the operation with zero manual testing to check it. No junior has ever asked me to review a pull request containing references to an open source framework that doesn't exist anywhere in my application. Yet these scenarios are commonplace in "AI" generated code.
- simonw 2y agoThat problem genuinely doesn't matter to me at all. If an LLM hallucinates a method that doesn't exist I find out the moment I try and run the code. If I'm using ChatGPT Code Interpreter (for Python) or Claude analysis mode (for JavaScript) I don't even have to intervene: the LLM can run in a loop, generating code, testing that it executes without errors and correcting any mistakes it makes. I still need to carefully review the code, but the mistakes which cause it not to run at all are by far the least amount of work to identify.
- mvdtnz 2y agoYes I've seen the dreck you produce with LLMs. Not a shining endorsement in my eyes. https://news.ycombinator.com/item?id=41929174 https://news.ycombinator.com/item?id=41929174
- simonw 2y agoWhich of those did you think were dreck? I think the source code for tools like this one is genuinely good code: https://github.com/simonw/tools/blob/main/extract-urls.html https://github.com/simonw/tools/blob/main/extract-urls.html What do you see that's wrong with that?
- mvdtnz 2y agoIt's a toy. It doesn't do useful work. The code is fine for the pathetically small sample but that coding style does not scale to real software scales.
- NitpickLawyer 2y ago|_____| <- dreck code ... ... ... |_____| <- it's good code, but toy problem I guess we all see where the goalposts will be tomorrow. Good code, good problem, I don't like the language. Or something :)
- mvdtnz 2y ago"Dreck" means worthless rubbish. Code that solves useless toy problems is worthless rubbish.
- signatoremo 2y agoIf you are to share your code with us, are you sure that we wouldn’t find any worthless rubbish code? If they do exist, can you be certain that they took you less than 5 mins to build like the GP asserted? Take SQLite Wasm as an example: https://simonwillison.net/2024/Oct/21/claude-artifacts/#sqlite-in-wasm-demo https://simonwillison.net/2024/Oct/21/claude-artifacts/#sqli... Perhaps you don’t use database everyday, but a web based sql client is very common, very far from worthless, let alone rubbish. Imagine a developer being able to stand up this module in 5 mins, using it as the starting point for further work.
- mvdtnz 2y agoAre you getting offended on behalf of an "AI"? Seriously mate?
- simonw 2y agoThe purpose of my "14 things I built in the last week" post was not to demonstrate large software - it was to show how the cost of building small applications has effectively fallen close to zero for me. I can knock out small but useful applications in genuinely less time than it would take me to Google for an existing solution to the same problem. You can call them dreck if you like. I call (most of) them useful solutions.