3 ms·

> LLM’s never provide code that pass my sniff test If that statement isn't coming from ego, then where is it coming from? It's provably true that LLM's can gen

> LLM’s never provide code that pass my sniff test If that statement isn't coming from ego, then where is it coming from? It's provably true that LLM's can generate working code. They've been trained on billions of examples. Developers seem to focus on the set of cases that LLM's produce code that doesn't work, and use that as evidence that these tools are "useless".

selfmodruntime 2y ago
> If that statement isn't coming from ego, then where is it coming from? It's provably true that LLM's can generate working code. They've been trained on billions of examples. I am pro AI and I'm probably even overvaluing the value AI brings. However, for me, this doesn't work in more "esoteric" programming languages or those with stricter rulesets like Rust. LLMS provide fine JS code, since there's no compiler to satisfy, but CPP without undefined behaviour or compiling Rust code is rare. There's also no chance of LLMS providing compiling code if you're using a library version with a newer API than the one in the training set.
hmillison 2y ago
there's a lot more involved in senior dev work beyond producing code that works. if the stakeholders knew how to do what they needed to build and how, then they could use LLMs, but translating complex requirements into code is something that these tools are not even close to cracking.
- rybosworld 2y ago
  > there's a lot more involved in senior dev work beyond producing code that works. Completely agree. What I don't agree with is statements like these: > LLM’s never provide code that pass my sniff test To me, these (false) absolutions about chat bot capabilities, are being rehashed so frequently, that it derails every conversation about using LLM's for dev work. You'll find similar statements in nearly every thread about LLM's for coding tasks. It's provably true that LLM's can produce working code. It's also true, that some increasingly large portion of coding is being offloaded to LLM's. In my opinion, developers need to grow out of this attitude that they are John Henry and they'll outpace the mechanical drilling machine. It's a tired conversation.
  - JTyQZSnP3cQGa8B 2y ago
    > It's provably true that LLM's can produce working code Then why can't I see this magical code that is produced? I mean a real big application with a purpose and multiple dependencies, not yet another ReactJS todo list. I've seen comments like that a hundred times already but not one repository that could be equivalent to what I currently do. For me the experience of LLM is a bad tool that calls functions that are obsolete or do not exist at all, not very earth-shattering.
  - rurp 2y ago
    > It's provably true that LLM's can produce working code. You've restated this point several times but the reason it's not more convincing to many people is that simply producing code that works is rarely an actual goal on many projects. On larger projects it's much more about producing code that is consistent with the rest of the project, and is easily extensible, and is readable for your teammates, and is easy to debug when something goes wrong, is testable, and so on. The code working is a necessary condition, but is insufficient to tell if it's a valuable contribution.
  - munk-a 2y ago
    > It's provably true that LLM's can produce working code. This is correct - but it's also true that LLMs can produce flawed code. To me the cost of telling whether code is correct or flawed is larger than the cost of me just writing correct code. This may be an AuDHD thing but I can better comprehend the correctness of a solution if I'm watching (and doing) the making of that solution than if I'm reading it after the fact.
  - simianparrot 2y ago
    The code working is the bare minimum. The code being right for the project and context is the basic expectation. The code being _good_ at solving its intended problem is the desired outcome, which is a combination of tradeoffs between performance, readability, ease of refactoring later, modularity, etc. LLM's can sometimes provide the bare minimum. And then you have to refactor and massage it all the way to the good bit, but unlike looking up other people's endeavors on something like Stack Overflow, with the LLM's code I have no context why it "thought" that was a good idea. If I ask it, it may parrot something from the relevant training set, or it might be bullshitting completely. The end result? This is _more_ work for a senior dev, not less. Hence why it has never passed my sniff test. Its code is at best the quality of code even junior developers wouldn't open a PR for yet. Or if they did they'd be asked to explain how and why and quickly learn to not open the code for review before they've properly considered the implications.
- pooper 2y ago
  > if the stakeholders knew how to do what they needed to build and how, then they could use LLMs, but translating complex requirements into code is something that these tools are not even close to cracking. They don't have to replace you to reduce headcount. They could increase your workload so where they needed five senior developers, they can do with maybe three. That's like six one way and half a dozen the other way because two developers lost a job, right?
- n4r9 2y ago
  Yeah. Code that works is a fraction of the aim. You also want code that a good junior can read and debug in the midst of a production issue, is robust against new or updated requirements, has at least as good performance as the competitors, and uses appropriate libraries in a sparse manner. You also need to be able to state when a requirement would loosen the conceptual cohesion of the code, and to push back on requirements thdt can already be achieved in just as easy a way.
oops 2y ago
My experience so far has been: if I know what I want well enough to explain it to an LLM then it’s been easier for me to just write the code. Iterating on prompts, reading and understanding the LLM’s code, validating that it works and fixing bugs is still time consuming. It has been interesting as a rubber duck, exploring a new topic or language, some code golf, but so far not for production code for me.
- qup 2y ago
  Okay, but as soon as you need to do the same thing in [programming language you don't know], then it's not easier for you to write the code anymore, even though you understand the problem domain just as well. Now, understand that most people don't have the same grasp of [your programming language] that you have, so it's probably not easier for them to write it.
  - oops 2y ago
    I don’t disagree with anything you said and I don’t think anything you said disagrees with my original comment :) I actually said in my comment that exploring a new language is one area I find LLMs to be interesting.
IshKebab 2y ago
> It's provably true that LLM's can generate working code. Yeah for simple examples, especially in web dev. As soon as you step outside those bounds they make mistakes all the time. As I said, they're still useful because roughly correct but buggy code is often quite helpful when you're programming. But there's zero chance you can just say "write me an driver for the nRF905 using Embassy and embedded-hal" and get something working. Whereas I, a human, can do that.
- fragmede 2y ago
  The question is, how long would it take you to get as far as this chat does when starting from scratch? https://chatgpt.com/share/6760c3b3-bae8-8009-8744-c25d5602bfb4 https://chatgpt.com/share/6760c3b3-bae8-8009-8744-c25d5602bf...
  - IshKebab 2y ago
    Yeah now actually check the nRF905 documentation. You'll find it has basically made everything up.
  - bccdee 2y ago
    How confident are you in the correctness of that code? Because, one way or another, you're still going to need to become fluent enough in the problem domain & the API that you can fully review the implementation and make sure chatgpt hasn't hallucinated in any weird problems. And chatgpt can't really explain its own work, so if anything seems funny, you're going to have to sleuth it out yourself. And at that point, it's just 339 lines of code, including imports, comments, and misc formatting. How much time have you really saved?
ben_w 2y ago
> It's provably true that LLM's can generate working code. What I've seen of them, the good ones mostly produce OK code. Not terrible, usually works. Although I like them even for that low-ish bar, although I find them to be both a time-saver and a personal motivation assistant, they're still a thing that needs a real domain expert to spot the mistakes they make. > Developers seem to focus on the set of cases that LLM's produce code that doesn't work, and use that as evidence that these tools are "useless". I do find it amusing how many humans turned out to be stuck thinking in boolean terms, dismissing the I in AGI, calling them as "useless" because it "can't take my job". Same with the G in AGI, dismissing the breadth of something that speaks 50 languages when humans who speak five or six languages are considered unusually skilled.
mplanchard 2y ago
Working code some subset of the time, for popular languages. It’s not good at Rust nor at other smaller languages. Tried to ask it for some help with Janet and it was hopelessly wrong, even with prompting to try to get it to correct its mistakes. Even if it did work, working code is barely half the battle.
- BoxOfRain 2y ago
  I'd say ChatGPT at least is a fair bit better at Python than it is at Scala which seems to match your experience.
deleted 2y ago
[deleted]
bccdee 2y ago
> It's provably true that LLM's can generate working code They produce mostly working code, often with odd design decisions that the bot can't fully justify. The difficult part of coding is cultivating a mental model of the problem + solution space. When you let an LLM write code for you, your mental model falls behind. You can read the new code closely, internalize it, double-check the docs, and keep your mental model up to date (which takes longer than you think), or you can forge ahead, confident that it all more or less looks right. The second option is easier, faster, and very tempting, and it is the reason why various studies have found that code written with LLM assistance introduces more bugs than code written without. There are plenty of innovations that have made programming a little bit faster (autocomplete, garbage collection, what-have-you), but none of them were a silver bullet. LLMs aren't either. The essential complexity of the work hasn't changed, and a chat bot can't manage it for you. In my (admittedly limited) experience with code assistants, I've found that the faster you move with an LLM, the more time you have to spend debugging afterwards, and the more difficult that process becomes.

selfmodruntime 2y ago

> If that statement isn't coming from ego, then where is it coming from? It's provably true that LLM's can generate working code. They've been trained on billions of examples. I am pro AI and I'm probably even overvaluing the value AI brings. However, for me, this doesn't work in more "esoteric" programming languages or those with stricter rulesets like Rust. LLMS provide fine JS code, since there's no compiler to satisfy, but CPP without undefined behaviour or compiling Rust code is rare. There's also no chance of LLMS providing compiling code if you're using a library version with a newer API than the one in the training set.

hmillison 2y ago

there's a lot more involved in senior dev work beyond producing code that works. if the stakeholders knew how to do what they needed to build and how, then they could use LLMs, but translating complex requirements into code is something that these tools are not even close to cracking.

rybosworld 2y ago
> there's a lot more involved in senior dev work beyond producing code that works. Completely agree. What I don't agree with is statements like these: > LLM’s never provide code that pass my sniff test To me, these (false) absolutions about chat bot capabilities, are being rehashed so frequently, that it derails every conversation about using LLM's for dev work. You'll find similar statements in nearly every thread about LLM's for coding tasks. It's provably true that LLM's can produce working code. It's also true, that some increasingly large portion of coding is being offloaded to LLM's. In my opinion, developers need to grow out of this attitude that they are John Henry and they'll outpace the mechanical drilling machine. It's a tired conversation.
- JTyQZSnP3cQGa8B 2y ago
  > It's provably true that LLM's can produce working code Then why can't I see this magical code that is produced? I mean a real big application with a purpose and multiple dependencies, not yet another ReactJS todo list. I've seen comments like that a hundred times already but not one repository that could be equivalent to what I currently do. For me the experience of LLM is a bad tool that calls functions that are obsolete or do not exist at all, not very earth-shattering.
- rurp 2y ago
  > It's provably true that LLM's can produce working code. You've restated this point several times but the reason it's not more convincing to many people is that simply producing code that works is rarely an actual goal on many projects. On larger projects it's much more about producing code that is consistent with the rest of the project, and is easily extensible, and is readable for your teammates, and is easy to debug when something goes wrong, is testable, and so on. The code working is a necessary condition, but is insufficient to tell if it's a valuable contribution.
- munk-a 2y ago
  > It's provably true that LLM's can produce working code. This is correct - but it's also true that LLMs can produce flawed code. To me the cost of telling whether code is correct or flawed is larger than the cost of me just writing correct code. This may be an AuDHD thing but I can better comprehend the correctness of a solution if I'm watching (and doing) the making of that solution than if I'm reading it after the fact.
- simianparrot 2y ago
  The code working is the bare minimum. The code being right for the project and context is the basic expectation. The code being _good_ at solving its intended problem is the desired outcome, which is a combination of tradeoffs between performance, readability, ease of refactoring later, modularity, etc. LLM's can sometimes provide the bare minimum. And then you have to refactor and massage it all the way to the good bit, but unlike looking up other people's endeavors on something like Stack Overflow, with the LLM's code I have no context why it "thought" that was a good idea. If I ask it, it may parrot something from the relevant training set, or it might be bullshitting completely. The end result? This is _more_ work for a senior dev, not less. Hence why it has never passed my sniff test. Its code is at best the quality of code even junior developers wouldn't open a PR for yet. Or if they did they'd be asked to explain how and why and quickly learn to not open the code for review before they've properly considered the implications.
pooper 2y ago
> if the stakeholders knew how to do what they needed to build and how, then they could use LLMs, but translating complex requirements into code is something that these tools are not even close to cracking. They don't have to replace you to reduce headcount. They could increase your workload so where they needed five senior developers, they can do with maybe three. That's like six one way and half a dozen the other way because two developers lost a job, right?
n4r9 2y ago
Yeah. Code that works is a fraction of the aim. You also want code that a good junior can read and debug in the midst of a production issue, is robust against new or updated requirements, has at least as good performance as the competitors, and uses appropriate libraries in a sparse manner. You also need to be able to state when a requirement would loosen the conceptual cohesion of the code, and to push back on requirements thdt can already be achieved in just as easy a way.

oops 2y ago

My experience so far has been: if I know what I want well enough to explain it to an LLM then it’s been easier for me to just write the code. Iterating on prompts, reading and understanding the LLM’s code, validating that it works and fixing bugs is still time consuming. It has been interesting as a rubber duck, exploring a new topic or language, some code golf, but so far not for production code for me.

qup 2y ago
Okay, but as soon as you need to do the same thing in [programming language you don't know], then it's not easier for you to write the code anymore, even though you understand the problem domain just as well. Now, understand that most people don't have the same grasp of [your programming language] that you have, so it's probably not easier for them to write it.
- oops 2y ago
  I don’t disagree with anything you said and I don’t think anything you said disagrees with my original comment :) I actually said in my comment that exploring a new language is one area I find LLMs to be interesting.

IshKebab 2y ago

> It's provably true that LLM's can generate working code. Yeah for simple examples, especially in web dev. As soon as you step outside those bounds they make mistakes all the time. As I said, they're still useful because roughly correct but buggy code is often quite helpful when you're programming. But there's zero chance you can just say "write me an driver for the nRF905 using Embassy and embedded-hal" and get something working. Whereas I, a human, can do that.

fragmede 2y ago
The question is, how long would it take you to get as far as this chat does when starting from scratch? https://chatgpt.com/share/6760c3b3-bae8-8009-8744-c25d5602bfb4 https://chatgpt.com/share/6760c3b3-bae8-8009-8744-c25d5602bf...
- IshKebab 2y ago
  Yeah now actually check the nRF905 documentation. You'll find it has basically made everything up.
- bccdee 2y ago
  How confident are you in the correctness of that code? Because, one way or another, you're still going to need to become fluent enough in the problem domain & the API that you can fully review the implementation and make sure chatgpt hasn't hallucinated in any weird problems. And chatgpt can't really explain its own work, so if anything seems funny, you're going to have to sleuth it out yourself. And at that point, it's just 339 lines of code, including imports, comments, and misc formatting. How much time have you really saved?

ben_w 2y ago

> It's provably true that LLM's can generate working code. What I've seen of them, the good ones mostly produce OK code. Not terrible, usually works. Although I like them even for that low-ish bar, although I find them to be both a time-saver and a personal motivation assistant, they're still a thing that needs a real domain expert to spot the mistakes they make. > Developers seem to focus on the set of cases that LLM's produce code that doesn't work, and use that as evidence that these tools are "useless". I do find it amusing how many humans turned out to be stuck thinking in boolean terms, dismissing the I in AGI, calling them as "useless" because it "can't take my job". Same with the G in AGI, dismissing the breadth of something that speaks 50 languages when humans who speak five or six languages are considered unusually skilled.

mplanchard 2y ago

Working code some subset of the time, for popular languages. It’s not good at Rust nor at other smaller languages. Tried to ask it for some help with Janet and it was hopelessly wrong, even with prompting to try to get it to correct its mistakes. Even if it did work, working code is barely half the battle.

BoxOfRain 2y ago
I'd say ChatGPT at least is a fair bit better at Python than it is at Scala which seems to match your experience.

deleted 2y ago

[deleted]

bccdee 2y ago

> It's provably true that LLM's can generate working code They produce mostly working code, often with odd design decisions that the bot can't fully justify. The difficult part of coding is cultivating a mental model of the problem + solution space. When you let an LLM write code for you, your mental model falls behind. You can read the new code closely, internalize it, double-check the docs, and keep your mental model up to date (which takes longer than you think), or you can forge ahead, confident that it all more or less looks right. The second option is easier, faster, and very tempting, and it is the reason why various studies have found that code written with LLM assistance introduces more bugs than code written without. There are plenty of innovations that have made programming a little bit faster (autocomplete, garbage collection, what-have-you), but none of them were a silver bullet. LLMs aren't either. The essential complexity of the work hasn't changed, and a chat bot can't manage it for you. In my (admittedly limited) experience with code assistants, I've found that the faster you move with an LLM, the more time you have to spend debugging afterwards, and the more difficult that process becomes.