4 ms·

> Or generating 4 versions of PR for the same task so that the human could just pick the best one. That sounds awful. A truly terrible and demotivating way to

by latexr 1y ago

> Or generating 4 versions of PR for the same task so that the human could just pick the best one. That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality. Why are we doing this to ourselves and embracing it? A few years ago, it would have been seen as a joke to say “the future of software development will be to have a million monkey interns banging on one million keyboards and submit a million PRs, then choose one”. Today, it’s lauded as a brilliant business and cost-saving idea. We’re beyond doomed. The first major catastrophe caused by sloppy AI code can’t come soon enough. The sooner it happens, the better chance we have to self-correct.

bonoboTP 1y ago
If it's monkeylike quality and you need a million tries, it's shit. It you need four tries and one of those is top-tier professional programmer quality, then it's good.
- agos 1y ago
  if the thing producing the four PRs can't distinguish the top tier one, I have strong doubts that it can even produce it
  - solaire_oa 1y ago
    Making 4 PRs for a well-known solution sounds insane, yes, but to be the devil's advocate, you could plausibly be working with an ambiguous task: "Create 4 PRs with 4 different dependency libraries, so that I can compare their implementations." Technically it wouldn't need to pick the best one. I have apprehension about the future of software engineering, but comparison does technically seem like a valid use case.
- deleted 1y ago
  [deleted]
- layer8 1y ago
  The problem is, for any change, you have to understand the existing code base to assess the quality of the change in the four tries. This means, you aren’t relieved from being familiar with the code and reviewing everything. For many developers this review-only work style isn’t an exciting prospect. And it will remain that way until you can delegate development tasks to AI with a 99+% success rate so that you don’t have to review their output and understand the code base anymore. At which point developers will become truly obsolete.
- solaire_oa 1y ago
  Top-tier professional programmer quality is exceedingly, impractically optimistic, for a few reasons. 1. There's a low probability of that in the first place. 2. You need to be a top-tier professional programmer to recognize that type of quality (i.e. a junior engineer could select one of the 3 shit PRs) 3. When it doesn't produce TTPPQ, you wasted tons of time prompting and reviewing shit code and still need to deliver, net negative. I'm not doubting the utility of LLMs but the scattershot approach just feels like gambling to me.
  - zelphirkalt 1y ago
    Also as a consequence of (1) the LLMs are trained on mediocre code mostly, so they often output mediocre or bad solutions.
diggan 1y ago
> A truly terrible and demotivating way to work and produce anything of real quality You clearly have strong feelings about it, which is fine, but it would be much more interesting to know exactly why it would terrible and demotivating, and why it cannot produce anything of quality? And what is "real quality" and does that mean "fake quality" exists? > million monkey interns banging on one million keyboards and submit a million PRs I'm not sure if you misunderstand LLMs, or the famous "monkeys writing Shakespeare" part, but that example is more about randomness and infinity than about probabilistic machines somewhat working towards a goal with some non-determinism. > We’re beyond doomed The good news is that we've been doomed for a long time, yet we persist. If you take a look at how the internet is basically held up by duct-tape at this point, I think you'd feel slightly more comfortable with how crap absolutely everything is. Like 1% of software is actually Good Software while the rest barely works on a good day.
- 3dsnano 1y ago
  > And what is "real quality" and does that mean "fake quality" exists? I think there is no real quality or fake quality, just quality. I am referencing the quality that Persig and C. Alexander have written about. It’s… qualitative, so it’s hard to measure but easy to feel. Humans are really good at perceiving it then making objective decisions. LLMs don’t know what it is (they’ve heard about it and think they know).
  - abdullin 1y ago
    It is actually funny that current AI+Coding tools benefit a lot from domain context and other information along the lines of Domain-Driven Design (which was inspired by the pattern language of C. Alexander). A few teams have started incorporating `CONTEXT.MD` into module descriptions to leverage this.
  - diggan 1y ago
    > LLMs don’t know what it is Of course they don't, they're probability/prediction machines, they don't "know" anything, not even that Paris is the capital of France. What they do "know" is that once someone writes "The capital of France is", the most likely tokens to come after that, is "Paris". But they don't understand the concept, nor anything else, just that probably 54123 comes after 6723 (or whatever the tokens are). Once you understand this, I think it's easy to reason about why they don't understand code quality, why they couldn't ever understand it, and how you can make them output quality code regardless.
koakuma-chan 1y ago
> That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality This is the right way to work with generative AI, and it already is an extremely common and established practice when working with image generation.
- notTooFarGone 1y ago
  I can recognize images in one look. How about that 400 Line change that touches 7 files?
  - koakuma-chan 1y ago
    In my prompt I ask the LLM to write a short summary of how it solved the problem, run multiple instances of LLM concurrently, compare their summaries, and use the output of whichever LLM seems to have interpreted instructions the best, or arrived at the best solution.
    - elt895 1y ago
      And you trust that the summary matches what was actually done? Your experience with the level of LLMs understanding of code changes must significantly differ from mine.
      - koakuma-chan 1y ago
        It matched every time so far.
  - abdullin 1y ago
    Exactly! This is why there has to be "write me a detailed implementation plan" step in between. Which files is it going to change, how, what are the gotchas, which tests will be affected or added etc. It is easier to review one document and point out missing bits, than chase the loose ends. Once the plan is done and good, it is usually a smooth path to the PR.
osigurdson 1y ago
I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code. Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.
- elif 1y ago
  Yep this is what Andrej talks about around 20 minutes into this talk. You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. The second you start being vague, even if it WOULD be clear to a person with common sense, the LLM views that vagueness as a potential aspect of it's own creative liberty.
  - jebarker 1y ago
    > the LLM views that vagueness as a potential aspect of it's own creative liberty. I think that anthropomorphism actually clouds what’s going on here. There’s no creative choice inside an LLM. More description in the prompt just means more constraints on the latent space. You still have no certainty whether the LLM models the particular part of the world you’re constraining it to in the way you hope it does though.
  - 9rx 1y ago
    > You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. If only there was a language one could use that enables describing all of your requirements in a unambiguous manner, ensuring that you have provided all the necessary detail. Oh wait.
  - joshuahedlund 1y ago
    > You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail I understand YMMV, but I have yet to find a use case where this takes me less time than writing the code myself.
  - 1y ago
ponector 1y ago
>That sounds awful. Not for the cloud provider. AWS bill to the moon!
chamomeal 1y ago
I say this all the time! Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit. Also I can’t really imagine that in the first place. At my current job, each task is like 95% understanding all the little bits, and then 5% writing the code. If you’re reviewing PRs from a bot all day, you’ll still need to understand all the bits before you accept it. So how much time is that really gonna save?
- diggan 1y ago
  > Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit. On the other hand, does anyone really wanna be a code-monkey implementing CRUD applications over and over by following product specifications by "product managers" that barely seem to understand the product they're "managing"? See, we can make bad faith arguments both ways, but what's the point?
  - nevertoolate 1y ago
    Issue is if product people will do the “coding” and you have to fix it is miserable
    - diggan 1y ago
      Even worse would be if we asked the accountants to do the coding, then you'll learn what miserable means. What was the point again?
      - nevertoolate 1y ago
        Yes
  - consumer451 1y ago
    I hesitate to divide a group as diverse as software devs into two categories, but here I go: I have a feeling that devs who love LLM coding tools are more product-driven than those who hate them. Put another way, maybe devs with their own product ideas love LLM coding tools, whilr devs without them do not. I am genuinely not trying to throw shade here in any way. Does this rough division ring true to anyone else? Is there any better way to put it?

by latexr 1y ago

bonoboTP 1y ago

If it's monkeylike quality and you need a million tries, it's shit. It you need four tries and one of those is top-tier professional programmer quality, then it's good.

agos 1y ago
if the thing producing the four PRs can't distinguish the top tier one, I have strong doubts that it can even produce it
- solaire_oa 1y ago
  Making 4 PRs for a well-known solution sounds insane, yes, but to be the devil's advocate, you could plausibly be working with an ambiguous task: "Create 4 PRs with 4 different dependency libraries, so that I can compare their implementations." Technically it wouldn't need to pick the best one. I have apprehension about the future of software engineering, but comparison does technically seem like a valid use case.
deleted 1y ago
[deleted]
layer8 1y ago
The problem is, for any change, you have to understand the existing code base to assess the quality of the change in the four tries. This means, you aren’t relieved from being familiar with the code and reviewing everything. For many developers this review-only work style isn’t an exciting prospect. And it will remain that way until you can delegate development tasks to AI with a 99+% success rate so that you don’t have to review their output and understand the code base anymore. At which point developers will become truly obsolete.
solaire_oa 1y ago
Top-tier professional programmer quality is exceedingly, impractically optimistic, for a few reasons. 1. There's a low probability of that in the first place. 2. You need to be a top-tier professional programmer to recognize that type of quality (i.e. a junior engineer could select one of the 3 shit PRs) 3. When it doesn't produce TTPPQ, you wasted tons of time prompting and reviewing shit code and still need to deliver, net negative. I'm not doubting the utility of LLMs but the scattershot approach just feels like gambling to me.
- zelphirkalt 1y ago
  Also as a consequence of (1) the LLMs are trained on mediocre code mostly, so they often output mediocre or bad solutions.

diggan 1y ago

> A truly terrible and demotivating way to work and produce anything of real quality You clearly have strong feelings about it, which is fine, but it would be much more interesting to know exactly why it would terrible and demotivating, and why it cannot produce anything of quality? And what is "real quality" and does that mean "fake quality" exists? > million monkey interns banging on one million keyboards and submit a million PRs I'm not sure if you misunderstand LLMs, or the famous "monkeys writing Shakespeare" part, but that example is more about randomness and infinity than about probabilistic machines somewhat working towards a goal with some non-determinism. > We’re beyond doomed The good news is that we've been doomed for a long time, yet we persist. If you take a look at how the internet is basically held up by duct-tape at this point, I think you'd feel slightly more comfortable with how crap absolutely everything is. Like 1% of software is actually Good Software while the rest barely works on a good day.

3dsnano 1y ago
> And what is "real quality" and does that mean "fake quality" exists? I think there is no real quality or fake quality, just quality. I am referencing the quality that Persig and C. Alexander have written about. It’s… qualitative, so it’s hard to measure but easy to feel. Humans are really good at perceiving it then making objective decisions. LLMs don’t know what it is (they’ve heard about it and think they know).
- abdullin 1y ago
  It is actually funny that current AI+Coding tools benefit a lot from domain context and other information along the lines of Domain-Driven Design (which was inspired by the pattern language of C. Alexander). A few teams have started incorporating `CONTEXT.MD` into module descriptions to leverage this.
- diggan 1y ago
  > LLMs don’t know what it is Of course they don't, they're probability/prediction machines, they don't "know" anything, not even that Paris is the capital of France. What they do "know" is that once someone writes "The capital of France is", the most likely tokens to come after that, is "Paris". But they don't understand the concept, nor anything else, just that probably 54123 comes after 6723 (or whatever the tokens are). Once you understand this, I think it's easy to reason about why they don't understand code quality, why they couldn't ever understand it, and how you can make them output quality code regardless.

koakuma-chan 1y ago

> That sounds awful. A truly terrible and demotivating way to work and produce anything of real quality This is the right way to work with generative AI, and it already is an extremely common and established practice when working with image generation.

notTooFarGone 1y ago
I can recognize images in one look. How about that 400 Line change that touches 7 files?
- koakuma-chan 1y ago
  In my prompt I ask the LLM to write a short summary of how it solved the problem, run multiple instances of LLM concurrently, compare their summaries, and use the output of whichever LLM seems to have interpreted instructions the best, or arrived at the best solution.
  - elt895 1y ago
    And you trust that the summary matches what was actually done? Your experience with the level of LLMs understanding of code changes must significantly differ from mine.
    - koakuma-chan 1y ago
      It matched every time so far.
- abdullin 1y ago
  Exactly! This is why there has to be "write me a detailed implementation plan" step in between. Which files is it going to change, how, what are the gotchas, which tests will be affected or added etc. It is easier to review one document and point out missing bits, than chase the loose ends. Once the plan is done and good, it is usually a smooth path to the PR.

osigurdson 1y ago

I'm not sure that AI code has to be sloppy. I've had some success with hand coding some examples and then asking codex to rigorously adhere to prior conventions. This can end up with very self consistent code. Agree though on the "pick the best PR" workflow. This is pure model training work and you should be compensated for it.

elif 1y ago
Yep this is what Andrej talks about around 20 minutes into this talk. You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. The second you start being vague, even if it WOULD be clear to a person with common sense, the LLM views that vagueness as a potential aspect of it's own creative liberty.
- jebarker 1y ago
  > the LLM views that vagueness as a potential aspect of it's own creative liberty. I think that anthropomorphism actually clouds what’s going on here. There’s no creative choice inside an LLM. More description in the prompt just means more constraints on the latent space. You still have no certainty whether the LLM models the particular part of the world you’re constraining it to in the way you hope it does though.
- 9rx 1y ago
  > You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail. If only there was a language one could use that enables describing all of your requirements in a unambiguous manner, ensuring that you have provided all the necessary detail. Oh wait.
- joshuahedlund 1y ago
  > You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail I understand YMMV, but I have yet to find a use case where this takes me less time than writing the code myself.
- 1y ago

ponector 1y ago

>That sounds awful. Not for the cloud provider. AWS bill to the moon!

chamomeal 1y ago

I say this all the time! Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit. Also I can’t really imagine that in the first place. At my current job, each task is like 95% understanding all the little bits, and then 5% writing the code. If you’re reviewing PRs from a bot all day, you’ll still need to understand all the bits before you accept it. So how much time is that really gonna save?

diggan 1y ago
> Does anybody really want to be an assembly line QA reviewer for an automated code factory? Sounds like shit. On the other hand, does anyone really wanna be a code-monkey implementing CRUD applications over and over by following product specifications by "product managers" that barely seem to understand the product they're "managing"? See, we can make bad faith arguments both ways, but what's the point?
- nevertoolate 1y ago
  Issue is if product people will do the “coding” and you have to fix it is miserable
  - diggan 1y ago
    Even worse would be if we asked the accountants to do the coding, then you'll learn what miserable means. What was the point again?
    - nevertoolate 1y ago
      Yes
- consumer451 1y ago
  I hesitate to divide a group as diverse as software devs into two categories, but here I go: I have a feeling that devs who love LLM coding tools are more product-driven than those who hate them. Put another way, maybe devs with their own product ideas love LLM coding tools, whilr devs without them do not. I am genuinely not trying to throw shade here in any way. Does this rough division ring true to anyone else? Is there any better way to put it?