3 ms·

> Meanwhile I asked AI to make me a test and it mocked out everything I wanted to test, testing nothing, but passing. I wonder how much of these kinds of tests

> Meanwhile I asked AI to make me a test and it mocked out everything I wanted to test, testing nothing, but passing. I wonder how much of these kinds of tests we have now This sort of thing is what I'm most worried about with programmers who are very bullish on AI. I don't think it's too controversial to say that most developers are much worse at reviewing code than they are at writing code AI generated code is a code review exercise. If you accept my premise that most devs are worse at reviewing code than writing it, this should ring some alarm bells

icedchai 2y ago
You're spot on about developers being worse at reviewing code. With generated code, you still need to understand it if you want to maintain it. I had another person send me some AI generated code that was close to 90% working. It was missing something simple (something like appending to an array instead of overwriting it...) The original developer could not understand or debug it. I'm afraid of the crap we're going to see over the next few years.
tfandango 2y ago
This is a great point. This mocked test, easy to literally click a button, look like you did something good, but potentially add massive negative value when the code fails but the tests all pass. It can be a very helpful tool, but you need to understand the code it produces, and it seems like people are missing that point.
- bluefirebrand 2y ago
  It is the "we copied this snippet from stack overflow without understanding it" problem, except turned up to incredibly high volumes
bakuninsbart 2y ago
There are multiple factors exacerbating this even further: 1) AI generated code is "soulless". Reviewing code can be engaging if you are trying to understand your colleagues thought process. Sometimes you learn something new, sometimes you can feel superior, or politely disagree. AI ain't worth the hassle. 2) You unlearn how to write code by relying heavily on LLMs. On some level, it is similar to an IDE giving you context clues and definitions. On another, it can replace the hard work of thinking things through with mediocre yet usually workable solutions. And that's the big trap.
- bluefirebrand 2y ago
  > Sometimes you learn something new, sometimes you can feel superior, or politely disagree. AI ain't worth the hassle. Often I am looking for opportunities to mentor a less experienced developer Why bother if they are just offloading their thinking to AI
  - rickydroll 2y ago
    > Why bother if they are just offloading their thinking to AI Because you get to make them cry when you show them the error of their AI ways. Consider it a teaching moment if you can show them why the AI-generated code is insufficient for production.
wjholden 2y ago
You could make an analogy to the engineer babysitting a self-driving car during testing: you'd need to be a better driver than most to recognize when the machine is about to make a mistake and intervene.
christkv 2y ago
Pff we will just use another AI to do the review /s
carlmr 2y ago
I think I'm somewhat decent at code reviewing, what I'm seeing currently is though that the PRs become larger and larger. Because AI just rewrites everything, even if you tell it to fix things. This leads to devs pushing more and bigger code changes. I'm convinced no company has enough senior developers to review these vast amounts of sloppy code.

icedchai 2y ago

You're spot on about developers being worse at reviewing code. With generated code, you still need to understand it if you want to maintain it. I had another person send me some AI generated code that was close to 90% working. It was missing something simple (something like appending to an array instead of overwriting it...) The original developer could not understand or debug it. I'm afraid of the crap we're going to see over the next few years.

tfandango 2y ago

This is a great point. This mocked test, easy to literally click a button, look like you did something good, but potentially add massive negative value when the code fails but the tests all pass. It can be a very helpful tool, but you need to understand the code it produces, and it seems like people are missing that point.

bluefirebrand 2y ago
It is the "we copied this snippet from stack overflow without understanding it" problem, except turned up to incredibly high volumes

bakuninsbart 2y ago

There are multiple factors exacerbating this even further: 1) AI generated code is "soulless". Reviewing code can be engaging if you are trying to understand your colleagues thought process. Sometimes you learn something new, sometimes you can feel superior, or politely disagree. AI ain't worth the hassle. 2) You unlearn how to write code by relying heavily on LLMs. On some level, it is similar to an IDE giving you context clues and definitions. On another, it can replace the hard work of thinking things through with mediocre yet usually workable solutions. And that's the big trap.

bluefirebrand 2y ago
> Sometimes you learn something new, sometimes you can feel superior, or politely disagree. AI ain't worth the hassle. Often I am looking for opportunities to mentor a less experienced developer Why bother if they are just offloading their thinking to AI
- rickydroll 2y ago
  > Why bother if they are just offloading their thinking to AI Because you get to make them cry when you show them the error of their AI ways. Consider it a teaching moment if you can show them why the AI-generated code is insufficient for production.

wjholden 2y ago

You could make an analogy to the engineer babysitting a self-driving car during testing: you'd need to be a better driver than most to recognize when the machine is about to make a mistake and intervene.

christkv 2y ago

Pff we will just use another AI to do the review /s

carlmr 2y ago

I think I'm somewhat decent at code reviewing, what I'm seeing currently is though that the PRs become larger and larger. Because AI just rewrites everything, even if you tell it to fix things. This leads to devs pushing more and bigger code changes. I'm convinced no company has enough senior developers to review these vast amounts of sloppy code.