6 ms·

> It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy. Isnt this what code reviews are for? I catch a decent

> It was clearly the work of someone inexperienced walking through walls in an AI-fuelled coding frenzy. Isnt this what code reviews are for? I catch a decent amount of code that looks AI generated. Typically, some very foreign pattern or syntax that this engineers never used nor is common in the codebase. Or something weirdly obtuse that could be refactored and shows a lack of understanding. Normally I ask something like, "Interesting approach! Is there a reason to do it this way over (mention a similar pattern in our codebase)?" or if it's egregious, I might ask, "Can you explain this to me?". This feel similar to early career engineers copy pasting stack overflow code. Now its just faster and easier for them to do. It's still fairly easy to spot though.

adastra22 2y ago
Not all organizations can afford code reviews.
- AlexandrB 2y ago
  If you're trying to do things efficiently you can't afford not doing code reviews. Only well-funded organizations can afford to write the same code twice after the first attempt has to be thrown out.
  - rixed 2y ago
    I don't understand. Is code review supposed to avoid writing coffee twice?
    - aaronbrethorst 2y ago
      I recommend drinking coffee before requesting a code review or posting on an online forum.
  - nox101 2y ago
    I work at a FANMG company and see the value of code reviews for th code we're working on. But, I shipped 20 commercial products before that with no code review, just play testing (games). Our velocity without code review was easily 2x what it is with. For those projects I think that would still be true whereas I'd never suggest getting rid of code review at my current job. The point being I think it depends on the project, it's size, audience, lifetime, etc whether or not code review is a win. my gut says it usually is a win but not always
    - PoignardAzur 2y ago
      That's super weird. At the one company I was where we did code reviews, they took at most one hour per day. The reviewer would go through the changes, ask for explanations when they didn't understand something, suggest small modifications to fit the codebase's style/structure/vibe, and move on. They definitely didn't take 50% of our time. Half my time wasn't even spent coding, it was spent thinking about problems (and it was a workplace with an unusually high coding-over-thinking ratio, because of its good practices).
      - 331c8c71 2y ago
        > At the one company I was where we did code reviews, they took at most one hour per day. > They definitely didn't take 50% of our time. Imo introducing code reviews might very well decrease the velocity by 50% even if the actual time spent on code reviews is much less. I see at least two reasons for this: 1) increased need for synchronization/communication 2) increased subjective frictions and mental overhead (if only for task switching). Code reviews are not special in this respect, similar effects can be caused by any changes to the process.
        strogonoff 2y ago
        Thorough code reviews are expensive. When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing. Less time may be required if you have good rapport and long work relationship with the person, but I would say halving productivity sounds about right if we are talking mandatory reviews. Using an LLM to generate code means having to do the same. The longer the autocompleted text, the more careful you have to be vetting it. Lower predictability, compared to a person and especially one you know well, means that time requirement would not go down too much.
        deergomoo 2y ago
        > When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing Agree 100%, even when LLMs aren’t involved. Certainly not all code reviews are like this—in many cases the approach is fundamentally sound, but you might have some suggestions to improve clarity or performance. But where I work it’s not all that rare to open up a PR and find that the person has gone full speed in the wrong direction. In those cases you’re losing at least a morning to do a lot of the same leg work they did to find a solution for the problem, to then get them set off again down a better path. Now in the ideal case they understood the problem they needed to solve and perhaps just didn’t have the knowledge or experience to know why their initial solution wasn’t good; in such a scenario the second attempt is usually much quicker and much higher quality. But introduce LLMs into the mix and you can make no guarantees about what, if anything, was understood. That’s where things really start to go off the rails imo.
  - dkersten 2y ago
    Not just thrown out, but also the customer relations when the bugs affect customers, the investigation to find the cause of the bugs, the migrations to fix any bad/corrupted data because of the bugs. Code review is a far cheaper way to catch errors, bad code, footguns and places where bugs might creep in than letting them into production and having them impact customers. Code reviews also help keep code tidier and cleaner which makes future understanding and changes easier. Small companies can’t afford to omit code reviews.
- kragen 2y ago
  usually 'organization' excludes projects consisting of just one person, and if there are two people working on a project that involves writing code, the cheapest amount of that code for both of them to be familiar with is never zero, though it is often less than 100% of all the code they write. so i don't think that 'not all organizations can afford code reviews' is true, interpreted in the usual way. maybe you can be more specific about the scenario you're thinking of?
  - adastra22 2y ago
    As mentioned in the other comments, code reviews are typically not done in the game industry, for example, or other write-once, never read scenarios. Very early stage startups typically don’t have formal code reviews either.
    - kragen 2y ago
      while that is true, and some of it is rational, other times it's just another stupid way people waste money and fuck up their projects. because even in a startup or a game there is code that people waste time debugging alone, or waiting for the one guy who knows it to get back from vacation, or out of bed reviews don't have to be formal to be reviews
- littlestymaar 2y ago
  No organization can afford not having code reviews.
  - adastra22 2y ago
    No blanket statements apply to all organizations.
    - littlestymaar 2y ago
      “No organization can work in absence of oxygen” proves you wrong.
- seattle_spring 2y ago
  Can you elaborate on some scenarios you are describing? The only organization I can imagine that can't do code reviews would be organizations of 1. I don't even see how performing code reviews could ever be a net negative in terms of money nor overall productivity.
  - ben_w 2y ago
    Not OP, but I've been in a few organisations with many devs but also where I was the only iOS dev — the other devs being Android, back-end, or web. I would have preferred to have someone to collaborate with at each of those places.
    - oytis 2y ago
      I think it should be possible (and encouraged) for an Android developer to know enough about ios to do review and vice versa.
      - JTyQZSnP3cQGa8B 2y ago
        Or pair-programming reviews. It doesn't matter if you don't know a language, it's still very useful.
    - sensanaty 2y ago
      Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
    - sensanaty 2y ago
      Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
- AlwaysRock 2y ago
  Those organizations shouldn't complain about code quality.
steve1977 2y ago
> Isnt this what code reviews are for? Wouldn't you need to have people with a proper understanding of the programming language and framework to do the code reviews?
- fhd2 2y ago
  You would. And if the juniors don't really _write_ code in the languages you use, if they don't make mistakes and do research themselves, it'll take them a lot longer to learn them sufficiently to be able to do those reviews. Listening to people talk in Japanese, I picked up enough to have some idea what they're talking about. I can't speak the language beyond some canned phrases. I certainly wouldn't claim to know Japanese. And I definitely wouldn't get a job writing books in Japanese, armed with Google Translate. For programming, my solution to this right now is lots of pair programming. Really gives you a good idea of where somebody is at, what they still need to learn, and lots of teaching opportunities. I just hired a junior and we spend about 8 hours a week pairing.
- AlwaysRock 2y ago
  Yeah. Are we assuming 100% of the people working on this project are incompetent? I thought OP was talking about new people joining a project producing low quality code that was likely generated by AI. Code reviews, done by existing more senior members of the project/team, should prevent this. If it isnt then the project has more issues than AI generated code.
arrowsmith 2y ago
> It's still fairly easy to spot though. Or maybe it's far more common than you realise, and you're only spotting the obvious ones.
- AlwaysRock 2y ago
  Sure. If its generating 85% code that passes a code review and 15% of code that doesnt... I'll take those percentages. That is certainly higher than the average early career engineers pr.
vouaobrasil 2y ago
> Isnt this what code reviews are for? There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down. No thanks.
- CuriouslyC 2y ago
  I would agree with there being no substitute for doing something correctly the first place, but I would argue that in this case the "first place" is hiring/training better so your employees don't try to throw unrefined AI shit at the walls, but instead take the output of AI and hone it before creating a PR. If you have an engineering culture that doesn't emphasize thorough code review (at least of juniors, leads and architects emergency-pushing is a different story) that's a problem. In addition to catching bugs, that's a major vector for passing on knowledge.
  - vouaobrasil 2y ago
    I agree with you, no doubt there. But still, technology often offers the path of least resistance. So even if you want to hire better people, what of all the people who will grow up training themselves with Copilot? Yes, you can filter them out but it will be harder and harder to find good people. And then, companies will want to go after the bottom line: what if they can get a bigger and more complex product out there with AI assistance but with code that is more unreliable in the long term? I tell you, it's spiralling out of control. Besides, even if you're doing the hiring, you may not have control if there is a profit motive. Us technical people cannot control the race to the bottom line, especially over the period of decades.
ungreased0675 2y ago
>Isnt this what code reviews are for? A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.
- throwup238 2y ago
  Code review isn’t inspection, that would be testing - whether automatic or manual. Code review is the feedback mechanism that allows quality control to improve the process, teaching the engineers how to do it right in the first place.
bcrosby95 2y ago
It's harder to read code than write it. It's even harder to read code than use an LLM to write it for you.
- AlwaysRock 2y ago
  Yup. That is why code reviews are important... Read and understand the code that is going out before it goes out.

adastra22 2y ago

Not all organizations can afford code reviews.

AlexandrB 2y ago
If you're trying to do things efficiently you can't afford not doing code reviews. Only well-funded organizations can afford to write the same code twice after the first attempt has to be thrown out.
- rixed 2y ago
  I don't understand. Is code review supposed to avoid writing coffee twice?
  - aaronbrethorst 2y ago
    I recommend drinking coffee before requesting a code review or posting on an online forum.
- nox101 2y ago
  I work at a FANMG company and see the value of code reviews for th code we're working on. But, I shipped 20 commercial products before that with no code review, just play testing (games). Our velocity without code review was easily 2x what it is with. For those projects I think that would still be true whereas I'd never suggest getting rid of code review at my current job. The point being I think it depends on the project, it's size, audience, lifetime, etc whether or not code review is a win. my gut says it usually is a win but not always
  - PoignardAzur 2y ago
    That's super weird. At the one company I was where we did code reviews, they took at most one hour per day. The reviewer would go through the changes, ask for explanations when they didn't understand something, suggest small modifications to fit the codebase's style/structure/vibe, and move on. They definitely didn't take 50% of our time. Half my time wasn't even spent coding, it was spent thinking about problems (and it was a workplace with an unusually high coding-over-thinking ratio, because of its good practices).
    - 331c8c71 2y ago
      > At the one company I was where we did code reviews, they took at most one hour per day. > They definitely didn't take 50% of our time. Imo introducing code reviews might very well decrease the velocity by 50% even if the actual time spent on code reviews is much less. I see at least two reasons for this: 1) increased need for synchronization/communication 2) increased subjective frictions and mental overhead (if only for task switching). Code reviews are not special in this respect, similar effects can be caused by any changes to the process.
      - strogonoff 2y ago
        Thorough code reviews are expensive. When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing. Less time may be required if you have good rapport and long work relationship with the person, but I would say halving productivity sounds about right if we are talking mandatory reviews. Using an LLM to generate code means having to do the same. The longer the autocompleted text, the more careful you have to be vetting it. Lower predictability, compared to a person and especially one you know well, means that time requirement would not go down too much.
        deergomoo 2y ago
        > When I review code, I have to know how to solve the problem well in the first place—that is, do all of the effort that went into writing the original code under review, minus time spent typing Agree 100%, even when LLMs aren’t involved. Certainly not all code reviews are like this—in many cases the approach is fundamentally sound, but you might have some suggestions to improve clarity or performance. But where I work it’s not all that rare to open up a PR and find that the person has gone full speed in the wrong direction. In those cases you’re losing at least a morning to do a lot of the same leg work they did to find a solution for the problem, to then get them set off again down a better path. Now in the ideal case they understood the problem they needed to solve and perhaps just didn’t have the knowledge or experience to know why their initial solution wasn’t good; in such a scenario the second attempt is usually much quicker and much higher quality. But introduce LLMs into the mix and you can make no guarantees about what, if anything, was understood. That’s where things really start to go off the rails imo.
- dkersten 2y ago
  Not just thrown out, but also the customer relations when the bugs affect customers, the investigation to find the cause of the bugs, the migrations to fix any bad/corrupted data because of the bugs. Code review is a far cheaper way to catch errors, bad code, footguns and places where bugs might creep in than letting them into production and having them impact customers. Code reviews also help keep code tidier and cleaner which makes future understanding and changes easier. Small companies can’t afford to omit code reviews.
kragen 2y ago
usually 'organization' excludes projects consisting of just one person, and if there are two people working on a project that involves writing code, the cheapest amount of that code for both of them to be familiar with is never zero, though it is often less than 100% of all the code they write. so i don't think that 'not all organizations can afford code reviews' is true, interpreted in the usual way. maybe you can be more specific about the scenario you're thinking of?
- adastra22 2y ago
  As mentioned in the other comments, code reviews are typically not done in the game industry, for example, or other write-once, never read scenarios. Very early stage startups typically don’t have formal code reviews either.
  - kragen 2y ago
    while that is true, and some of it is rational, other times it's just another stupid way people waste money and fuck up their projects. because even in a startup or a game there is code that people waste time debugging alone, or waiting for the one guy who knows it to get back from vacation, or out of bed reviews don't have to be formal to be reviews
littlestymaar 2y ago
No organization can afford not having code reviews.
- adastra22 2y ago
  No blanket statements apply to all organizations.
  - littlestymaar 2y ago
    “No organization can work in absence of oxygen” proves you wrong.
seattle_spring 2y ago
Can you elaborate on some scenarios you are describing? The only organization I can imagine that can't do code reviews would be organizations of 1. I don't even see how performing code reviews could ever be a net negative in terms of money nor overall productivity.
- ben_w 2y ago
  Not OP, but I've been in a few organisations with many devs but also where I was the only iOS dev — the other devs being Android, back-end, or web. I would have preferred to have someone to collaborate with at each of those places.
  - oytis 2y ago
    I think it should be possible (and encouraged) for an Android developer to know enough about ios to do review and vice versa.
    - JTyQZSnP3cQGa8B 2y ago
      Or pair-programming reviews. It doesn't matter if you don't know a language, it's still very useful.
  - sensanaty 2y ago
    Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
  - sensanaty 2y ago
    Even if you're not working in the same language/framework/whatever, you can still do code reviews. You might not know all the syntax and high-level details, but even just an overview of the logic happening in the code is already better than 0 review of any kind
AlwaysRock 2y ago
Those organizations shouldn't complain about code quality.

steve1977 2y ago

> Isnt this what code reviews are for? Wouldn't you need to have people with a proper understanding of the programming language and framework to do the code reviews?

fhd2 2y ago
You would. And if the juniors don't really _write_ code in the languages you use, if they don't make mistakes and do research themselves, it'll take them a lot longer to learn them sufficiently to be able to do those reviews. Listening to people talk in Japanese, I picked up enough to have some idea what they're talking about. I can't speak the language beyond some canned phrases. I certainly wouldn't claim to know Japanese. And I definitely wouldn't get a job writing books in Japanese, armed with Google Translate. For programming, my solution to this right now is lots of pair programming. Really gives you a good idea of where somebody is at, what they still need to learn, and lots of teaching opportunities. I just hired a junior and we spend about 8 hours a week pairing.
AlwaysRock 2y ago
Yeah. Are we assuming 100% of the people working on this project are incompetent? I thought OP was talking about new people joining a project producing low quality code that was likely generated by AI. Code reviews, done by existing more senior members of the project/team, should prevent this. If it isnt then the project has more issues than AI generated code.

arrowsmith 2y ago

> It's still fairly easy to spot though. Or maybe it's far more common than you realise, and you're only spotting the obvious ones.

AlwaysRock 2y ago
Sure. If its generating 85% code that passes a code review and 15% of code that doesnt... I'll take those percentages. That is certainly higher than the average early career engineers pr.

vouaobrasil 2y ago

> Isnt this what code reviews are for? There is no substitute to doing something correctly in the first place. The problem is that in the real world, deadlines and lack of time will always cause the default solution to be accepted a small percentage of time even when it is not ideal. The increasing creep of AI will only exacerbate that and most technophiles will default into thinking of a new and improved AI tool to help with the problem, until it will be AI tools all the way down. No thanks.

CuriouslyC 2y ago
I would agree with there being no substitute for doing something correctly the first place, but I would argue that in this case the "first place" is hiring/training better so your employees don't try to throw unrefined AI shit at the walls, but instead take the output of AI and hone it before creating a PR. If you have an engineering culture that doesn't emphasize thorough code review (at least of juniors, leads and architects emergency-pushing is a different story) that's a problem. In addition to catching bugs, that's a major vector for passing on knowledge.
- vouaobrasil 2y ago
  I agree with you, no doubt there. But still, technology often offers the path of least resistance. So even if you want to hire better people, what of all the people who will grow up training themselves with Copilot? Yes, you can filter them out but it will be harder and harder to find good people. And then, companies will want to go after the bottom line: what if they can get a bigger and more complex product out there with AI assistance but with code that is more unreliable in the long term? I tell you, it's spiralling out of control. Besides, even if you're doing the hiring, you may not have control if there is a profit motive. Us technical people cannot control the race to the bottom line, especially over the period of decades.

ungreased0675 2y ago

>Isnt this what code reviews are for? A foundational concept of quality control is to not rely on inspection to catch production defects. Why not? It diffuses responsibility, lets more problems get to the customer and is less efficient than doing it correctly to start with.

throwup238 2y ago
Code review isn’t inspection, that would be testing - whether automatic or manual. Code review is the feedback mechanism that allows quality control to improve the process, teaching the engineers how to do it right in the first place.

bcrosby95 2y ago

It's harder to read code than write it. It's even harder to read code than use an LLM to write it for you.

AlwaysRock 2y ago
Yup. That is why code reviews are important... Read and understand the code that is going out before it goes out.