7 ms·

> You're not supposed to trust the tool This is just an incredible statement. I can't think of another development tool we'd say this about. I'm not saying you

> You're not supposed to trust the tool This is just an incredible statement. I can't think of another development tool we'd say this about. I'm not saying you're wrong, or that it's wrong to have tools we can't just, just... wow... what a sea change.

theonething 1y ago
> I can't think of another development tool we'd say this about. Because no other dev tool actually generates unique code like AI does. So you treat it like the other components of your team that generates code, the other developers. Do you trust other developers to write good code without mistakes without getting it reviewed by others. Of course not.
- anonymars 1y ago
  I trust my colleagues to write code that compiles, at the very least
  - ModernMech 1y ago
    Oh at the very least I trust them to not take code that compiles and immediately assess that it's broken.
- chrisweekly 1y ago
  But of course everyone absolutely NEEDS to use AI for codereviews! How else could the huge volume of AI-generated code be managed?
- seabird 1y ago
  Yes, actually, I do! I trust my teammates with tens of thousands of hours of experience in programming, embedded hardware, our problem spaces, etc. to write from a fully formed worldview, and for their code to work as intended (as far as anybody can tell before it enters preliminary testing by users) by the time the rest of the team reviews it. Most code review is uneventful. Have some pride in your work and you'll be amazed at what's possible.
  - theonething 1y ago
    so your saying that yes you do "trust other developers to write good code without mistakes without getting it reviewed by others." And then you say "by the time the rest of the team reviews it. Most code review is uneventful." So you trust your team to develop without the need for code review but yet, your team does code review. So what is the purpose of these code reviews? Is it the case that you actually don't think they are necessary, but perhaps management insists on them? You actually answer this question yourself: > Most code review is uneventful. Keyword here is "most" as opposed to "all" So based your team's applied practices and your own words, code review is for the purpose of catching mistakes and other needed corrections. But it seems to me if you trust your team not to make mistakes, code review is superfluous. As an aside, it seems your team culture doesn't make room for juniors because if your team had juniors I think it would be even more foolish to trust them not to make mistakes. Maybe a junior free culture works for your company, but that's not the case for every company. My main point is code review is not superfluous no matter the skill level; junior, senior, or AI simply because everyone and every AI makes mistakes. So I don't trust those three classes of code emitters to not ever make mistakes or bad choices (i.e. be perfect) and therefore I think code review is useful. Have some honesty and humility and you'll amazed at what's possible.
    - seabird 1y ago
      I never said that code review was useless, I said "yes, I do" to your question as to whether or not I "trust other developers to write good code without mistakes without getting it reviewed by others". Of course I can trust them to do the right thing even when nobody's looking, and review it anyway in the off-chance they overlooked something. I can't trust AI to do that. The purpose of the review is to find and fix occasional small details before it goes to physical testing. It does not involve constant babysitting of the developer. It's a little silly to bring up honesty when you spent that entire comment dancing around the reality that AI makes an inordinately large number of mistakes. I will pick the domain expert who refuses to touch AI over a generic programmer with access to it ten times out of ten. The entire team as it is now (me included) were juniors. It's a traditional engineering environment in a location where people don't aggressively move between jobs at the drop of a hat. You don't need to constantly train younger developers when you can retain people.
- forgetfreeman 1y ago
  "Do you trust other developers to write good code without mistakes without getting it reviewed by others." Literally yes. Test coverage and QA to catch bugs sure but needing everything manually reviewed by someone else sounds like working in a sweatshop full of intern-level code bootcamp graduates, or if you prefer an absolute dumpster fire of incompetence.
  - ryandrake 1y ago
    I would accept mistakes and inconsistency from a human, especially one not very experienced or skilled. But I expect perfection and consistency from a machine. When I command my computer to do something, I expect it to do it correctly, the same way every time, to convert a particular input to an exact particular output, every time. I don't expect it to guess, or randomly insert garbage, or behave non-deterministically. Those things are called defects(bugs) and I'd want them to be fixed.
    - senordevnyc 1y ago
      Then you are going to hate the future.
      - ryandrake 1y ago
        Way ahead of you. I already hate the present, at least the current sad state of the software industry.
    - forgetfreeman 1y ago
      Exactly this.
    - tevon 1y ago
      This seems like a particularly limited view of what a machine is. Specifically expecting it to behave deterministically.
      - ModernMech 1y ago
ryandrake 1y ago
Imagine if your compiler just randomly and non-deterministically compiled valid code to incorrect binaries, and the tool's developer couldn't really tell you why it happens, how often it was expected to happen, how severe the problem was expected to be, and told you to just not trust your compiler to create correct machine code. Imagine if your calculator app randomly and non-deterministically performed arithmetic incorrectly, and you similarly couldn't get correctness expectations from the developer. Imagine if any of your communication tools randomly and non-deterministically translated your messages into gibberish... I think we'd all throw away such tools, but we are expected to accept it if it's an "AI tool?"
- deleted 1y ago
  [deleted]
- andrei_says_ 1y ago
  Imagine that you yourself never use these tools directly but your employees do. And the sellers of said tools swear that the tools are amazing and correct and will save you millions. They keep telling you that any employee who highlights problems with the tools are just trying to save their job. Your investors tell you that the toolmakers are already saving money for your competitors. Now, do you want that second house and white lotus vacation or not? Making good tools is difficult. Bending perception (“is reality”) is easier and enterprise sales, just like good propaganda, work. The gold rush will leave a lot of bodies behind but the shovelmakers will make a killing.
  - ModernMech 1y ago
    I feel like there's a lot of motivated reasoning going on, yeah.
- ToValueFunfetti 1y ago
  If the only calculators that existed failed at 5% of the calculations, or if the only communication tools miscommunicated 5% of the time, we would still use both all the time. They would be far less than 95% as useful as perfect versions, but drastically better then not having the tools at all.
shipp02 1y ago
In Mechanical Engineering, this is 100% a thing with fluid dynamics simulation. You need to know if the output is BS based on a number of factors that I don't understand.
tevon 1y ago
Stackoverflow is like this, you read an answer but are not fully sure if its right or if it fits your needs. Of course there is a review system for a reason, but we frequently use "untrusted" tools in development. That one guy in a github issue that said "this worked for me"
ModernMech 1y ago
Imagine! Imagine if 0.05% of the time gcc just injected random code into your binaries. Imagine, you swing a hammer and 1% of the time it just phases into the wall. Tools are supposed to be reliable.
- arvinsim 1y ago
  There are no existing AI tools that guarantee correct code 100% of the time. If there is such a tool, programmers will be on path of immediate reskilling or lose their jobs very quickly.

theonething 1y ago

> I can't think of another development tool we'd say this about. Because no other dev tool actually generates unique code like AI does. So you treat it like the other components of your team that generates code, the other developers. Do you trust other developers to write good code without mistakes without getting it reviewed by others. Of course not.

anonymars 1y ago
I trust my colleagues to write code that compiles, at the very least
- ModernMech 1y ago
  Oh at the very least I trust them to not take code that compiles and immediately assess that it's broken.
chrisweekly 1y ago
But of course everyone absolutely NEEDS to use AI for codereviews! How else could the huge volume of AI-generated code be managed?
seabird 1y ago
Yes, actually, I do! I trust my teammates with tens of thousands of hours of experience in programming, embedded hardware, our problem spaces, etc. to write from a fully formed worldview, and for their code to work as intended (as far as anybody can tell before it enters preliminary testing by users) by the time the rest of the team reviews it. Most code review is uneventful. Have some pride in your work and you'll be amazed at what's possible.
- theonething 1y ago
  so your saying that yes you do "trust other developers to write good code without mistakes without getting it reviewed by others." And then you say "by the time the rest of the team reviews it. Most code review is uneventful." So you trust your team to develop without the need for code review but yet, your team does code review. So what is the purpose of these code reviews? Is it the case that you actually don't think they are necessary, but perhaps management insists on them? You actually answer this question yourself: > Most code review is uneventful. Keyword here is "most" as opposed to "all" So based your team's applied practices and your own words, code review is for the purpose of catching mistakes and other needed corrections. But it seems to me if you trust your team not to make mistakes, code review is superfluous. As an aside, it seems your team culture doesn't make room for juniors because if your team had juniors I think it would be even more foolish to trust them not to make mistakes. Maybe a junior free culture works for your company, but that's not the case for every company. My main point is code review is not superfluous no matter the skill level; junior, senior, or AI simply because everyone and every AI makes mistakes. So I don't trust those three classes of code emitters to not ever make mistakes or bad choices (i.e. be perfect) and therefore I think code review is useful. Have some honesty and humility and you'll amazed at what's possible.
  - seabird 1y ago
    I never said that code review was useless, I said "yes, I do" to your question as to whether or not I "trust other developers to write good code without mistakes without getting it reviewed by others". Of course I can trust them to do the right thing even when nobody's looking, and review it anyway in the off-chance they overlooked something. I can't trust AI to do that. The purpose of the review is to find and fix occasional small details before it goes to physical testing. It does not involve constant babysitting of the developer. It's a little silly to bring up honesty when you spent that entire comment dancing around the reality that AI makes an inordinately large number of mistakes. I will pick the domain expert who refuses to touch AI over a generic programmer with access to it ten times out of ten. The entire team as it is now (me included) were juniors. It's a traditional engineering environment in a location where people don't aggressively move between jobs at the drop of a hat. You don't need to constantly train younger developers when you can retain people.
forgetfreeman 1y ago
"Do you trust other developers to write good code without mistakes without getting it reviewed by others." Literally yes. Test coverage and QA to catch bugs sure but needing everything manually reviewed by someone else sounds like working in a sweatshop full of intern-level code bootcamp graduates, or if you prefer an absolute dumpster fire of incompetence.
- ryandrake 1y ago
  I would accept mistakes and inconsistency from a human, especially one not very experienced or skilled. But I expect perfection and consistency from a machine. When I command my computer to do something, I expect it to do it correctly, the same way every time, to convert a particular input to an exact particular output, every time. I don't expect it to guess, or randomly insert garbage, or behave non-deterministically. Those things are called defects(bugs) and I'd want them to be fixed.
  - senordevnyc 1y ago
    Then you are going to hate the future.
    - ryandrake 1y ago
      Way ahead of you. I already hate the present, at least the current sad state of the software industry.
  - forgetfreeman 1y ago
    Exactly this.
  - tevon 1y ago
    This seems like a particularly limited view of what a machine is. Specifically expecting it to behave deterministically.
    - ModernMech 1y ago

ryandrake 1y ago

Imagine if your compiler just randomly and non-deterministically compiled valid code to incorrect binaries, and the tool's developer couldn't really tell you why it happens, how often it was expected to happen, how severe the problem was expected to be, and told you to just not trust your compiler to create correct machine code. Imagine if your calculator app randomly and non-deterministically performed arithmetic incorrectly, and you similarly couldn't get correctness expectations from the developer. Imagine if any of your communication tools randomly and non-deterministically translated your messages into gibberish... I think we'd all throw away such tools, but we are expected to accept it if it's an "AI tool?"

deleted 1y ago
[deleted]
andrei_says_ 1y ago
Imagine that you yourself never use these tools directly but your employees do. And the sellers of said tools swear that the tools are amazing and correct and will save you millions. They keep telling you that any employee who highlights problems with the tools are just trying to save their job. Your investors tell you that the toolmakers are already saving money for your competitors. Now, do you want that second house and white lotus vacation or not? Making good tools is difficult. Bending perception (“is reality”) is easier and enterprise sales, just like good propaganda, work. The gold rush will leave a lot of bodies behind but the shovelmakers will make a killing.
- ModernMech 1y ago
  I feel like there's a lot of motivated reasoning going on, yeah.
ToValueFunfetti 1y ago
If the only calculators that existed failed at 5% of the calculations, or if the only communication tools miscommunicated 5% of the time, we would still use both all the time. They would be far less than 95% as useful as perfect versions, but drastically better then not having the tools at all.

shipp02 1y ago

In Mechanical Engineering, this is 100% a thing with fluid dynamics simulation. You need to know if the output is BS based on a number of factors that I don't understand.

tevon 1y ago

Stackoverflow is like this, you read an answer but are not fully sure if its right or if it fits your needs. Of course there is a review system for a reason, but we frequently use "untrusted" tools in development. That one guy in a github issue that said "this worked for me"

ModernMech 1y ago

Imagine! Imagine if 0.05% of the time gcc just injected random code into your binaries. Imagine, you swing a hammer and 1% of the time it just phases into the wall. Tools are supposed to be reliable.

arvinsim 1y ago
There are no existing AI tools that guarantee correct code 100% of the time. If there is such a tool, programmers will be on path of immediate reskilling or lose their jobs very quickly.