6 ms·

You know that teammate that makes more work for everyone else on the team because they do what they are asked to do but in the most buggy and incomprehensible w

by ryeats 1y ago

You know that teammate that makes more work for everyone else on the team because they do what they are asked to do but in the most buggy and incomprehensible way, that when you finally get them to move on to another team and you realize how much time you spent corralling them and fixing their subtle bugs and now when they are gone work doesn't seem like so much of a chore. That's AI.

threatripper 1y ago
You sound bitter. Did you try using more AI for the bug fixing? It gets better and better.
- ryeats 1y ago
  My interest tend to be bleeding edge where there is little training data. I do use AI to rubber duck but can rarely use it's output directly.
  - threatripper 1y ago
    I see. In my experience current LLMs are great for generating boilerplate code for basic UIs but fail at polishing UI and business logic. If it's important you need to rewrite the core logic completely because they may introduce subtle bugs due to misunderstandings or sloppiness.
    - ryeats 1y ago
      Yep you are also right, some amount of boilerplate code is perfectly reasonable since some problems are similar but just different enough and unique enough they don't merit designing an architecture that gets rid of the boilerplate. this is probably the most useful thing that AI could do for us. I think I am more worried as a maintainer that we won't see that we are copying all that boilerplate too often and it's subtle bugs are multiplied and now we have to maintain all that code because AI doesn't yet do that.
- skydhash 1y ago
  Cognitive load are not related to the difficulty of a task. It’s about how much mental energy is spent monitoring it. To reduce cognitive load, you either boost confidence or avoid caring. You can’t have confidence in AI output and most people proposing it looks like they’re preaching to not care about quality (because quantity yay).
  - threatripper 1y ago
    But quality is going up a lot. Granted, it's not up to human levels yet, but it is going up fast. Also we will see more complex quality control in AI output, tailored to specific use cases and sold at a premium. Right now these don't exist and if they existed it would be too expensive to run 100x requests for the same amount of output. So humans are stuck in quality control, for now.
- Arainach 1y ago
  One of the biggest problems with AI is that it doesn't get better and better. It makes the same mistakes over and over instead of learning like a junior eng would. AI is like the absolute worst outsourced devs I've ever worked with - enthusiastically saying "yes I can do that" to everything and then delivering absolute garbage that takes me longer to fix/convince them to do right than it would have taken for me to just do it myself.
  - threatripper 1y ago
    Current models have no memory, they don't learn. You have to learn for them for now. You have to put the learnings in the instructions and in code comments. If you don't describe WHAT your code SHOULD do and WHY you write it in THAT particular way it will have no idea and the code may just look like bad non-standard code waiting to be "improved". It works best if you keep close to mainstream styles and if you keep it easy and straight-forward.
DavidPiper 1y ago
We need to update Hanlon's Razor: Never attribute to AI that which is adequately explained by incompetence.
- xerox13ster 1y ago
  And just like the original Hanlon’s Razor, this is not an excuse to be stupid or incompetent. It is not a reason to accept stupidity or incompetence. We should reject these things and demand better.
- chistev 1y ago
  Thank you.
0xEF 1y ago
You are not wrong, but I pose the argument that too many people approach Gen AI as a replacement instead of a tool, and therein lies the root of the problem. When I use Claude for code, for example, I am not asking it to write my code. I'm asking it to review what I have written and either suggest improvements or ways to troubleshoot a problem I am having. I also don't always follow its advice, either, but that depends on how much I understand the reply. Sometimes it outputs something that makes sense based on my current skill level, sometimes it proposes things that I know nothing about, in which case I ask it to break it down further so I can go search the Internet for more info and see if I can learn more, which pushes the limits of my skill level. It works well, since my goal is to improve what I bring to the table and I have learned a lot, both about coding and about prompt engineering. When I talk to other people, they accuse me of having the AI do all the work for me because that's how they approach their use of it. They want the AI to produce the whole project, as opposed to just using it as a second brain to offload some mental chunking. That's where Gen AI fails and the user spends all their time correcting convoluted mistakes caused by confabulation, unless they're making a simple monolithic program or script, but even then there's often hiccups. Point is, Gen AI is a great tool, if you approach it with the right mindset. The hammer does not build the whole house, but it can certainly help.
- cardanome 1y ago
  Generative AI is like micromanaging an talented Junior Dev that never improves. And I mean micromanaging to such a toxic degree that not human would ever put up with that. It works but it simply not what most people want. If you love to code then you just abstracted away the most fun parts and have to only do the boring parts now. If you love to manage, well managing actual humans and seeing them grow and become independent is much more fulfilling. On a side note, I feel like prompting and context management is something that is easier for me personally as a person with ADHD as I am already used to working with forms of intelligence that are different to my own. I am used to having to explicitly state my needs. My neurotypical co-workers get frustrated that the LLM can't read their minds and always tell me that it should know what they want. When it nudge them to give it more context and explain better what they need they often resist and say they shouldn't have to. Of course I am stereotyping a bit here but still an interesting observation. Prompting is indeed a skill. Though I believe the skill ceiling will lower once tools get better so I wouldn't bank too much on it. What is going to be valuable for a long time is probably general software architecture skills.
bambax 1y ago
I'm extremely wary of AI myself, especially for creative tasks like writing or making images, etc., but this feels a little over the top. If you let it run wild then yes the result is disaster, but for well defined jobs with a small perimeter AI can save a lot of time.
- runiq 1y ago
  In the context of code, where review bandwidth is the bottleneck, I think it's spot on. In the arts, comparatively -- be they writing, drawing, or music -- you can feel almost at a glance that something is off. There's a bit of a vibe check thing going on, and if that doesn't pass, it's back to the drawing board. You don't inherit technical debt like you do with code.
billy99k 1y ago
You can think that..and you will eventually be left behind. AI is not going anywhere and can be used as a performance booster. Eventually, it will be a requirement for most tech-based jobs.
- deleted 1y ago
  [deleted]
- andersmurphy 1y ago
  This reminds me of crypto’s “have fun being poor”. Except now it’s “have fun being left behind/being unemployed”. The more things change the more things stay the same.
  - mwigdahl 1y ago
    Yes, and it was exactly the same with compilers. All hype and fad -- everyone who's serious about software development writes in assembly.
    - andersmurphy 1y ago
      It's false comparison compilers are deterministic. The only probabilistic behavior I've seen has been for performance (query planning/branch prediction). I mean you're not wrong the serious people drop into assembly when they need too. Even if you work in a context where you can't or don't drop down into assembly being able to make your own compilers is incredibly useful.
      - mwigdahl 1y ago
        Sure, compilers are deterministic and LLMs are not. If you're asserting that a probabilistic process can't get you to a deterministic outcome, Monte Carlo integration would like to have a word. My point was that comparing the rise of AI tooling to the rise of HLL compilers is a much better comparison than comparing it to crypto. HLL compilers were originally seen as crutches and inferior tools and that "real" programmers used assembly. Compiler-generated code was derided as inefficient and ugly. And it was! In the early days, a good programmer who knew the machine could outdo the compiler. But that didn't stop a huge expansion of new programmers who could write COBOL and FORTRAN but never learned assembly. And the compilers got better over time. These days it's a rare wizard who can outdo a compiler's optimizations, and it takes multiple orders of magnitude longer for those rare humans to achieve it. LLM tooling isn't going away. Even in these very early days, it enables non-programmers to construct basic applications that work, using English requests! And the tools have gotten better on almost a monthly basis. You can like them or not like them, just like the early programmers could like or not like compilers. But dismissing them as analogous to empty crypto hype is a bad comparison.
blibble 1y ago
> You know that teammate now imagine he can be scaled indefinitely you thought software was bad today? imagine Microsoft Teams in 5 years time
- ThatMedicIsASpy 1y ago
  I only need to look at the past 5 years of Windows
- darthcircuit 1y ago
  I’m not even looking forward to Microsoft teams on Monday.
Spooky23 1y ago
Just like a poorly managed team, you need to learn how to manage AI to get value from it. All ambiguous processes are like this. In my case, I find the value with LLMs with respect to writing is consolidation. Use it to make outlines, not writing. One example is I record voice memos when driving or jogging and turn them into documents that can be the basis for all sorts of things. End of the day it saves me alot of time and arguably makes me more effective. AI goes bad because it’s not smart, and it will pretend that it is. Figure out the things it does well for your scenario and exploit it.
bdangubic 1y ago
smart people are reading comments like and going “I am glad I am in the same market as people making such comments” :)
- IAmGraydon 1y ago
  I’m glad for now. Understanding how to utilize AI to your advantage is still an edge at the moment, but it won’t be long before almost everyone figures it out.
  - bdangubic 1y ago
    it’ll be years because 87.93% of SWEs are subpar like the post I made comment on.
  - raincole 1y ago
    Yeah. Interestingly enough, I've found utilizing AI is a very shallow skill that anyone should be able to learn in days. But (luckily) people have some tendency preventing them from doing so.
    - bdangubic 1y ago
      with all due respect, this cannot be further from the truth. not only can you not get good in days but it is an ongoing journey. I have spent many, many month learning ins and outs and still spend an hour or two every day on learning/perfecting/…
- ookblah 1y ago
  seriously, the near future is going to be: 1) people who reject it completely for whatever reason. 2) people who use it lazily and produce a lot of garbage (lets be honest, this is probably going to happen a lot which is why maybe group #1 hates this future. reminds me of the outsourcing era) 3) people who selectively use it to their advantage. no point in groups 1 and 3 trying to convince each other of anything.
  - 1y ago

by ryeats 1y ago

threatripper 1y ago

You sound bitter. Did you try using more AI for the bug fixing? It gets better and better.

ryeats 1y ago
My interest tend to be bleeding edge where there is little training data. I do use AI to rubber duck but can rarely use it's output directly.
- threatripper 1y ago
  I see. In my experience current LLMs are great for generating boilerplate code for basic UIs but fail at polishing UI and business logic. If it's important you need to rewrite the core logic completely because they may introduce subtle bugs due to misunderstandings or sloppiness.
  - ryeats 1y ago
    Yep you are also right, some amount of boilerplate code is perfectly reasonable since some problems are similar but just different enough and unique enough they don't merit designing an architecture that gets rid of the boilerplate. this is probably the most useful thing that AI could do for us. I think I am more worried as a maintainer that we won't see that we are copying all that boilerplate too often and it's subtle bugs are multiplied and now we have to maintain all that code because AI doesn't yet do that.
skydhash 1y ago
Cognitive load are not related to the difficulty of a task. It’s about how much mental energy is spent monitoring it. To reduce cognitive load, you either boost confidence or avoid caring. You can’t have confidence in AI output and most people proposing it looks like they’re preaching to not care about quality (because quantity yay).
- threatripper 1y ago
  But quality is going up a lot. Granted, it's not up to human levels yet, but it is going up fast. Also we will see more complex quality control in AI output, tailored to specific use cases and sold at a premium. Right now these don't exist and if they existed it would be too expensive to run 100x requests for the same amount of output. So humans are stuck in quality control, for now.
Arainach 1y ago
One of the biggest problems with AI is that it doesn't get better and better. It makes the same mistakes over and over instead of learning like a junior eng would. AI is like the absolute worst outsourced devs I've ever worked with - enthusiastically saying "yes I can do that" to everything and then delivering absolute garbage that takes me longer to fix/convince them to do right than it would have taken for me to just do it myself.
- threatripper 1y ago
  Current models have no memory, they don't learn. You have to learn for them for now. You have to put the learnings in the instructions and in code comments. If you don't describe WHAT your code SHOULD do and WHY you write it in THAT particular way it will have no idea and the code may just look like bad non-standard code waiting to be "improved". It works best if you keep close to mainstream styles and if you keep it easy and straight-forward.

DavidPiper 1y ago

We need to update Hanlon's Razor: Never attribute to AI that which is adequately explained by incompetence.

xerox13ster 1y ago
And just like the original Hanlon’s Razor, this is not an excuse to be stupid or incompetent. It is not a reason to accept stupidity or incompetence. We should reject these things and demand better.
chistev 1y ago
Thank you.

0xEF 1y ago

You are not wrong, but I pose the argument that too many people approach Gen AI as a replacement instead of a tool, and therein lies the root of the problem. When I use Claude for code, for example, I am not asking it to write my code. I'm asking it to review what I have written and either suggest improvements or ways to troubleshoot a problem I am having. I also don't always follow its advice, either, but that depends on how much I understand the reply. Sometimes it outputs something that makes sense based on my current skill level, sometimes it proposes things that I know nothing about, in which case I ask it to break it down further so I can go search the Internet for more info and see if I can learn more, which pushes the limits of my skill level. It works well, since my goal is to improve what I bring to the table and I have learned a lot, both about coding and about prompt engineering. When I talk to other people, they accuse me of having the AI do all the work for me because that's how they approach their use of it. They want the AI to produce the whole project, as opposed to just using it as a second brain to offload some mental chunking. That's where Gen AI fails and the user spends all their time correcting convoluted mistakes caused by confabulation, unless they're making a simple monolithic program or script, but even then there's often hiccups. Point is, Gen AI is a great tool, if you approach it with the right mindset. The hammer does not build the whole house, but it can certainly help.

cardanome 1y ago
Generative AI is like micromanaging an talented Junior Dev that never improves. And I mean micromanaging to such a toxic degree that not human would ever put up with that. It works but it simply not what most people want. If you love to code then you just abstracted away the most fun parts and have to only do the boring parts now. If you love to manage, well managing actual humans and seeing them grow and become independent is much more fulfilling. On a side note, I feel like prompting and context management is something that is easier for me personally as a person with ADHD as I am already used to working with forms of intelligence that are different to my own. I am used to having to explicitly state my needs. My neurotypical co-workers get frustrated that the LLM can't read their minds and always tell me that it should know what they want. When it nudge them to give it more context and explain better what they need they often resist and say they shouldn't have to. Of course I am stereotyping a bit here but still an interesting observation. Prompting is indeed a skill. Though I believe the skill ceiling will lower once tools get better so I wouldn't bank too much on it. What is going to be valuable for a long time is probably general software architecture skills.

bambax 1y ago

I'm extremely wary of AI myself, especially for creative tasks like writing or making images, etc., but this feels a little over the top. If you let it run wild then yes the result is disaster, but for well defined jobs with a small perimeter AI can save a lot of time.

runiq 1y ago
In the context of code, where review bandwidth is the bottleneck, I think it's spot on. In the arts, comparatively -- be they writing, drawing, or music -- you can feel almost at a glance that something is off. There's a bit of a vibe check thing going on, and if that doesn't pass, it's back to the drawing board. You don't inherit technical debt like you do with code.

billy99k 1y ago

You can think that..and you will eventually be left behind. AI is not going anywhere and can be used as a performance booster. Eventually, it will be a requirement for most tech-based jobs.

deleted 1y ago
[deleted]
andersmurphy 1y ago
This reminds me of crypto’s “have fun being poor”. Except now it’s “have fun being left behind/being unemployed”. The more things change the more things stay the same.
- mwigdahl 1y ago
  Yes, and it was exactly the same with compilers. All hype and fad -- everyone who's serious about software development writes in assembly.
  - andersmurphy 1y ago
    It's false comparison compilers are deterministic. The only probabilistic behavior I've seen has been for performance (query planning/branch prediction). I mean you're not wrong the serious people drop into assembly when they need too. Even if you work in a context where you can't or don't drop down into assembly being able to make your own compilers is incredibly useful.
    - mwigdahl 1y ago
      Sure, compilers are deterministic and LLMs are not. If you're asserting that a probabilistic process can't get you to a deterministic outcome, Monte Carlo integration would like to have a word. My point was that comparing the rise of AI tooling to the rise of HLL compilers is a much better comparison than comparing it to crypto. HLL compilers were originally seen as crutches and inferior tools and that "real" programmers used assembly. Compiler-generated code was derided as inefficient and ugly. And it was! In the early days, a good programmer who knew the machine could outdo the compiler. But that didn't stop a huge expansion of new programmers who could write COBOL and FORTRAN but never learned assembly. And the compilers got better over time. These days it's a rare wizard who can outdo a compiler's optimizations, and it takes multiple orders of magnitude longer for those rare humans to achieve it. LLM tooling isn't going away. Even in these very early days, it enables non-programmers to construct basic applications that work, using English requests! And the tools have gotten better on almost a monthly basis. You can like them or not like them, just like the early programmers could like or not like compilers. But dismissing them as analogous to empty crypto hype is a bad comparison.

blibble 1y ago

> You know that teammate now imagine he can be scaled indefinitely you thought software was bad today? imagine Microsoft Teams in 5 years time

ThatMedicIsASpy 1y ago
I only need to look at the past 5 years of Windows
darthcircuit 1y ago
I’m not even looking forward to Microsoft teams on Monday.

Spooky23 1y ago

Just like a poorly managed team, you need to learn how to manage AI to get value from it. All ambiguous processes are like this. In my case, I find the value with LLMs with respect to writing is consolidation. Use it to make outlines, not writing. One example is I record voice memos when driving or jogging and turn them into documents that can be the basis for all sorts of things. End of the day it saves me alot of time and arguably makes me more effective. AI goes bad because it’s not smart, and it will pretend that it is. Figure out the things it does well for your scenario and exploit it.

bdangubic 1y ago

smart people are reading comments like and going “I am glad I am in the same market as people making such comments” :)

IAmGraydon 1y ago
I’m glad for now. Understanding how to utilize AI to your advantage is still an edge at the moment, but it won’t be long before almost everyone figures it out.
- bdangubic 1y ago
  it’ll be years because 87.93% of SWEs are subpar like the post I made comment on.
- raincole 1y ago
  Yeah. Interestingly enough, I've found utilizing AI is a very shallow skill that anyone should be able to learn in days. But (luckily) people have some tendency preventing them from doing so.
  - bdangubic 1y ago
    with all due respect, this cannot be further from the truth. not only can you not get good in days but it is an ongoing journey. I have spent many, many month learning ins and outs and still spend an hour or two every day on learning/perfecting/…
ookblah 1y ago
seriously, the near future is going to be: 1) people who reject it completely for whatever reason. 2) people who use it lazily and produce a lot of garbage (lets be honest, this is probably going to happen a lot which is why maybe group #1 hates this future. reminds me of the outsourcing era) 3) people who selectively use it to their advantage. no point in groups 1 and 3 trying to convince each other of anything.
- 1y ago