4 ms·

That is already the case for me. The amount of times I've read "apologies for the oversight, you are absolutely correct" is staggering: 8 or 9 out of 10 times.

That is already the case for me. The amount of times I've read "apologies for the oversight, you are absolutely correct" is staggering: 8 or 9 out of 10 times. Meanwhile I constantly see people mindlessly copy paying llm generated code and subsequently furious when it doesn't do what they expected it to do. Which, btw, is the better option: I'd rather have something obviously broken as opposed to something seemingly working.

autobodie 1y ago
In my experience, LLMs are extremely inclined to modify code just to pass tests instead of meeting requirements.
- fwip 1y ago
  When they're not modifying the tests to match buggy behavior. :P
devjab 1y ago
Are you using the LLM's through a browser chatbot? Because the AI-agents we use with direct code-access aren't very chatty. I'd also argue that they are more capable than a lot of junior programmers, at least around here. We're almost at a point where you can feed the agents short specific tasks, and they will perform them well enough to not really require anything outside of a code review. That being said, the prediction engine still can't do any real engineering. If you don't specifically task them with using things like Python generators, you're very likely to have a piece of code that eats up a gazillion memory. Which unfortunately don't set them appart from a lot of Python programmers I know, but it is an example of how the LLM's are exactly as bad as you mention. On the positive side, it helps with people actually writing the specification tasks in more detail than just "add feature". Where AI-agents are the most useful for us is with legacy code that nobody prioritise. We have a data extractor which was written in the previous millennium. It basically uses around two hunded hard-coded coordinates to extact data from a specific type of documents which arrive by fax. It's worked for 30ish years because the documents haven't changed... but it recently did, and it took co-pilot like 30 seconds to correct the coordinates. Something that would've likely taken a human a full day of excruciating boredom. I have no idea how our industry expect anyone to become experts in the age of vibe coding though.
- teeray 1y ago
  > Because the AI-agents we use with direct code-access aren't very chatty. So they’re even more confident in their wrongness
  - devjab 1y ago
    When there is less chat they appear less confident, but I think you're pretty spot on in point out why they are dangerous. If you're using them in an area where you're an expert, it's very easy to review the "diff" suggestion they come up with and decide whether it's bullshit or not. If you're using them in an area where you're not an expert, then how will you know?
- furyofantares 1y ago
  > Because the AI-agents we use with direct code-access aren't very chatty. Every time I tell claude code something it did is wrong, or might be wrong, or even just ask a leading question about a potential bug it just wrote, it leads with "You're absolutely correct!" before even invoking any tools. Maybe you've just become used to ignoring this. I mostly ignore it but it is a bit annoying when I'm trying to use the agent to help me figure out if the code it wrote is correct, so I ask it some question it should be capable of helping with and it leads with "you're absolutely correct". I didn't make a proposition that can be correct or not, and it didn't do any work yet to to investigate my question - it feels like it has poisoned its own context by leading with this.
  - devjab 1y ago
    It may have to do with workflow. I rarely talk with the AI agent, I task it with a VIBE.md or a specific outlined prompt that relates to inline "COPILOT: ...." comments, and then I review the changes and either keep or dismiss them. When I dismiss them I'll mostly rewrite the promt and do it again in a new context window. I did get curious though. So I decided to look up some of the times where I did correct it after I dismissed a change. I only looked at a couple of prompts but most of the AI responses looked like this: "There are two issues...", "The error is because...", "The error persists because...", "A new route, /class_ids/fully_owned, has been added...". I was feeling confident that it wasn't bullshitting me at that point, but then I get to this one: "Thank you for the details. The error response..." Now, that is the AI agent. If I use the browser or one of their "apps" the LLM politeness encouragement bullshit alone will often be longer than the entire chat response in an agent. Like this is the entire response to what it was tasked with in my example: "To add this route, I'll implement a new endpoint that queries all unique class_id values and checks if all items with that class_id have is_owned == True. The route will return a list of such class_id values. I'll add this as a new GET route, e.g., /class_ids/fully_owned." ChatGPT (which is supposedly the same model) will spend those lines telling me what a great question it was.
mexicocitinluez 1y ago
> 8 or 9 out of 10 times. Not they don't. This is 100% a made up statistic.
- bluefirebrand 1y ago
  It isn't even being presented as a statistic it is someone saying what they have experienced
  - mexicocitinluez 1y ago
    nice. i'll add this to the list of "totally useless replies".
  - mexicocitinluez 1y ago
    not to respond again but whats even funnier is that IT IS a statistic. saying "90% of the time I get a certain response" is literally a statistic.

autobodie 1y ago

In my experience, LLMs are extremely inclined to modify code just to pass tests instead of meeting requirements.

fwip 1y ago
When they're not modifying the tests to match buggy behavior. :P

devjab 1y ago

Are you using the LLM's through a browser chatbot? Because the AI-agents we use with direct code-access aren't very chatty. I'd also argue that they are more capable than a lot of junior programmers, at least around here. We're almost at a point where you can feed the agents short specific tasks, and they will perform them well enough to not really require anything outside of a code review. That being said, the prediction engine still can't do any real engineering. If you don't specifically task them with using things like Python generators, you're very likely to have a piece of code that eats up a gazillion memory. Which unfortunately don't set them appart from a lot of Python programmers I know, but it is an example of how the LLM's are exactly as bad as you mention. On the positive side, it helps with people actually writing the specification tasks in more detail than just "add feature". Where AI-agents are the most useful for us is with legacy code that nobody prioritise. We have a data extractor which was written in the previous millennium. It basically uses around two hunded hard-coded coordinates to extact data from a specific type of documents which arrive by fax. It's worked for 30ish years because the documents haven't changed... but it recently did, and it took co-pilot like 30 seconds to correct the coordinates. Something that would've likely taken a human a full day of excruciating boredom. I have no idea how our industry expect anyone to become experts in the age of vibe coding though.

teeray 1y ago
> Because the AI-agents we use with direct code-access aren't very chatty. So they’re even more confident in their wrongness
- devjab 1y ago
  When there is less chat they appear less confident, but I think you're pretty spot on in point out why they are dangerous. If you're using them in an area where you're an expert, it's very easy to review the "diff" suggestion they come up with and decide whether it's bullshit or not. If you're using them in an area where you're not an expert, then how will you know?
furyofantares 1y ago
> Because the AI-agents we use with direct code-access aren't very chatty. Every time I tell claude code something it did is wrong, or might be wrong, or even just ask a leading question about a potential bug it just wrote, it leads with "You're absolutely correct!" before even invoking any tools. Maybe you've just become used to ignoring this. I mostly ignore it but it is a bit annoying when I'm trying to use the agent to help me figure out if the code it wrote is correct, so I ask it some question it should be capable of helping with and it leads with "you're absolutely correct". I didn't make a proposition that can be correct or not, and it didn't do any work yet to to investigate my question - it feels like it has poisoned its own context by leading with this.
- devjab 1y ago
  It may have to do with workflow. I rarely talk with the AI agent, I task it with a VIBE.md or a specific outlined prompt that relates to inline "COPILOT: ...." comments, and then I review the changes and either keep or dismiss them. When I dismiss them I'll mostly rewrite the promt and do it again in a new context window. I did get curious though. So I decided to look up some of the times where I did correct it after I dismissed a change. I only looked at a couple of prompts but most of the AI responses looked like this: "There are two issues...", "The error is because...", "The error persists because...", "A new route, /class_ids/fully_owned, has been added...". I was feeling confident that it wasn't bullshitting me at that point, but then I get to this one: "Thank you for the details. The error response..." Now, that is the AI agent. If I use the browser or one of their "apps" the LLM politeness encouragement bullshit alone will often be longer than the entire chat response in an agent. Like this is the entire response to what it was tasked with in my example: "To add this route, I'll implement a new endpoint that queries all unique class_id values and checks if all items with that class_id have is_owned == True. The route will return a list of such class_id values. I'll add this as a new GET route, e.g., /class_ids/fully_owned." ChatGPT (which is supposedly the same model) will spend those lines telling me what a great question it was.

mexicocitinluez 1y ago

> 8 or 9 out of 10 times. Not they don't. This is 100% a made up statistic.

bluefirebrand 1y ago
It isn't even being presented as a statistic it is someone saying what they have experienced
- mexicocitinluez 1y ago
  nice. i'll add this to the list of "totally useless replies".
- mexicocitinluez 1y ago
  not to respond again but whats even funnier is that IT IS a statistic. saying "90% of the time I get a certain response" is literally a statistic.