5 ms·

Ask a modern LLM, like Gemini Pro 2.5. Takes a few minutes to get the answer, including gathering the code and pasting it into the prompt.

csmantle 1y ago
> Takes a few minutes to get the answer [...] ... then waste a few hundred minutes being misled by hallucination. It's quite the opposite of what "cracking open the code" is.
- 9dev 1y ago
  Not to forget the energy and computational power wasted to get that answer as well. It’s mindboggling how willingly some people will let their brain get degenerate by handing out shallow debugging tasks to LLMs.
  - ipaddr 1y ago
    You could look at it as wasting your brain on tasks like that. You start off with a full cup of water each task takes a portion. Farming out thought to an llm can allow you to focus on the next task or the overall before your cup is empty and you need to rest.
  - moi2388 1y ago
    The average query is equivalent to 1.1m of driving an electric car. It’s fine, really.
- tacitusarc 1y ago
  I ran into one of the most frightening instances of this recently with Gemini 2.5 Pro. It insisted that Go 1.25 had made a breaking change to the filepath.Join API. It hallucinated documentation to that effect on both the standard page and release notes. It refused to use web search to correct itself. When I finally (by convincing it that is was another AI checking the previous AIs work) got it to read the page, it claimed that the Go team had modified their release notes after the fact to remove information about the breaking change. I find myself increasingly convinced that regardless of the “intelligence” of LLMs, they should be kept far away from access to critical systems.
  - llbbdd 1y ago
    I've found that when any of these agents start going down a really wrong path, you just have to start a new session. I don't think I've ever had success at "redirecting" it once it starts doing weird shit and I assume this is a limitation of next-token prediction since the wrong path is still in the context window. When this happens I often have success telling it to summarize the TODOs/next steps, edit them if I have to remove weird or incorrect goals, and then paste them into a new session.
  - cyanydeez 1y ago
    Like social media, they'll seem benign until they're inervated the populace and start a digital fascism.
- erichocean 1y ago
  > including gathering the code LLMs are very reliable when asked about things in their own context window, which is what I recommended.
  - llbbdd 1y ago
    I'm increasingly convinced that most of the people still complaining about hallucinations with regard to programming just haven't actually used any of the tools in more than a year or two. Or they ran into a bias-confirming speedbump and gave up. Agents obviously hallucinate, because their default and only mode is hallucination, but seeing people insist that they do it too much to be useful just feels like I'm reading an archive of HN from 2022.
    - milch 1y ago
      Personally I think they are useful, but in a much narrow way than they are often sold as. For things I'm very familiar with, they seem to reduce my productivity by a good chunk. For things I don't want to do like writing some kinds of tests it's probably about the same, but then I don't have to do it, which is a win. For things I'm not very familiar with it probably is at least 2x faster to do with LLM, but that tends to diminish quickly. For example, I recently vibe coded a website using NextJS without knowing almost anything about it. Incredibly fast to get started by applying my existing knowledge of other systems/concepts and using the LLM to extend it to a new space. A week or so of full time work on it later I'm at the point where I know I can get most things done faster by hand, with the occasional LLM detour for things I haven't touched before
    - ipaddr 1y ago
      It depends on the model knowledge base and what you are trying to do. Something modern with the Buffalo framework in golang many hallucinations. A php blog written in 2005 no hallucinations.

csmantle 1y ago

> Takes a few minutes to get the answer [...] ... then waste a few hundred minutes being misled by hallucination. It's quite the opposite of what "cracking open the code" is.

9dev 1y ago
Not to forget the energy and computational power wasted to get that answer as well. It’s mindboggling how willingly some people will let their brain get degenerate by handing out shallow debugging tasks to LLMs.
- ipaddr 1y ago
  You could look at it as wasting your brain on tasks like that. You start off with a full cup of water each task takes a portion. Farming out thought to an llm can allow you to focus on the next task or the overall before your cup is empty and you need to rest.
- moi2388 1y ago
  The average query is equivalent to 1.1m of driving an electric car. It’s fine, really.
tacitusarc 1y ago
I ran into one of the most frightening instances of this recently with Gemini 2.5 Pro. It insisted that Go 1.25 had made a breaking change to the filepath.Join API. It hallucinated documentation to that effect on both the standard page and release notes. It refused to use web search to correct itself. When I finally (by convincing it that is was another AI checking the previous AIs work) got it to read the page, it claimed that the Go team had modified their release notes after the fact to remove information about the breaking change. I find myself increasingly convinced that regardless of the “intelligence” of LLMs, they should be kept far away from access to critical systems.
- llbbdd 1y ago
  I've found that when any of these agents start going down a really wrong path, you just have to start a new session. I don't think I've ever had success at "redirecting" it once it starts doing weird shit and I assume this is a limitation of next-token prediction since the wrong path is still in the context window. When this happens I often have success telling it to summarize the TODOs/next steps, edit them if I have to remove weird or incorrect goals, and then paste them into a new session.
- cyanydeez 1y ago
  Like social media, they'll seem benign until they're inervated the populace and start a digital fascism.
erichocean 1y ago
> including gathering the code LLMs are very reliable when asked about things in their own context window, which is what I recommended.
- llbbdd 1y ago
  I'm increasingly convinced that most of the people still complaining about hallucinations with regard to programming just haven't actually used any of the tools in more than a year or two. Or they ran into a bias-confirming speedbump and gave up. Agents obviously hallucinate, because their default and only mode is hallucination, but seeing people insist that they do it too much to be useful just feels like I'm reading an archive of HN from 2022.
  - milch 1y ago
    Personally I think they are useful, but in a much narrow way than they are often sold as. For things I'm very familiar with, they seem to reduce my productivity by a good chunk. For things I don't want to do like writing some kinds of tests it's probably about the same, but then I don't have to do it, which is a win. For things I'm not very familiar with it probably is at least 2x faster to do with LLM, but that tends to diminish quickly. For example, I recently vibe coded a website using NextJS without knowing almost anything about it. Incredibly fast to get started by applying my existing knowledge of other systems/concepts and using the LLM to extend it to a new space. A week or so of full time work on it later I'm at the point where I know I can get most things done faster by hand, with the occasional LLM detour for things I haven't touched before
  - ipaddr 1y ago
    It depends on the model knowledge base and what you are trying to do. Something modern with the Buffalo framework in golang many hallucinations. A php blog written in 2005 no hallucinations.