4 ms·

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will

by GuB-42 2y ago

Of course they are a bug. Just that hallucination emerge from the normal function of a LLM doesn't make it "not a bug". No programmer in their right mind will call the lack of bound checking resulting in garbled output "not a bug", even though it is a totally normal thing to do from the point of view of a CPU. It is a bug and you need additional code to fix it, for example by checking for out-of-bounds condition and returning an error if it happens. Same thing for LLM hallucinations. LLMs naturally hallucinate, but it is not what we want, so it is a bug. And to fix it, we need to engineer solutions that prevent the hallucinations from happening, maybe resulting in an "I don't know" response that would be analogous to an error message. How you do it may be different from a simple "if", with probabilities and all that, but the general idea is the same: recognizing error cases and responding accordingly. I guess it is comes down to how you define a bug, but how else would you call a result that is not fit for purpose?

devmor 2y ago
A bug is generally treated as undefined and undesirable side effects of a program. Hallucinations are undesirable but not undefined. We know that the process creates them and expect them. It’d be like using floats to calculate dollars and cents and calling the resulting math a bug - it’s not, you just used the technology wrong.
lolinder 2y ago
A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing paradigm that was chosen, one that has to be worked around. It's closer to network lag than it is to bounds checking—it's an undesirable characteristic, but one that we knew about when we chose to make a network application. We'll do our best to mitigate it to acceptable levels, but it's certainly not a bug, it's just a fact of the paradigm.
- 9rx 2y ago
  Bug, like any other word, is defined however the speaker defines it. While your usage is certainly common in technical groups, the common "layman" usage is closer to what the parent suggests.
  - lolinder 2y ago
    And is there a compelling reason for us, while engaged in technical discussion with our technical peers about the technical mitigations for a technical defect, to use the layman usage rather than the term of art?
- ToucanLoucan 2y ago
  Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinate laws that do not exist. Recipes: it is not acceptable that it would tell people to make pizza with glue, or mustard gas to remove stains. Or, in my case: it is not acceptable for a developer assisting AI to hallucinate into existence libraries that are not real and not only will not solve my problem, but that will cause me to lose hours of my day trying to figure out where to get said library. If pneumatic tires failed to hold air as often as LLM's hallucinate, we wouldn't use them. That's not to say a tire can't blow out, sure they can, happens all the time. It's about the rate of failure. Or hell, to bring it back to your metaphor, if your network experienced high latency at the rate most LLM's hallucinate, I might actually suggest you not network computers, or at the very least, I'd say you should be replaced at whatever company you work for since you're clearly unqualified to manage a network.
  - lolinder 2y ago
    The benefits of networking outweigh the drawbacks in many situations, but not all, and good engineers avoid the network in cases where the lag would be unacceptable (i.e., real-time computing applications such as assembly line software). The same applies to LLMs—even if we're never able to get the rate of failure down below 5%, there are some applications that that would be fine for. The important thing isn't that the rate of failure be below a specific threshold before the technology is adopted anywhere, the important thing is that engineers working on this technology have an understanding of the fundamental limitations of the computing paradigm and design accordingly—up to and including telling leadership that LLMs are a fundamentally inappropriate tool for the job.
d0mine 2y ago
The difference is that you can fix IndexError by modifying your code but no amount of prompt manipulation may fix hallucinations. For that you need solutions outside LLMs.
PittleyDunkin 2y ago
> Of course they are a bug. A bug implies fixable behavior rather than expected behavior. An LLM making shit up is expected behavior. > LLMs naturally hallucinate, but it is not what we want, so it is a bug. Maybe you just don't want an LLM! This is what LLMs do. Maybe you want a decision tree or a scripted chatbot? > And to fix it, we need to engineer solutions that prevent the hallucinations from happening, maybe resulting in an "I don't know" response that would be analogous to an error message. I'm sure we'll figure out how to do this when we can fix the same bug in humans, too. Given that humans can't even agree when we're right or wrong—much less sense the incoherency of their own worldviews—I doubt we're going to see a solution to this in our lifetimes.
deleted 2y ago
[deleted]
stonemetal12 2y ago
> LLMs naturally hallucinate, but it is not what we want, so it is a bug. I rolled a one in D&D, it is not what I wanted, so it is a bug. Remove it from all my dice.
- SirMaster 2y ago
  What? You are telling me that when you roll a 6 sided dice you are not expecting any of the 1-6 as a result? If a 6-sided dice produced a 7 that would be a bug. When you rolled a dice, I would argue that you knew you wanted a random number from 1-6, not that you wanted a specific number or not a specific number. If you wanted that you wouldn't have used a dice. When I ask an LLM to write code for me and it references a completely made up library that doesn't exist and has never existed, is this really analogous to your dice example?
  - drewbeck 2y ago
    IMO it is because you just asked a bunch of dice to write code for you.
  - stonemetal12 2y ago
    >You are telling me that when you roll a 6 sided dice you are not expecting any of the 1-6 as a result? The statement I replied to wasn't any non-expected result is a bug, it was non-desired output is a bug (hence the joke about not desiring an expected output). LLMs producing "funny" (hallucination) outputs are expected but only sometimes not desired, therefore not a bug in my opinion. How do you use an LLM in story telling if it isn't allowed to produce fictious outputs?
jrm4 2y ago
Not a bug at all, IMHO. If someone puts the wrong address for their business; Google picks it up, and someone Googles and gets the wrong address, it says nothing about "bugs in software."
mrguyorama 2y ago
>Of course they are a bug No. When you build a bloom filter and it says "X is in the set" and X is NOT in the set, that's not a bug, that's an inherent behavior of the very theory of a probabilistic data structure. It is something that WILL happen, that you MUST expect to happen, and you MUST build around. >And to fix it, we need to engineer solutions that prevent the hallucinations from happening The whole point is that this is fundamentally impossible.

by GuB-42 2y ago

devmor 2y ago

A bug is generally treated as undefined and undesirable side effects of a program. Hallucinations are undesirable but not undefined. We know that the process creates them and expect them. It’d be like using floats to calculate dollars and cents and calling the resulting math a bug - it’s not, you just used the technology wrong.

lolinder 2y ago

A bug is defined as an unexpected defect. You can fix an unexpected defect by correcting the error in the code that led to the defect. In your example of lack of bounds checking there's a very concrete answer that will instantly fix the defect—add bounds checking. Hallucinations are not unexpected in LLMs and cannot be fixed by correcting an error in the code. Instead they are fundamental property of the computing paradigm that was chosen, one that has to be worked around. It's closer to network lag than it is to bounds checking—it's an undesirable characteristic, but one that we knew about when we chose to make a network application. We'll do our best to mitigate it to acceptable levels, but it's certainly not a bug, it's just a fact of the paradigm.

9rx 2y ago
Bug, like any other word, is defined however the speaker defines it. While your usage is certainly common in technical groups, the common "layman" usage is closer to what the parent suggests.
- lolinder 2y ago
  And is there a compelling reason for us, while engaged in technical discussion with our technical peers about the technical mitigations for a technical defect, to use the layman usage rather than the term of art?
ToucanLoucan 2y ago
Except we put up with network lag because it's an understandable, if undesirable, caveat to an otherwise useful technology. No one would ever say that because a network is sometimes slow, that it is then preferable to not have computers networked. The benefits clearly outweigh the drawbacks. This is not true for many applications of LLM. Generating legal documents, for example: it is not acceptable that it hallucinate laws that do not exist. Recipes: it is not acceptable that it would tell people to make pizza with glue, or mustard gas to remove stains. Or, in my case: it is not acceptable for a developer assisting AI to hallucinate into existence libraries that are not real and not only will not solve my problem, but that will cause me to lose hours of my day trying to figure out where to get said library. If pneumatic tires failed to hold air as often as LLM's hallucinate, we wouldn't use them. That's not to say a tire can't blow out, sure they can, happens all the time. It's about the rate of failure. Or hell, to bring it back to your metaphor, if your network experienced high latency at the rate most LLM's hallucinate, I might actually suggest you not network computers, or at the very least, I'd say you should be replaced at whatever company you work for since you're clearly unqualified to manage a network.
- lolinder 2y ago
  The benefits of networking outweigh the drawbacks in many situations, but not all, and good engineers avoid the network in cases where the lag would be unacceptable (i.e., real-time computing applications such as assembly line software). The same applies to LLMs—even if we're never able to get the rate of failure down below 5%, there are some applications that that would be fine for. The important thing isn't that the rate of failure be below a specific threshold before the technology is adopted anywhere, the important thing is that engineers working on this technology have an understanding of the fundamental limitations of the computing paradigm and design accordingly—up to and including telling leadership that LLMs are a fundamentally inappropriate tool for the job.

d0mine 2y ago

The difference is that you can fix IndexError by modifying your code but no amount of prompt manipulation may fix hallucinations. For that you need solutions outside LLMs.

PittleyDunkin 2y ago

> Of course they are a bug. A bug implies fixable behavior rather than expected behavior. An LLM making shit up is expected behavior. > LLMs naturally hallucinate, but it is not what we want, so it is a bug. Maybe you just don't want an LLM! This is what LLMs do. Maybe you want a decision tree or a scripted chatbot? > And to fix it, we need to engineer solutions that prevent the hallucinations from happening, maybe resulting in an "I don't know" response that would be analogous to an error message. I'm sure we'll figure out how to do this when we can fix the same bug in humans, too. Given that humans can't even agree when we're right or wrong—much less sense the incoherency of their own worldviews—I doubt we're going to see a solution to this in our lifetimes.

deleted 2y ago

[deleted]

stonemetal12 2y ago

> LLMs naturally hallucinate, but it is not what we want, so it is a bug. I rolled a one in D&D, it is not what I wanted, so it is a bug. Remove it from all my dice.

SirMaster 2y ago
What? You are telling me that when you roll a 6 sided dice you are not expecting any of the 1-6 as a result? If a 6-sided dice produced a 7 that would be a bug. When you rolled a dice, I would argue that you knew you wanted a random number from 1-6, not that you wanted a specific number or not a specific number. If you wanted that you wouldn't have used a dice. When I ask an LLM to write code for me and it references a completely made up library that doesn't exist and has never existed, is this really analogous to your dice example?
- drewbeck 2y ago
  IMO it is because you just asked a bunch of dice to write code for you.
- stonemetal12 2y ago
  >You are telling me that when you roll a 6 sided dice you are not expecting any of the 1-6 as a result? The statement I replied to wasn't any non-expected result is a bug, it was non-desired output is a bug (hence the joke about not desiring an expected output). LLMs producing "funny" (hallucination) outputs are expected but only sometimes not desired, therefore not a bug in my opinion. How do you use an LLM in story telling if it isn't allowed to produce fictious outputs?

jrm4 2y ago

Not a bug at all, IMHO. If someone puts the wrong address for their business; Google picks it up, and someone Googles and gets the wrong address, it says nothing about "bugs in software."

mrguyorama 2y ago

>Of course they are a bug No. When you build a bloom filter and it says "X is in the set" and X is NOT in the set, that's not a bug, that's an inherent behavior of the very theory of a probabilistic data structure. It is something that WILL happen, that you MUST expect to happen, and you MUST build around. >And to fix it, we need to engineer solutions that prevent the hallucinations from happening The whole point is that this is fundamentally impossible.