3 ms·

There’s a difference, though: if the erroneous data comes from human input error, it can easily be corrected, and if the error is due to a software bug, it can

There’s a difference, though: if the erroneous data comes from human input error, it can easily be corrected, and if the error is due to a software bug, it can be diagnosed and corrected; but errors that come from LLMs (the main type of AI in question here, and indicative enough of AIs in general, as it stands, though probably worse) are basically unfixable (that is: we don’t know how it would even be possible to fix them in general, so all you can do is patch things here and there, which will only probably fix the problems, and will increase the frequency of other problems).

brookst 3y ago
How is an error from an LLM less correctable than one from a human? If "Paris is the capital of Germany" makes it into a news article, how does the origin of the error impact its correctability? Reinforcement learning seems pretty analogous to how we correct people who are wrong about something. I'm really not seeing a category difference here?
- chrismorgan 3y ago
  With a human, if you have an employee, you can instruct them to correct an error, and if they refuse to or just don’t, you can fire them—you have recourse. LLMs are complex like humans, but you lack the ability to make it do what you want, or to have recourse. If the model has gained the impression that Paris is the capital of Germany (a more concrete and fixable error than most, frankly), this is woven into its parameters throughout, and it’s really hard to fix, and for LLMs we basically have no idea how to actually truly do so. With such a simple error as “Paris is the capital of Germany”, we do know how to fine-tune the model to fairly effectively mask that error, but the important errors are often closer to what you might consider procedural than factual—rather than its “memory” containing a wrong fact, it’s “thinking” in the wrong way. But all up, I’d call “lack of recourse” the most important difference.

brookst 3y ago

How is an error from an LLM less correctable than one from a human? If "Paris is the capital of Germany" makes it into a news article, how does the origin of the error impact its correctability? Reinforcement learning seems pretty analogous to how we correct people who are wrong about something. I'm really not seeing a category difference here?

chrismorgan 3y ago
With a human, if you have an employee, you can instruct them to correct an error, and if they refuse to or just don’t, you can fire them—you have recourse. LLMs are complex like humans, but you lack the ability to make it do what you want, or to have recourse. If the model has gained the impression that Paris is the capital of Germany (a more concrete and fixable error than most, frankly), this is woven into its parameters throughout, and it’s really hard to fix, and for LLMs we basically have no idea how to actually truly do so. With such a simple error as “Paris is the capital of Germany”, we do know how to fine-tune the model to fairly effectively mask that error, but the important errors are often closer to what you might consider procedural than factual—rather than its “memory” containing a wrong fact, it’s “thinking” in the wrong way. But all up, I’d call “lack of recourse” the most important difference.