3 ms·

I think we both agree that having a good metaphor matters to performing good, efficient research. Or at least that's what I would argue and it's what your "mech

I think we both agree that having a good metaphor matters to performing good, efficient research. Or at least that's what I would argue and it's what your "mechanical bird" example communicates. So then, the question becomes whether the language of psychology, which has its own troubles, provides a very good starting metaphor for what these very exciting systems do. You and I seem to disagree about that. For me, it's a lot more productive to study and communicate about these systems using more strongly established and less suggestive engineering and industrial metaphors. In my language, what's demonstrated in the paper is that these systems have a measurable and mutable degree of unreliability. That is, they can generate output that is often useful but sometimes wrong, and the likelihood (but not occassion) of their errors is an engineering problem to optimize. There are many systems that are useful and unreliable (incliding people!), but not many that are available to software engineers with the potential that these systems present. You can say that "deception" is a better description than "unreliable" if you want, but to my ear it sounds like it encourages people to look at these systems as "cunning" or "threatening" organisms instead of as practical and approachable technologies.

famouswaffles 3y ago
Yes, they are unreliable. But unreliable how ? How does this unreliability manifest ? Many different things are unreliable. Even traditional software can be unreliable. But imposing the kind of unreliability that traditional software brings to neural networks will lead to output or behavior that to you would be confusing, unintuitive and unpredictable. Introducing "deception" to this discussion allows for a predictive model of how this unreliability manifests, where you may expect it and so on. This model introduced by "deception" as it turns out is far more predictive than a "computer bug" model. Could it also introduce some ideas that may not square exactly with reality? Yes but it's certainly better than the alternative. >but to my ear it sounds like it encourages people to look at these systems as "cunning" or "threatening" organisms instead of as practical and approachable technologies. The idea is that the previous look will be better for approaching how to decide to deal with Agents you spin up with LLMs than the latter.
- swatcoder 3y ago
  You keep drawing dichotomies that I don't hold. The choices of language are not exhausted by "deception" and "computing bug". Both those words are your invention. I wouldn't even call this behavior a bug at all, since, as I pointed out in my original comment, it's a design characteristic of these systems to replicate the kind of material fed into them as training data, and the output we're seeing here correctly manifests that. Do you really think the only way to say "Don't use ChatGPT where strict adherence is essential because it's not that kind of system" is to say that the text generator may "deceived" someone, with all of the secondary implications that the word choice suggests? You can't conceive that there are any other metaphors and word choices that might both be more accurate and point in more fruitful research directions?
  - famouswaffles 3y ago
    >The choices of language are not exhausted by "deception" and "computing bug". Both those words are your invention. It's just an illustration. >Do you really think the only way to say "Don't use ChatGPT where strict adherence is essential because it's not that kind of system" is to say that the text generator may "deceived" someone, with all of the secondary implications that the word choice suggests? You can't conceive that there are any other metaphors and word choices that might both be more accurate and point in more fruitful research directions? I have yet to see anyone who harped on "don't anthropomize" provide better predictive models of output behavior. "Unreliable" is just a statement of fact. Output you might expect from those secondary implications manifest in the text or action output as well so that's fine. This brings me back to the second point in my original reply. You don't know how deception works in the brain and you don't know how deceptive like output works in LLMs. Couple that with Humanity's track record on these sort of discussions and those "secondary implications" may well be true.
    - swatcoder 3y ago
      In this case, the most productuce model is to look at as a text generator that produces documents that look like its training data, and the very predictive hypothesis one would draw from that is the one I put in my original hypothesis, and that was insightfully expanded upon by the person that replied to me. That model produces a hypothesis that fully accords with the experiment and does so with fewer assumptions about things like theory of mind, consciosness, or agency, which are all associated with "deception" Now, the model with the least assumptions is not always the most productive model, but here it works exceedingly well. The paper would have been stronger (less contestible) if framed in that model, but would admittedly have received less attention.

famouswaffles 3y ago

Yes, they are unreliable. But unreliable how ? How does this unreliability manifest ? Many different things are unreliable. Even traditional software can be unreliable. But imposing the kind of unreliability that traditional software brings to neural networks will lead to output or behavior that to you would be confusing, unintuitive and unpredictable. Introducing "deception" to this discussion allows for a predictive model of how this unreliability manifests, where you may expect it and so on. This model introduced by "deception" as it turns out is far more predictive than a "computer bug" model. Could it also introduce some ideas that may not square exactly with reality? Yes but it's certainly better than the alternative. >but to my ear it sounds like it encourages people to look at these systems as "cunning" or "threatening" organisms instead of as practical and approachable technologies. The idea is that the previous look will be better for approaching how to decide to deal with Agents you spin up with LLMs than the latter.

swatcoder 3y ago
You keep drawing dichotomies that I don't hold. The choices of language are not exhausted by "deception" and "computing bug". Both those words are your invention. I wouldn't even call this behavior a bug at all, since, as I pointed out in my original comment, it's a design characteristic of these systems to replicate the kind of material fed into them as training data, and the output we're seeing here correctly manifests that. Do you really think the only way to say "Don't use ChatGPT where strict adherence is essential because it's not that kind of system" is to say that the text generator may "deceived" someone, with all of the secondary implications that the word choice suggests? You can't conceive that there are any other metaphors and word choices that might both be more accurate and point in more fruitful research directions?
- famouswaffles 3y ago
  >The choices of language are not exhausted by "deception" and "computing bug". Both those words are your invention. It's just an illustration. >Do you really think the only way to say "Don't use ChatGPT where strict adherence is essential because it's not that kind of system" is to say that the text generator may "deceived" someone, with all of the secondary implications that the word choice suggests? You can't conceive that there are any other metaphors and word choices that might both be more accurate and point in more fruitful research directions? I have yet to see anyone who harped on "don't anthropomize" provide better predictive models of output behavior. "Unreliable" is just a statement of fact. Output you might expect from those secondary implications manifest in the text or action output as well so that's fine. This brings me back to the second point in my original reply. You don't know how deception works in the brain and you don't know how deceptive like output works in LLMs. Couple that with Humanity's track record on these sort of discussions and those "secondary implications" may well be true.
  - swatcoder 3y ago
    In this case, the most productuce model is to look at as a text generator that produces documents that look like its training data, and the very predictive hypothesis one would draw from that is the one I put in my original hypothesis, and that was insightfully expanded upon by the person that replied to me. That model produces a hypothesis that fully accords with the experiment and does so with fewer assumptions about things like theory of mind, consciosness, or agency, which are all associated with "deception" Now, the model with the least assumptions is not always the most productive model, but here it works exceedingly well. The paper would have been stronger (less contestible) if framed in that model, but would admittedly have received less attention.