4 ms·

A human may also possess malevolent tendencies that a silicon intelligence lacks. The question is not if they are equals, the question is if their differences

by ta8645 1y ago

A human may also possess malevolent tendencies that a silicon intelligence lacks. The question is not if they are equals, the question is if their differences matter to the endeavour of therapy. Maybe a human's superior hand-eye coordination matters, maybe it doesn't. Maybe a silicon agent's superior memory recall matters, maybe it doesn't. And so on.

AdieuToLogic 1y ago
> A human may also possess malevolent tendencies that a silicon intelligence lacks. And an LLM may be trained on malevolent data of which a human is unaware. > The question is not if they are equals, the question is if their differences matter to the endeavour of therapy. I did not pose the question of equality and apologize if the following was ambiguous in any way: ... they are humans which possess intrinsic aptitudes a statistical text (token) generator simply does not have. Let me now clarify - "silicon" does not have capabilities humans have relevant to successfully performing therapy. Specifically, LLM's are not an equal to human therapists excluding the pathological cases identified above.
- ta8645 1y ago
  > Let me now clarify - "silicon" does not have capabilities humans have relevant to successfully performing therapy. I think you're wrong, but that isn't really my point. A well-trained LLM that lacks any malevolent data, may well be better than a human psychopath who happens to have therapy credentials. And it may also be better than nothing at all for someone who is unable to reach a human therapist for one reason or another. For today, I'll agree with you, that the best human therapists that exist today, are better than the best silicon therapists that exist today. But I don't think that situation will persist any longer than such differences persisted in chess playing capabilities. Where for years I heard many people making the same mistake you're making, of saying that silicon could never demonstrate the flair and creativity of human chess players; that turned out to be false. It's simply human hubris to believe we possess capabilities that are impossible to duplicate in silicon.
  - joe_the_user 1y ago
    A well-trained LLM that lacks any malevolent data... The scale needed to produce an LLM that is fluent enough to be convincing precludes fine-grained filtering of input data. The usual methods of controlling an LLM essentially involve a broad-brush "don't say stuff like that" (RLHF) that inherently misses a lot of subtlties. And even more, defining malevolent data is extremely difficult. Therapists often go along with things a patient say because otherwise they break rapport. But therapists have to balk once the patient dives into destructive delusions. But data of a therapy can't be easily labeled with "here's where you have to stop", just to name one problem.
    - Dylan16807 1y ago
      So that's all true but the same argument works if you say a low percentage of malevolent data, and that's far from impossible.
      - joe_the_user 1y ago
        It's remarkable how many people are uncritically talking of "malevolent data" as it is was a well-defined concept that everyone knows is the source of bad things. A simple good search reveals ... this very thread as a primary source on the topic of "malevolent data" (ha, ha). But it should be noted that all other sources mentioning the phrase define it as data intentionally modified to produce a bad effect. It seems clear the problems of badly behaved LLMs don't come from this. Sycophancy, notably, doesn't just appear out of "sycophantic data" cleverly inserted by the association of allied sycophants.
        Dylan16807 1y ago
        I don't find it very remarkable that when one person makes up a term that's pretty easy to understand, other people in the same conversation use the same term. In the context of this conversation, it was a response to someone talking about malevolent human therapists, and worried about AIs being trained to do the same things. So that means it's text where one of the participants is acting malevolently in those same ways.
- BeetleB 1y ago
  The frustrating thing about your and several other arguments in this submission is that there is no rationale or data. All you are saying is "LLMs are not/cannot be good at therapy". The only (fake) rationale is "They are not humans." The whole comment comes across as tautological.
  - ClumsyPilot 1y ago
    The frustrating thing about your argument is that it runs on a pretence that we must prove squares aren’t circles. A person may be unable to provide mathematical proof and yes be obviously correct. The totally obvious thing you are missing is that most people will not encourage obviously self-destructive behaviour because they are not psychopaths. And they can get another person to intervene if necessary Chatbots do not have such concerts
    - BeetleB 1y ago
      I'm not sure I get the actual point you're making. To begin with, not all therapy involves people at risk of harming themselves. Easily over 95% of people who can benefit from therapy are at no more risk of harming themselves than the average person. Were a therapy chatbot to suggest something like it to them, the response will either be amusement or annoyance ("why am I wasting time on this?") Arguments from extremes (outliers) are the stuff of logical fallacies. As many keep pointing out, there are plenty of cases of licensed therapists causing harm. Most of the time it is unintentional, but for sure there are those who knowingly abused their position and took advantage of their patients. I'd love to see a study comparing the two ratios to see whether the human therapist or the LLM fare worse. I think most commenters here need to engage with real therapists more, so they can get a reality check on the field. I know therapists. I've been to some. I took a course from a seasoned therapist who also was a professor and had trained them. You know the whole replication crisis in psychology? Licensed therapy is no different. There's very little real science backing most of it (even the professor admitted it). Sure, there are some great therapists out there. The norm is barely better than you or I. Again, no exaggeration. So if the state of the art improves, and we then have a study showing some LLM therapists are better than the average licensed human one, I for one will not think it a great achievement.
nullc 1y ago
> A human may also possess malevolent tendencies that a silicon intelligence lacks. Sure but we're _generally_ more guarded against "Pastor is just a perv that wants to see me nude" then we are against "Chatbot wants to turn me into the unibomber because it was trained on fictional narrative arcs and it makes its decisions by a literal random number generator, and I just rolled a critical fail".

by ta8645 1y ago

AdieuToLogic 1y ago

> A human may also possess malevolent tendencies that a silicon intelligence lacks. And an LLM may be trained on malevolent data of which a human is unaware. > The question is not if they are equals, the question is if their differences matter to the endeavour of therapy. I did not pose the question of equality and apologize if the following was ambiguous in any way: ... they are humans which possess intrinsic aptitudes a statistical text (token) generator simply does not have. Let me now clarify - "silicon" does not have capabilities humans have relevant to successfully performing therapy. Specifically, LLM's are not an equal to human therapists excluding the pathological cases identified above.

ta8645 1y ago
> Let me now clarify - "silicon" does not have capabilities humans have relevant to successfully performing therapy. I think you're wrong, but that isn't really my point. A well-trained LLM that lacks any malevolent data, may well be better than a human psychopath who happens to have therapy credentials. And it may also be better than nothing at all for someone who is unable to reach a human therapist for one reason or another. For today, I'll agree with you, that the best human therapists that exist today, are better than the best silicon therapists that exist today. But I don't think that situation will persist any longer than such differences persisted in chess playing capabilities. Where for years I heard many people making the same mistake you're making, of saying that silicon could never demonstrate the flair and creativity of human chess players; that turned out to be false. It's simply human hubris to believe we possess capabilities that are impossible to duplicate in silicon.
- joe_the_user 1y ago
  A well-trained LLM that lacks any malevolent data... The scale needed to produce an LLM that is fluent enough to be convincing precludes fine-grained filtering of input data. The usual methods of controlling an LLM essentially involve a broad-brush "don't say stuff like that" (RLHF) that inherently misses a lot of subtlties. And even more, defining malevolent data is extremely difficult. Therapists often go along with things a patient say because otherwise they break rapport. But therapists have to balk once the patient dives into destructive delusions. But data of a therapy can't be easily labeled with "here's where you have to stop", just to name one problem.
  - Dylan16807 1y ago
    So that's all true but the same argument works if you say a low percentage of malevolent data, and that's far from impossible.
    - joe_the_user 1y ago
      It's remarkable how many people are uncritically talking of "malevolent data" as it is was a well-defined concept that everyone knows is the source of bad things. A simple good search reveals ... this very thread as a primary source on the topic of "malevolent data" (ha, ha). But it should be noted that all other sources mentioning the phrase define it as data intentionally modified to produce a bad effect. It seems clear the problems of badly behaved LLMs don't come from this. Sycophancy, notably, doesn't just appear out of "sycophantic data" cleverly inserted by the association of allied sycophants.
      - Dylan16807 1y ago
        I don't find it very remarkable that when one person makes up a term that's pretty easy to understand, other people in the same conversation use the same term. In the context of this conversation, it was a response to someone talking about malevolent human therapists, and worried about AIs being trained to do the same things. So that means it's text where one of the participants is acting malevolently in those same ways.
BeetleB 1y ago
The frustrating thing about your and several other arguments in this submission is that there is no rationale or data. All you are saying is "LLMs are not/cannot be good at therapy". The only (fake) rationale is "They are not humans." The whole comment comes across as tautological.
- ClumsyPilot 1y ago
  The frustrating thing about your argument is that it runs on a pretence that we must prove squares aren’t circles. A person may be unable to provide mathematical proof and yes be obviously correct. The totally obvious thing you are missing is that most people will not encourage obviously self-destructive behaviour because they are not psychopaths. And they can get another person to intervene if necessary Chatbots do not have such concerts
  - BeetleB 1y ago
    I'm not sure I get the actual point you're making. To begin with, not all therapy involves people at risk of harming themselves. Easily over 95% of people who can benefit from therapy are at no more risk of harming themselves than the average person. Were a therapy chatbot to suggest something like it to them, the response will either be amusement or annoyance ("why am I wasting time on this?") Arguments from extremes (outliers) are the stuff of logical fallacies. As many keep pointing out, there are plenty of cases of licensed therapists causing harm. Most of the time it is unintentional, but for sure there are those who knowingly abused their position and took advantage of their patients. I'd love to see a study comparing the two ratios to see whether the human therapist or the LLM fare worse. I think most commenters here need to engage with real therapists more, so they can get a reality check on the field. I know therapists. I've been to some. I took a course from a seasoned therapist who also was a professor and had trained them. You know the whole replication crisis in psychology? Licensed therapy is no different. There's very little real science backing most of it (even the professor admitted it). Sure, there are some great therapists out there. The norm is barely better than you or I. Again, no exaggeration. So if the state of the art improves, and we then have a study showing some LLM therapists are better than the average licensed human one, I for one will not think it a great achievement.

nullc 1y ago

> A human may also possess malevolent tendencies that a silicon intelligence lacks. Sure but we're _generally_ more guarded against "Pastor is just a perv that wants to see me nude" then we are against "Chatbot wants to turn me into the unibomber because it was trained on fictional narrative arcs and it makes its decisions by a literal random number generator, and I just rolled a critical fail".