5 ms·

I’d be terrified to do this: - the insight into your mind that a private for profit company gets is immense and potentially very damaging when weaponized (eith

I’d be terrified to do this: - the insight into your mind that a private for profit company gets is immense and potentially very damaging when weaponized (either through a “whoops we got hacked” moment or intentionally for the next level of adtech) - chatgpt and other LLMs are known to hallucinate. What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you?

Llamamoe 4y ago
> chatgpt and other LLMs are known to hallucinate. What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? Therapy isn't magic always-correct advice either. It's about shifting your focus, attitudes, thought patterns through social influence, not giving you the right advice on each and every step. Even if it's just whatever, being heard out in a nonjudgmental manner, acknowledged, prompted to reflect, does a lot of good.
- ornornor 4y ago
  I get your point. I think it would bother me that's it's a robot/machine vs a real human, but that's just me. The same way that venting to my pet is somewhat cathartic but not very much compared to doing the same at my SO/parents/friends.
  - Llamamoe 4y ago
    I don't disagree with you. It feels somehow wrong to engage in theory of mind and the concomitant effects on your personality with an AI owned by a corporation. If OpenAI wished to, they could use it for insidious manipulation.
    - frozencell 4y ago
      It’s just a tool, you won’t ask humans to clean your back after going in the toilets.
haswell 4y ago
I share the privacy concerns, and look forward to running these kinds of models locally in the near future. > chatgpt and other LLMs are known to hallucinate. What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? As someone on a long-term therapy journey, I would be far less concerned about this. Therapy is rarely about doing exactly what one is told, it's about exploring your own thought processes. When a session does involve some piece of advice, or "do xyx for <benefit>", that is rarely enough to make it happen. Knowing something is good and actually doing it are two very different things, and it is exploring this delta that makes therapy valuable (in my personal experience). At some point, as that delta shrinks and one starts actually taking beneficial actions instead of just talking, the advice becomes more of a reminder / an entry point to the ground one has already covered, not something that could be considered prescriptive like "take this pill for 7 days". The point I'm trying to make is that if ChatGPT is the therapist, it doesn't make the person participating into a monkey who will just execute every command. Asking the bot to provide suggestions is more about jogging one's own thought processes than it is about carrying out specific tasks exactly as instructed. I do wonder how someone who hasn't worked with a therapist would navigate this. I could see the value of a bot like this as someone who already understands how the process works, but I could absolutely see a bot being actively harmful if it's the only support someone ever seeks. My first therapist was actively unhelpful due to lack of trauma-awareness, and I had to find someone else. So I could absolutely see a bot being unhelpful if used as the only therapeutic resource. On the flip side, ChatGPT might actually be more trauma-"aware" than some therapists, so who knows.
- alwaysbeconsing 4y ago
  This is all true, and it's not clear the grandparent is doing this. Last sentence of the original post: > Last weekend, I went on a hike because ChatGPT told me to. My life is better if I just do what it says. I'm not sure how literally to take that sentence, but it's worrisome.
  - haswell 4y ago
    I think my point was more that if they're doing what it says, that says more about where they’re at mentally (able to take action) and the quality of the advice (they’re willing to follow it). My stance here is based on an optimistic outlook that a person seeking therapeutic advice is by doing so demonstrating enough awareness that they’re probably capable of recognizing a good idea from a bad one. I realize this can get into other territory and there are very problematic failure modes in the worst cases. Regarding “My life is better if I just do what it says.”, I think concern is a fair reaction and I don’t think the author fully thought that through. But at the same time, it’s entirely possible that it’s true (for now). If someone continues to follow advice that is clearly either bad or not working, then it becomes concerning. But that was the other point of my anecdote. It became pretty clear to me what wasn’t working, even at a time that I wasn’t really sure how the whole thing worked.
paulcole 4y ago
> the insight into your mind that a private for profit company gets is immense and potentially very damaging when weaponized How exactly?
- ornornor 4y ago
  > (either through a “whoops we got hacked” moment or intentionally for the next level of adtech)
  - deleted 4y ago
    [deleted]
tux3 4y ago
>What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? What if you talk to a human, and their advice is wrong or makes you worse off in the long term, because they're just repeating something they heard somewhere? Here's my advice: Don't accept my advice blindly, humans make mistakes too.
- ornornor 4y ago
  Of course, but an AI can't explain how it got to what it's telling you. A human can, and you don't have to accept it wholesale; it's possible to judge it on its merits and argument. But no-one really understand how and why ChatGPT says what it does, so you can't trust anything it says unless you already know the answer. In this discussion, a human has studied psychology and has diplomas or certifications to prove it, an ethics framework it must follow, and responsibility for its mistakes. ChatGPT has none of that, it just regurgitates something it got from the Internet's wisdom or something it invented altogether. I'm not saying humans are never wrong, but at least their reasoning isn't a black box unlike ChatGPT and other LLMs.
  - HervalFreire 4y ago
    Most science in the social science is essentially black box studies. We see what goes.in and observe what comes out without any formal understanding of what goes on in the box itself. Additionally there's something called the replication crisis in the social sciences (psychology included) which is basically to the effect of a major discovery that most of these "black box" studies can not be reproduced. When someone runs the same experiment the results are all different. It goes to show that either many of the studies were fraudulent or statistical methodologies are flawed or both. Given that chatGPT therapeutic data is ALSO derived from the same training data I would say it's ok to trust chatGPT as much as you would trust psychologists. Both have a lot of bullshit with nuggets of truth.
Mezzie 4y ago
It's basically techno tarot cards in my view: The illusion of an external force helps you break certain internal inhibitions to consider your situation and problems more objectively.
ChildOfChaos 4y ago
I'm hugely curious why people are so worried that some AI has access to some thoughts of yours? Do you think you are somehow special? Just create a burner account and ask it what you want, everything it gets told, it's seen thousands of times over, does chatGPT or some data scientist somewhere really care that there is someone somewhere that is struggling with body issues, struggling in a relationship or struggling to find meaning in there lives? There are literally millions of people in the world with the same issue. The only time it might be a little embarrassing is if this info got leaked to friends and family with my name attached to it, else I don't get the problem, it seems to me people have an over inflated sense of self importance, nobody cares. If i worked at openAI and had full access to everyones chats, I would get bored within five minutes and not even want to read anymore.
- ornornor 4y ago
  > If i worked at openAI and had full access to everyones chats, I would get bored within five minutes and not even want to read anymore. Yes, obviously. But that's not what I'm worried about personally. I'm worried about the weaponization of this data in the future, either from OpenAI's greed to create the next advertising money-printing machine or from the data leaking through a breach. And because the interface with ChatGPT is so "natural" and "human like", it's easy to trust it and talk to it like a friend divulging very personal information about yourself. Imagine OpenAI (or whatever other AI) used the confidences you made to it or the specifics of your "therapy" sessions with it to get you to act in a certain way or buy certain things. Would you be comfortable with that? Well, that's irrelevant because they have the data already and can use it. Kinda like Cambdridge Analytica but on steroids because tailoring it to anyone's particular biases and way of thinking becomes trivial with ChatGPT and friends. Seeing how cavalier OpenAI has been with last week's breach and how fast they've flipped from being apparently benevolent to what they are now. And it's only been a few months of ChatGPT being available to the public.
  - ChildOfChaos 4y ago
    Yes, I would be fine with this. Not that much different from the current fingerprint techniques, it's just that on steroids but I don't understand the issue.
serpix 4y ago
The value of therapy outweighs the suspicion of some corporation using that data in my opinion. The benefits are large and extend from one individual to whole family chains, even communities.

Llamamoe 4y ago

> chatgpt and other LLMs are known to hallucinate. What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? Therapy isn't magic always-correct advice either. It's about shifting your focus, attitudes, thought patterns through social influence, not giving you the right advice on each and every step. Even if it's just whatever, being heard out in a nonjudgmental manner, acknowledged, prompted to reflect, does a lot of good.

ornornor 4y ago
I get your point. I think it would bother me that's it's a robot/machine vs a real human, but that's just me. The same way that venting to my pet is somewhat cathartic but not very much compared to doing the same at my SO/parents/friends.
- Llamamoe 4y ago
  I don't disagree with you. It feels somehow wrong to engage in theory of mind and the concomitant effects on your personality with an AI owned by a corporation. If OpenAI wished to, they could use it for insidious manipulation.
  - frozencell 4y ago
    It’s just a tool, you won’t ask humans to clean your back after going in the toilets.

haswell 4y ago

I share the privacy concerns, and look forward to running these kinds of models locally in the near future. > chatgpt and other LLMs are known to hallucinate. What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? As someone on a long-term therapy journey, I would be far less concerned about this. Therapy is rarely about doing exactly what one is told, it's about exploring your own thought processes. When a session does involve some piece of advice, or "do xyx for <benefit>", that is rarely enough to make it happen. Knowing something is good and actually doing it are two very different things, and it is exploring this delta that makes therapy valuable (in my personal experience). At some point, as that delta shrinks and one starts actually taking beneficial actions instead of just talking, the advice becomes more of a reminder / an entry point to the ground one has already covered, not something that could be considered prescriptive like "take this pill for 7 days". The point I'm trying to make is that if ChatGPT is the therapist, it doesn't make the person participating into a monkey who will just execute every command. Asking the bot to provide suggestions is more about jogging one's own thought processes than it is about carrying out specific tasks exactly as instructed. I do wonder how someone who hasn't worked with a therapist would navigate this. I could see the value of a bot like this as someone who already understands how the process works, but I could absolutely see a bot being actively harmful if it's the only support someone ever seeks. My first therapist was actively unhelpful due to lack of trauma-awareness, and I had to find someone else. So I could absolutely see a bot being unhelpful if used as the only therapeutic resource. On the flip side, ChatGPT might actually be more trauma-"aware" than some therapists, so who knows.

alwaysbeconsing 4y ago
This is all true, and it's not clear the grandparent is doing this. Last sentence of the original post: > Last weekend, I went on a hike because ChatGPT told me to. My life is better if I just do what it says. I'm not sure how literally to take that sentence, but it's worrisome.
- haswell 4y ago
  I think my point was more that if they're doing what it says, that says more about where they’re at mentally (able to take action) and the quality of the advice (they’re willing to follow it). My stance here is based on an optimistic outlook that a person seeking therapeutic advice is by doing so demonstrating enough awareness that they’re probably capable of recognizing a good idea from a bad one. I realize this can get into other territory and there are very problematic failure modes in the worst cases. Regarding “My life is better if I just do what it says.”, I think concern is a fair reaction and I don’t think the author fully thought that through. But at the same time, it’s entirely possible that it’s true (for now). If someone continues to follow advice that is clearly either bad or not working, then it becomes concerning. But that was the other point of my anecdote. It became pretty clear to me what wasn’t working, even at a time that I wasn’t really sure how the whole thing worked.

paulcole 4y ago

> the insight into your mind that a private for profit company gets is immense and potentially very damaging when weaponized How exactly?

ornornor 4y ago
> (either through a “whoops we got hacked” moment or intentionally for the next level of adtech)
- deleted 4y ago
  [deleted]

tux3 4y ago

>What if it’s advice is wrong or makes you worse in the long term because it’s just regurgitating whatever at you? What if you talk to a human, and their advice is wrong or makes you worse off in the long term, because they're just repeating something they heard somewhere? Here's my advice: Don't accept my advice blindly, humans make mistakes too.

ornornor 4y ago
Of course, but an AI can't explain how it got to what it's telling you. A human can, and you don't have to accept it wholesale; it's possible to judge it on its merits and argument. But no-one really understand how and why ChatGPT says what it does, so you can't trust anything it says unless you already know the answer. In this discussion, a human has studied psychology and has diplomas or certifications to prove it, an ethics framework it must follow, and responsibility for its mistakes. ChatGPT has none of that, it just regurgitates something it got from the Internet's wisdom or something it invented altogether. I'm not saying humans are never wrong, but at least their reasoning isn't a black box unlike ChatGPT and other LLMs.
- HervalFreire 4y ago
  Most science in the social science is essentially black box studies. We see what goes.in and observe what comes out without any formal understanding of what goes on in the box itself. Additionally there's something called the replication crisis in the social sciences (psychology included) which is basically to the effect of a major discovery that most of these "black box" studies can not be reproduced. When someone runs the same experiment the results are all different. It goes to show that either many of the studies were fraudulent or statistical methodologies are flawed or both. Given that chatGPT therapeutic data is ALSO derived from the same training data I would say it's ok to trust chatGPT as much as you would trust psychologists. Both have a lot of bullshit with nuggets of truth.

Mezzie 4y ago

It's basically techno tarot cards in my view: The illusion of an external force helps you break certain internal inhibitions to consider your situation and problems more objectively.

ChildOfChaos 4y ago

I'm hugely curious why people are so worried that some AI has access to some thoughts of yours? Do you think you are somehow special? Just create a burner account and ask it what you want, everything it gets told, it's seen thousands of times over, does chatGPT or some data scientist somewhere really care that there is someone somewhere that is struggling with body issues, struggling in a relationship or struggling to find meaning in there lives? There are literally millions of people in the world with the same issue. The only time it might be a little embarrassing is if this info got leaked to friends and family with my name attached to it, else I don't get the problem, it seems to me people have an over inflated sense of self importance, nobody cares. If i worked at openAI and had full access to everyones chats, I would get bored within five minutes and not even want to read anymore.

ornornor 4y ago
> If i worked at openAI and had full access to everyones chats, I would get bored within five minutes and not even want to read anymore. Yes, obviously. But that's not what I'm worried about personally. I'm worried about the weaponization of this data in the future, either from OpenAI's greed to create the next advertising money-printing machine or from the data leaking through a breach. And because the interface with ChatGPT is so "natural" and "human like", it's easy to trust it and talk to it like a friend divulging very personal information about yourself. Imagine OpenAI (or whatever other AI) used the confidences you made to it or the specifics of your "therapy" sessions with it to get you to act in a certain way or buy certain things. Would you be comfortable with that? Well, that's irrelevant because they have the data already and can use it. Kinda like Cambdridge Analytica but on steroids because tailoring it to anyone's particular biases and way of thinking becomes trivial with ChatGPT and friends. Seeing how cavalier OpenAI has been with last week's breach and how fast they've flipped from being apparently benevolent to what they are now. And it's only been a few months of ChatGPT being available to the public.
- ChildOfChaos 4y ago
  Yes, I would be fine with this. Not that much different from the current fingerprint techniques, it's just that on steroids but I don't understand the issue.

serpix 4y ago

The value of therapy outweighs the suspicion of some corporation using that data in my opinion. The benefits are large and extend from one individual to whole family chains, even communities.