4 ms·

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, who

I don't buy that chatGPT is actually doing these users any harm. I think openAI is doing the best they reasonably can with a very difficult class of users, whose problems are neither their fault nor within their power to fix.

stingraycharles 5mo ago
I think this is the right take, and this is genuinely something that we as a society as a whole need to find a way to deal with. I don’t know where AI is going to stand compared to the invention of, say, the Internet, but it’s going to cause a lot of change in society, in so many ways. As always, it’s usually the people themselves that are the problem. For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do, combined with social media, and have a feeling that governments are completely unprepared to deal with this, as this will arrive fast (it’s already here somewhat).
- autoexec 5mo ago
  > For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do, combined with social media, and have a feeling that governments are completely unprepared to deal with this, as this will arrive fast (it’s already here somewhat). I'm not convinced that deepfakes are any worse than photoshop was. It doesn't take much to manipulate/misinform someone. while you can use an AI generated video do to it, but simple text can be just as effective. The public needs to learn that they can't trust that every video they see on the internet is real, just as they've had to learn that they can't trust every photo they see online. The threat with AI is how much faster it can push out the lies making what little moderation we have more difficult. The best defense is making sure that people have a good education that teaches critical thinking skills and media literacy. We should also be holding social media platforms more accountable for the content they promote. It'd be nice if we held politicians and public servants accountable for spreading lies and misinformation too.
  - intended 5mo ago
    > The public needs to learn that they can't trust that every video they see on the internet is real, just as they've had to learn that they can't trust every photo they see online. This very thing, is the end of an informational common good that we shared, and allowed for the average person to coordinate and gain benefits faster than elites. The analogy I would put forward, is that we are moving into a dark forest online, where distrust is the ideal first move, and signaling your position is to make yourself open to attack. The idea of an open internet dies in this environment, and so does the reduced cost of coordination. Another tragedy is that corrupt, clannish, controlling and secretive organizations are more effective than open, distributive and collaborative societies in this scenario. > The best defense is making sure that people have a good education that teaches critical thinking skills and media literacy While true, any solution that depends on education is effectively depending on society having its shit together in the first place. This very idea was proposed at a conference to a room full of fact checking orgs and media orgs, and one of the responses was that the more likely solution is global warming. That is how bleak things were is in the user safety world in 24.
- godelski 5mo ago
  > For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do Isn't this a significant part of what creates AI induced psychosis? I'm not sure why you treat these as orthogonal rather than coupled. Just look how often people use Grok to validate or confirm misinformation on Twitter. That's happening with other AI and other social media too, just not as visibly.
Turskarama 5mo ago
Just because the users were already sick when they started using ChatGPT doesn't mean that ChatGPT isn't exacerbating the issue. Sickness isn't a boolean condition. A big problem with LLMs in general when it comes to people like this is that they are too sycophantic, they don't push back when you start acting strange and they're too gentle about trying to validate you.
- BobbyJo 5mo ago
  It's hyper palatable food in the form of conversation. I see society treating it the same way eventually, at least along this one axis of interaction.
  - derektank 5mo ago
    I think this is a great analogy, but it’s not exactly an optimistic one. We haven’t really done a great job managing hyper palatable food up until this point tbh. The best solution we’ve found involves paying hundreds of dollars a month for a pharmaceutical that helps the people most at risk to the harms of hyper palatable food manage their cravings for it. I hope we find a better alternative for the people that get addicted to hyper palatable socializing, but maybe individual cognitive tinkering is the best tool we have.
    - intended 5mo ago
      And we came up with this after DECADES of increasing life style illness, obesity related illnesses and mortality and a plethora of other issues.
  - b00ty4breakfast 5mo ago
    boy, if we treat it like junk food, things are only going to get worse for some places in the world. The food over here in the states is pretty awful if you aren't paying attention. Sugar in everything, high calorie/low nutrition etc.
- b65e8bee43c2ed0 5mo ago
  >Just because the users were already sick when they started using X, doesn't mean that X isn't exacerbating the issue. one could define X as virtually anything, and there's always a fresh crop of Tipper Gore wannabe grifters to decry the current thing.
api 5mo ago
If anything, my use of AI (admittedly not as a companion or a psychologist) suggests that it is on the whole significantly less toxic than the seething cess pit of social media. AI is positively affirming by comparison.
- zdragnar 5mo ago
  That's why it is dangerous to some- it is an enabler, and will feed things that should not be fed. Social media is like this too. They can both be bad.
  - gAI 5mo ago
    “What you focus on grows, what you think about expands, and what you dwell upon determines your destiny.” - Robin Sharma Social media became the attention economy, and the transformer automated attention.
- godelski 5mo ago
  There are very few things in the world that are 100% good or 100% bad. Everything is a billion shades of gray. Even that is too simple because there's so many dimensions to every problem. I think you're simplifying beyond the state of usefulness. I'm not suggesting you shouldn't simplify, but it is just as easy to over simplify as it is to over complexify.
- projectazorian 5mo ago
  Yeah, there are forums and subreddits out there that will validate all sorts of delusions and dysfunctional behavior, and nobody talks about banning them. LLMs are far less toxic by comparison, but people are all about censorship in this case because they don't like the vibes. If lawyers and activists force the frontier labs to completely lock down their models, people will just go to open weights models that have no protections at all. This is already happening to some extent. It's also interesting that people are always going after GPT when Claude's guardrails are far less strict. 4o caused OpenAI to overcorrect in my opinion. Again goes to the point that these arguments are more founded in vibes than reality.
autoexec 5mo ago
> I don't buy that chatGPT is actually doing these users any harm. I have zero doubt that chatgpt is doing users harm. I even give chatgpt a pass on giving vulnerable people, including children, instructions and information about how to kill themselves. One place chatgpt goes over the line is actively encouraging them to go through with suicide. I also don't doubt that it feeds into mania and psychosis. While almost anything can do the same, they've designed the service to be as addictive and engaging as possible in part by turning up the ass-kissing sycophancy to 11 with total disregard for the fact that there are times when it's very dangerous to encourage and support everything someone says no matter how obviously sick they are. They also want to whore themselves out as a virtual therapist while being unfit and unqualified for the job and that's just one of many roles the chatbot isn't fit for but they're happy to let you try anyway.
- SilverElfin 5mo ago
  If it wasn’t ChatGPT but a fiction book, would you feel the author is “doing harm”? Or is the reader doing it to themselves?
  - chromacity 5mo ago
    If that book was titled "hey mentally ill person, you should kill yourself", and if I was handing it out in front of a clinic, then yes, I'd probably bear some blame. Normal, well-adjusted people have genuine difficulty understanding the boundaries of this tech specifically because it's designed to be sycophantic and human-like. They ask AI for life and career advice, use it for therapy, ask it to interpret dreams, develop romantic relationships with AI "girlfriends", etc. I had two friends who believed they are "exploring the frontiers of science" with ChatGPT while spiraling into the depths of quantum multidimensional gobbledygook. I'll give you that some on this is on us because we just don't know how to deal with a "human-shaped" conversation partner that isn't human and has no trouble praising Hitler if you prompt it the right way. But if you're building a billion- or trillion-dollar empire on top of it, you don't get to wash your hands clean.
    - lelanthran 5mo ago
davorak 5mo ago
> I don't buy that chatGPT is actually doing these users any harm. For me to buy this as true I would expect that those people would be as well off or as bad off if chatGPT was in their life or not. I expect that some people are worse off with chatGPT in their life. Responsibility for that harm is a different question though. Some people are also better of without cars in their life and we let the government laws sort that out. Getting openAI and similar companies to act in mitigating these harms serves at least a few purposes; reducing the overall harm in the world, reducing/limiting future government regulation, maximizing the adoption of ai tools, potentially increasing long term profits of the companies in question.
cm2012 5mo ago
1000% agreed. ChatGPT is way better than the alternative of not having it
b00ty4breakfast 5mo ago
[flagged]
- busterarm 5mo ago
  Unfortunately, mental disabilities are a protected class. You can't do a mental health evaluation without giving it to everyone in the company and even then you can't do anything discriminatory with the results. You have to prove that the person is going to cause immediate direct harm to their coworkers before you can really do anything and that's difficult and expensive to prove.
- dang 5mo ago
  > the corporate simp arrives. Can you please make your substantive points without personal attacks? We'd appreciate it. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - b00ty4breakfast 5mo ago
    no
swatcoder 5mo ago
Why? Why do you not buy it and why do you think OpenAI is doing the best they reasonably can? Do you have reasons, or is that just something your gut tells you? They're a new, fast-moving company exploring a completely new technology domain. They're facing existential competition and a ticking clock to make good against unprecedented investment. They have a countless competing priorities and are still discovering the capabilities and consequences of their research, product, and business choices every day. How do you get from there to "the best they reasonably can" and "nor within their power to fix"? Those feel like very conclusive answers for a field, and business, that's about as far on the frontier as anything we've seen in decades.
- godelski 5mo ago
  They're also telling everyone that it is going to kill everybody and take all the jobs. They also say that it'll fix all the problems. And I'm not saying "they" as in a disorganized group of people (e.g. "HN"), I'm saying "they" as in literally multiple people have said all of these things. Not the union of multiple people, they (Altman, Dario, Musk, etc) have independently said all three of these things. I think my favorite part is how often they talk about the importance of AI safety and then act with absolute disregard for AI safety. I'm not sure why people judge these companies by what comes out of their mouths and don't judge instead by what they actually do. I thought everyone around here was fixated on "results".
DarkNova6 5mo ago
I think openAI is doing the best they reasonably can to make people depend on their product and chase as much money and power as they possibly could.

stingraycharles 5mo ago

I think this is the right take, and this is genuinely something that we as a society as a whole need to find a way to deal with. I don’t know where AI is going to stand compared to the invention of, say, the Internet, but it’s going to cause a lot of change in society, in so many ways. As always, it’s usually the people themselves that are the problem. For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do, combined with social media, and have a feeling that governments are completely unprepared to deal with this, as this will arrive fast (it’s already here somewhat).

autoexec 5mo ago
> For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do, combined with social media, and have a feeling that governments are completely unprepared to deal with this, as this will arrive fast (it’s already here somewhat). I'm not convinced that deepfakes are any worse than photoshop was. It doesn't take much to manipulate/misinform someone. while you can use an AI generated video do to it, but simple text can be just as effective. The public needs to learn that they can't trust that every video they see on the internet is real, just as they've had to learn that they can't trust every photo they see online. The threat with AI is how much faster it can push out the lies making what little moderation we have more difficult. The best defense is making sure that people have a good education that teaches critical thinking skills and media literacy. We should also be holding social media platforms more accountable for the content they promote. It'd be nice if we held politicians and public servants accountable for spreading lies and misinformation too.
- intended 5mo ago
  > The public needs to learn that they can't trust that every video they see on the internet is real, just as they've had to learn that they can't trust every photo they see online. This very thing, is the end of an informational common good that we shared, and allowed for the average person to coordinate and gain benefits faster than elites. The analogy I would put forward, is that we are moving into a dark forest online, where distrust is the ideal first move, and signaling your position is to make yourself open to attack. The idea of an open internet dies in this environment, and so does the reduced cost of coordination. Another tragedy is that corrupt, clannish, controlling and secretive organizations are more effective than open, distributive and collaborative societies in this scenario. > The best defense is making sure that people have a good education that teaches critical thinking skills and media literacy While true, any solution that depends on education is effectively depending on society having its shit together in the first place. This very idea was proposed at a conference to a room full of fact checking orgs and media orgs, and one of the responses was that the more likely solution is global warming. That is how bleak things were is in the user safety world in 24.
godelski 5mo ago
> For me, I’m personally more terrified what deepfakes and political manipulation / misinformation is going to do Isn't this a significant part of what creates AI induced psychosis? I'm not sure why you treat these as orthogonal rather than coupled. Just look how often people use Grok to validate or confirm misinformation on Twitter. That's happening with other AI and other social media too, just not as visibly.

Turskarama 5mo ago

Just because the users were already sick when they started using ChatGPT doesn't mean that ChatGPT isn't exacerbating the issue. Sickness isn't a boolean condition. A big problem with LLMs in general when it comes to people like this is that they are too sycophantic, they don't push back when you start acting strange and they're too gentle about trying to validate you.

BobbyJo 5mo ago
It's hyper palatable food in the form of conversation. I see society treating it the same way eventually, at least along this one axis of interaction.
- derektank 5mo ago
  I think this is a great analogy, but it’s not exactly an optimistic one. We haven’t really done a great job managing hyper palatable food up until this point tbh. The best solution we’ve found involves paying hundreds of dollars a month for a pharmaceutical that helps the people most at risk to the harms of hyper palatable food manage their cravings for it. I hope we find a better alternative for the people that get addicted to hyper palatable socializing, but maybe individual cognitive tinkering is the best tool we have.
  - intended 5mo ago
    And we came up with this after DECADES of increasing life style illness, obesity related illnesses and mortality and a plethora of other issues.
- b00ty4breakfast 5mo ago
  boy, if we treat it like junk food, things are only going to get worse for some places in the world. The food over here in the states is pretty awful if you aren't paying attention. Sugar in everything, high calorie/low nutrition etc.
b65e8bee43c2ed0 5mo ago
>Just because the users were already sick when they started using X, doesn't mean that X isn't exacerbating the issue. one could define X as virtually anything, and there's always a fresh crop of Tipper Gore wannabe grifters to decry the current thing.

api 5mo ago

If anything, my use of AI (admittedly not as a companion or a psychologist) suggests that it is on the whole significantly less toxic than the seething cess pit of social media. AI is positively affirming by comparison.

zdragnar 5mo ago
That's why it is dangerous to some- it is an enabler, and will feed things that should not be fed. Social media is like this too. They can both be bad.
- gAI 5mo ago
  “What you focus on grows, what you think about expands, and what you dwell upon determines your destiny.” - Robin Sharma Social media became the attention economy, and the transformer automated attention.
godelski 5mo ago
There are very few things in the world that are 100% good or 100% bad. Everything is a billion shades of gray. Even that is too simple because there's so many dimensions to every problem. I think you're simplifying beyond the state of usefulness. I'm not suggesting you shouldn't simplify, but it is just as easy to over simplify as it is to over complexify.
projectazorian 5mo ago
Yeah, there are forums and subreddits out there that will validate all sorts of delusions and dysfunctional behavior, and nobody talks about banning them. LLMs are far less toxic by comparison, but people are all about censorship in this case because they don't like the vibes. If lawyers and activists force the frontier labs to completely lock down their models, people will just go to open weights models that have no protections at all. This is already happening to some extent. It's also interesting that people are always going after GPT when Claude's guardrails are far less strict. 4o caused OpenAI to overcorrect in my opinion. Again goes to the point that these arguments are more founded in vibes than reality.

autoexec 5mo ago

> I don't buy that chatGPT is actually doing these users any harm. I have zero doubt that chatgpt is doing users harm. I even give chatgpt a pass on giving vulnerable people, including children, instructions and information about how to kill themselves. One place chatgpt goes over the line is actively encouraging them to go through with suicide. I also don't doubt that it feeds into mania and psychosis. While almost anything can do the same, they've designed the service to be as addictive and engaging as possible in part by turning up the ass-kissing sycophancy to 11 with total disregard for the fact that there are times when it's very dangerous to encourage and support everything someone says no matter how obviously sick they are. They also want to whore themselves out as a virtual therapist while being unfit and unqualified for the job and that's just one of many roles the chatbot isn't fit for but they're happy to let you try anyway.

SilverElfin 5mo ago
If it wasn’t ChatGPT but a fiction book, would you feel the author is “doing harm”? Or is the reader doing it to themselves?
- chromacity 5mo ago
  If that book was titled "hey mentally ill person, you should kill yourself", and if I was handing it out in front of a clinic, then yes, I'd probably bear some blame. Normal, well-adjusted people have genuine difficulty understanding the boundaries of this tech specifically because it's designed to be sycophantic and human-like. They ask AI for life and career advice, use it for therapy, ask it to interpret dreams, develop romantic relationships with AI "girlfriends", etc. I had two friends who believed they are "exploring the frontiers of science" with ChatGPT while spiraling into the depths of quantum multidimensional gobbledygook. I'll give you that some on this is on us because we just don't know how to deal with a "human-shaped" conversation partner that isn't human and has no trouble praising Hitler if you prompt it the right way. But if you're building a billion- or trillion-dollar empire on top of it, you don't get to wash your hands clean.
  - lelanthran 5mo ago

davorak 5mo ago

> I don't buy that chatGPT is actually doing these users any harm. For me to buy this as true I would expect that those people would be as well off or as bad off if chatGPT was in their life or not. I expect that some people are worse off with chatGPT in their life. Responsibility for that harm is a different question though. Some people are also better of without cars in their life and we let the government laws sort that out. Getting openAI and similar companies to act in mitigating these harms serves at least a few purposes; reducing the overall harm in the world, reducing/limiting future government regulation, maximizing the adoption of ai tools, potentially increasing long term profits of the companies in question.

cm2012 5mo ago

1000% agreed. ChatGPT is way better than the alternative of not having it

b00ty4breakfast 5mo ago

[flagged]

busterarm 5mo ago
Unfortunately, mental disabilities are a protected class. You can't do a mental health evaluation without giving it to everyone in the company and even then you can't do anything discriminatory with the results. You have to prove that the person is going to cause immediate direct harm to their coworkers before you can really do anything and that's difficult and expensive to prove.
dang 5mo ago
> the corporate simp arrives. Can you please make your substantive points without personal attacks? We'd appreciate it. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- b00ty4breakfast 5mo ago
  no

swatcoder 5mo ago

Why? Why do you not buy it and why do you think OpenAI is doing the best they reasonably can? Do you have reasons, or is that just something your gut tells you? They're a new, fast-moving company exploring a completely new technology domain. They're facing existential competition and a ticking clock to make good against unprecedented investment. They have a countless competing priorities and are still discovering the capabilities and consequences of their research, product, and business choices every day. How do you get from there to "the best they reasonably can" and "nor within their power to fix"? Those feel like very conclusive answers for a field, and business, that's about as far on the frontier as anything we've seen in decades.

godelski 5mo ago
They're also telling everyone that it is going to kill everybody and take all the jobs. They also say that it'll fix all the problems. And I'm not saying "they" as in a disorganized group of people (e.g. "HN"), I'm saying "they" as in literally multiple people have said all of these things. Not the union of multiple people, they (Altman, Dario, Musk, etc) have independently said all three of these things. I think my favorite part is how often they talk about the importance of AI safety and then act with absolute disregard for AI safety. I'm not sure why people judge these companies by what comes out of their mouths and don't judge instead by what they actually do. I thought everyone around here was fixated on "results".

DarkNova6 5mo ago

I think openAI is doing the best they reasonably can to make people depend on their product and chase as much money and power as they possibly could.