6 ms·

> We see that the nightmare scenario - a person with no previous psychosis history or risk factor becoming fully psychotic - was uncommon, at only 10% of cases.

> We see that the nightmare scenario - a person with no previous psychosis history or risk factor becoming fully psychotic - was uncommon, at only 10% of cases. Most people either had a previous psychosis history known to the respondent, or had some obvious risk factor, or were merely crackpots rather than full psychotics. It's unfortunate to see the author take this tack. This is essentially taking the conventional tack that insanity is separable: some people are "afflicted", some people just have strange ideas -- the implication of this article being that people who already have strange ideas were going to be crazy anyways, so GPT didn't contribute anything novel, just moved them along the path they were already moving regardless. But anyone with serious experience with schizophrenia would understand that this isn't how it works: 'biological' mental illness is tightly coupled to qualitative mental state, and bidirectionally at that. Not only do your chemicals influence your thoughts, your thoughts influence your chemicals, and it's possible for a vulnerable person to be pushed over the edge by either kind of input. We like to think that 'as long as nothing is chemically wrong' we're a-ok, but the truth is that it's possible for simple normal trains of thought to latch your brain into a very undesirable state. For this reason it is very important that vulnerable people be well-moored, anchored to reality by their friends and family. A normal person would take care to not support fantasies of government spying or divine miracles or &c where not appropriate, but ChatGPT will happily egg them on. These intermediate cases that Scott describes -- cases where someone is 'on the edge', but not yet detached from reality -- are the ones you really want to watch out for. So where he estimates an incidence rate of 1/100,000, I think his own data gives us a more accurate figure of ~1/20,000.

deleted 1y ago
[deleted]
olehif 1y ago
Scott is a psychiatrist.
- throwaway314155 1y ago
  Then he's not a very good one. https://web.archive.org/web/20210215053502/https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://web.archive.org/web/20210215053502/https://www.nytim...
  - rendang 1y ago
    What is the connection between the claim and the link?
    - meowface 1y ago
      There isn't any. (Also, on top of that, I think it's overall not a very good article.)
  - chermi 1y ago
    What a disgusting article.
  - kelnos 1y ago
    That's essentially a retaliatory hit piece the NYT printed because they were mad that Scott deleted his website because the NYT wanted to doxx him. Not saying there's no merit to the article, but it should be looked upon skeptically due to that bias.
    - ZYbCRq22HbJ2y7 1y ago
      > NYT wanted to doxx him NYT wanted to report on who he was. He doxxed himself years before that (as mentioned in that article). They eventually also reported on that (after Alexander revealed his name, seeing that it was going to come out anyway, I guess), which is an asshole thing to do, but not doxxing, IMO.
      - lmm 1y ago
        > NYT wanted to report on who he was. They wanted to report specifically his birth/legal name, with no plausible public interest reason. If it wasn't "stochastic terrorism" (as the buzzword of the day was) then it sure looked a lot like it. > He doxxed himself years before that Few people manage to keep anything 100% secret. Realistically private/public is a spectrum not a binary, and publication in the NYT is a pretty drastic step up.
- YeGoblynQueenne 1y ago
  Sigmund Freud was also a psychiatrist.
meowface 1y ago
I'm not trying to argue from authority or get into credibility wars*, but Scott is a professional psychiatrist who has treated dozens or hundreds of schizophrenic patients and has written many thorough essays on schizophrenia. Obviously someone could do that and still be wrong, but I think this is a carefully considered position on his part and not just wild assumptions. *(or, well, okay, I guess I de facto am, but if I say I'm not I at least acknowledge how it looks)
- mquander 1y ago
  You said it yourself. That's really not an appropriate response to a specific criticism.
  - meowface 1y ago
    I'm not trying to say that that should strongly increase the probability he's correct. I just think it's useful context, because the parent is potentially implying that the author is naively falling for common misconceptions ("following the conventional tack") rather than staking a deliberated claim. Or they might not be implying it but someone could come away with that conclusion.
  - kelnos 1y ago
    I mean, on one hand you have a professional psychiatrist who has treated many people for the disorder we're talking about, and on the other, we have a rando on HN who hasn't presented any credentials. Not saying the latter person is automatically wrong, but I think if you're going to argue against something said by someone who is a subject matter expert, the bar is a bit higher.
  - riwsky 1y ago
    The criticism invoked “anyone with serious experience with schizophrenia”, implying the author of the article is not such a one. Citing the author’s experience is a perfectly valid rebuttal to that implication. It’s not an argument from authority, but about it.
shayway 1y ago
The article's conclusion is exactly what you describe: that AI is bringing out latent predisposition toward psychosis through runaway feedback loops, that it's a bidirectional relationship where the chemicals influence thoughts and thoughts influence chemicals until we decide to call it psychosis. I hate to be the 'you didn't read the article' guy but that line taken out of context is the exact opposite of my takeaway for the article as a whole. For anyone else who skims comments before clicking I would invite you to read the whole thing (or at least get past the poorly-worded intro) before drawing conclusions.
kayodelycaon 1y ago
You might want to read the entire article. His depiction of bipolar is completely accurate. In fact it is so precisely accurate in every detail, and conveyed with no extraneous information, is indicative of someone who knows the disorder very well. When I write fiction or important emails, I am precise with the words I use. I notice these kind of details. I’m also bipolar and self-aware enough to be deeply familiar with it.
- phreeza 1y ago
  The author is a psychiatrist so it would make sense that he is familiar with the subject.
  - vintermann 1y ago
    And as I recall, he used to be a lot more clear that mental illness isn't always clear cut. I was surprised at the "obviously, we all know what mental illness is" attitude coming from him.
    - anon84873628 1y ago
      I imagine he's been doing this so long that taking a sober, delicate, appeal-to-every-reader approach to every article just isn't fun or practical. He wants to get down his thoughts for his core reader base and move on. What fraction goes viral anyway? And so what if some people misunderstand or don't like it? You're always going to have those people anyway no matter what :shrug:
epiccoleman 1y ago
> 'biological' mental illness is tightly coupled to qualitative mental state, and bidirectionally at that. Not only do your chemicals influence your thoughts, your thoughts influence your chemicals, and it's possible for a vulnerable person to be pushed over the edge by either kind of input. We like to think that 'as long as nothing is chemically wrong' we're a-ok, but the truth is that it's possible for simple normal trains of thought to latch your brain into a very undesirable state. It's interesting to see you mention this. After reading this post yesterday I wound up with some curious questions along these lines. I guess my question goes something like this: This article seems to assert that 'mental illness' must always have some underlying representation in the brain - that is, mental illness is caused by chemical imbalances or malformation in brain structure. But is it possible for a brain to become 'disordered' in a purely mental way? i.e. that to any way we know of "inspecting" the brain, it would look like a the hardware was healthy - but the "mind inside the brain" could somehow be stuck in a "thought trap"? Your post above seems to assert this could be the case. I think I've pretty much internalized a notion of consciousness that was purely bottom-up and materialistic. Thoughts are the product of brain state, brain state is the product of physics, which at "brain component scale" is deterministic. So it seems very spooky on its face that somehow thoughts themselves could have a bidirectional relationship with chemistry. I spent a bunch of time reading articles and (what else) chatting with Claude back and forth about this topic, and it's really interesting - it seems there are at least some arguments out there that information (or maybe even consciousness) can have causal effects on "stuff" (matter). There's the "Integrated Information Theory" of consciousness (which seems to be, if not exactly "fringe", at least widely disputed) and there's also this interesting notion of "downward causation" (basically the idea that higher-level systems can have causal effects on lower levels - I'm not clear on whether "thought having causal effects on chemistry" fits into this model). I've got 5 or 6 books coming my way from the local library system - it's a pretty fascinating topic, though I haven't dug deep enough to decide where I stand. Sorry for the ramble, but this article has at least inspired some interesting rabbit-hole diving for me. I'm curious - when you assert "Not only do your chemicals influence your thoughts, your thoughts influence your chemicals" - do you have evidence that backs that notion up? I'm not asking to cast doubt, but rather, I guess, because it sounds like maybe you've got some sources I might find interesting as I keep reading.
jedharris 1y ago
> it's possible for simple normal trains of thought to latch your brain into a very undesirable state. This seems very incorrect, or at least drastically underspecified. These trains of thought are "normal" (i.e. common and unremarkable) so why don't they "latch your brain into a very undesirable state" lots of the time? I don't think Scott or anyone up to speed on modern neuroscience would deny the coupling of mental state and brain chemistry--in fact I think it would be more accurate to say both of them are aspects of the dynamics of the brain. But this doesn't imply that "simple normal trains of thought" can latch our brain dynamics into bad states -- i.e. in dynamics language move us into a undesirable attractor. That would require a very problematic fragility in our normal self-regulation of brain dynamics.
- AstralStorm 1y ago
  See the key here is, the AI provides a very enticing social partner. Think of it as a version of making your drugged friend believe various random stuff. It works better if you're not a stranger and have an engaging or alarming style. LLMs are trained to produce pleasant responses that tailor to the user to maximize positive responses. (A more general version of engagement.) It stands to reason they would be effective at convincing someone.
anon84873628 1y ago
One of the questions that sets up the premise of the article in the first paragraph is, "Are the chatbots really driving people crazy, or just catching the attention of people who were crazy already?" That's why he's honing in on that specific scenario to determine if chatbots are uniquely crazy-making or something. The professional psychiatrist author is not unaware of the things you're saying. They're just not the purpose of the survey & article.
- bccdee 1y ago
  Well yeah, that's a false dichotomy. If you're vulnerable and a chatbot sends you into a spiral of psychosis, your pre-existing vulnerability doesn't negate the fact that a harm has been done to you. If you have a heart condition, and I shoot you with a Taser, and it kills you… I've killed you. You weren't "already dead" just because you were vulnerable.
  - anon84873628 1y ago
    Yes but "what is the effect of tasers on hearts" is an interesting question when tasers are brand new. If it kills people with obvious pre-existing risks then that is not very surprising. If it kills 50% of otherwise healthy people in a way we didn't anticipate, that is alarming and important to distinguish. Imagine someone does a quick survey to estimate that tasers aren't killing people we don't expect, and some readers respond saying how dare you ignore the vulnerable heart people. That's still an important thing to consider and maybe we should be careful with the mass scale rollout of tasers, but it wasn't really the immediate point.
    - bccdee 1y ago
      > Imagine someone does a quick survey to estimate that tasers aren't killing people we don't expect Given that the quote you cited was, "Are the chatbots really driving people crazy, or just catching the attention of people who were crazy already," I'd say the equivalent would be something like, "Are tasers really killing people, or were tasered heart attack victims dying already?" And yeah, I'd be mad about that framing! The fact that the people who die had a preexisting vulnerability does not mean they were "already dying" or that they were not "really killed."

deleted 1y ago

[deleted]

olehif 1y ago

Scott is a psychiatrist.

throwaway314155 1y ago
Then he's not a very good one. https://web.archive.org/web/20210215053502/https://www.nytimes.com/2021/02/13/technology/slate-star-codex-rationalists.html https://web.archive.org/web/20210215053502/https://www.nytim...
- rendang 1y ago
  What is the connection between the claim and the link?
  - meowface 1y ago
    There isn't any. (Also, on top of that, I think it's overall not a very good article.)
- chermi 1y ago
  What a disgusting article.
- kelnos 1y ago
  That's essentially a retaliatory hit piece the NYT printed because they were mad that Scott deleted his website because the NYT wanted to doxx him. Not saying there's no merit to the article, but it should be looked upon skeptically due to that bias.
  - ZYbCRq22HbJ2y7 1y ago
    > NYT wanted to doxx him NYT wanted to report on who he was. He doxxed himself years before that (as mentioned in that article). They eventually also reported on that (after Alexander revealed his name, seeing that it was going to come out anyway, I guess), which is an asshole thing to do, but not doxxing, IMO.
    - lmm 1y ago
      > NYT wanted to report on who he was. They wanted to report specifically his birth/legal name, with no plausible public interest reason. If it wasn't "stochastic terrorism" (as the buzzword of the day was) then it sure looked a lot like it. > He doxxed himself years before that Few people manage to keep anything 100% secret. Realistically private/public is a spectrum not a binary, and publication in the NYT is a pretty drastic step up.
YeGoblynQueenne 1y ago
Sigmund Freud was also a psychiatrist.

meowface 1y ago

I'm not trying to argue from authority or get into credibility wars*, but Scott is a professional psychiatrist who has treated dozens or hundreds of schizophrenic patients and has written many thorough essays on schizophrenia. Obviously someone could do that and still be wrong, but I think this is a carefully considered position on his part and not just wild assumptions. *(or, well, okay, I guess I de facto am, but if I say I'm not I at least acknowledge how it looks)

mquander 1y ago
You said it yourself. That's really not an appropriate response to a specific criticism.
- meowface 1y ago
  I'm not trying to say that that should strongly increase the probability he's correct. I just think it's useful context, because the parent is potentially implying that the author is naively falling for common misconceptions ("following the conventional tack") rather than staking a deliberated claim. Or they might not be implying it but someone could come away with that conclusion.
- kelnos 1y ago
  I mean, on one hand you have a professional psychiatrist who has treated many people for the disorder we're talking about, and on the other, we have a rando on HN who hasn't presented any credentials. Not saying the latter person is automatically wrong, but I think if you're going to argue against something said by someone who is a subject matter expert, the bar is a bit higher.
- riwsky 1y ago
  The criticism invoked “anyone with serious experience with schizophrenia”, implying the author of the article is not such a one. Citing the author’s experience is a perfectly valid rebuttal to that implication. It’s not an argument from authority, but about it.

shayway 1y ago

The article's conclusion is exactly what you describe: that AI is bringing out latent predisposition toward psychosis through runaway feedback loops, that it's a bidirectional relationship where the chemicals influence thoughts and thoughts influence chemicals until we decide to call it psychosis. I hate to be the 'you didn't read the article' guy but that line taken out of context is the exact opposite of my takeaway for the article as a whole. For anyone else who skims comments before clicking I would invite you to read the whole thing (or at least get past the poorly-worded intro) before drawing conclusions.

kayodelycaon 1y ago

You might want to read the entire article. His depiction of bipolar is completely accurate. In fact it is so precisely accurate in every detail, and conveyed with no extraneous information, is indicative of someone who knows the disorder very well. When I write fiction or important emails, I am precise with the words I use. I notice these kind of details. I’m also bipolar and self-aware enough to be deeply familiar with it.

phreeza 1y ago
The author is a psychiatrist so it would make sense that he is familiar with the subject.
- vintermann 1y ago
  And as I recall, he used to be a lot more clear that mental illness isn't always clear cut. I was surprised at the "obviously, we all know what mental illness is" attitude coming from him.
  - anon84873628 1y ago
    I imagine he's been doing this so long that taking a sober, delicate, appeal-to-every-reader approach to every article just isn't fun or practical. He wants to get down his thoughts for his core reader base and move on. What fraction goes viral anyway? And so what if some people misunderstand or don't like it? You're always going to have those people anyway no matter what :shrug:

epiccoleman 1y ago

> 'biological' mental illness is tightly coupled to qualitative mental state, and bidirectionally at that. Not only do your chemicals influence your thoughts, your thoughts influence your chemicals, and it's possible for a vulnerable person to be pushed over the edge by either kind of input. We like to think that 'as long as nothing is chemically wrong' we're a-ok, but the truth is that it's possible for simple normal trains of thought to latch your brain into a very undesirable state. It's interesting to see you mention this. After reading this post yesterday I wound up with some curious questions along these lines. I guess my question goes something like this: This article seems to assert that 'mental illness' must always have some underlying representation in the brain - that is, mental illness is caused by chemical imbalances or malformation in brain structure. But is it possible for a brain to become 'disordered' in a purely mental way? i.e. that to any way we know of "inspecting" the brain, it would look like a the hardware was healthy - but the "mind inside the brain" could somehow be stuck in a "thought trap"? Your post above seems to assert this could be the case. I think I've pretty much internalized a notion of consciousness that was purely bottom-up and materialistic. Thoughts are the product of brain state, brain state is the product of physics, which at "brain component scale" is deterministic. So it seems very spooky on its face that somehow thoughts themselves could have a bidirectional relationship with chemistry. I spent a bunch of time reading articles and (what else) chatting with Claude back and forth about this topic, and it's really interesting - it seems there are at least some arguments out there that information (or maybe even consciousness) can have causal effects on "stuff" (matter). There's the "Integrated Information Theory" of consciousness (which seems to be, if not exactly "fringe", at least widely disputed) and there's also this interesting notion of "downward causation" (basically the idea that higher-level systems can have causal effects on lower levels - I'm not clear on whether "thought having causal effects on chemistry" fits into this model). I've got 5 or 6 books coming my way from the local library system - it's a pretty fascinating topic, though I haven't dug deep enough to decide where I stand. Sorry for the ramble, but this article has at least inspired some interesting rabbit-hole diving for me. I'm curious - when you assert "Not only do your chemicals influence your thoughts, your thoughts influence your chemicals" - do you have evidence that backs that notion up? I'm not asking to cast doubt, but rather, I guess, because it sounds like maybe you've got some sources I might find interesting as I keep reading.

jedharris 1y ago

> it's possible for simple normal trains of thought to latch your brain into a very undesirable state. This seems very incorrect, or at least drastically underspecified. These trains of thought are "normal" (i.e. common and unremarkable) so why don't they "latch your brain into a very undesirable state" lots of the time? I don't think Scott or anyone up to speed on modern neuroscience would deny the coupling of mental state and brain chemistry--in fact I think it would be more accurate to say both of them are aspects of the dynamics of the brain. But this doesn't imply that "simple normal trains of thought" can latch our brain dynamics into bad states -- i.e. in dynamics language move us into a undesirable attractor. That would require a very problematic fragility in our normal self-regulation of brain dynamics.

AstralStorm 1y ago
See the key here is, the AI provides a very enticing social partner. Think of it as a version of making your drugged friend believe various random stuff. It works better if you're not a stranger and have an engaging or alarming style. LLMs are trained to produce pleasant responses that tailor to the user to maximize positive responses. (A more general version of engagement.) It stands to reason they would be effective at convincing someone.

anon84873628 1y ago

One of the questions that sets up the premise of the article in the first paragraph is, "Are the chatbots really driving people crazy, or just catching the attention of people who were crazy already?" That's why he's honing in on that specific scenario to determine if chatbots are uniquely crazy-making or something. The professional psychiatrist author is not unaware of the things you're saying. They're just not the purpose of the survey & article.

bccdee 1y ago
Well yeah, that's a false dichotomy. If you're vulnerable and a chatbot sends you into a spiral of psychosis, your pre-existing vulnerability doesn't negate the fact that a harm has been done to you. If you have a heart condition, and I shoot you with a Taser, and it kills you… I've killed you. You weren't "already dead" just because you were vulnerable.
- anon84873628 1y ago
  Yes but "what is the effect of tasers on hearts" is an interesting question when tasers are brand new. If it kills people with obvious pre-existing risks then that is not very surprising. If it kills 50% of otherwise healthy people in a way we didn't anticipate, that is alarming and important to distinguish. Imagine someone does a quick survey to estimate that tasers aren't killing people we don't expect, and some readers respond saying how dare you ignore the vulnerable heart people. That's still an important thing to consider and maybe we should be careful with the mass scale rollout of tasers, but it wasn't really the immediate point.
  - bccdee 1y ago
    > Imagine someone does a quick survey to estimate that tasers aren't killing people we don't expect Given that the quote you cited was, "Are the chatbots really driving people crazy, or just catching the attention of people who were crazy already," I'd say the equivalent would be something like, "Are tasers really killing people, or were tasered heart attack victims dying already?" And yeah, I'd be mad about that framing! The fact that the people who die had a preexisting vulnerability does not mean they were "already dying" or that they were not "really killed."