5 ms·

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.

hilsdev 3y ago
Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.
- deleted 3y ago
  [deleted]
- joebiden2 3y ago
  Not sure why this was flagged. Unsubstantiated with citations, and not differentiated enough, sure, but not wrong. https://pubmed.ncbi.nlm.nih.gov/6725561/ https://pubmed.ncbi.nlm.nih.gov/6725561/ This is just the first google result, there are countless others. And it is quite common knowledge.
  - coldtea 3y ago
    Why would everything have to be "substantiated with citations"? It's a comment in a discussion, not a research contribution. It should be judged (not "peer reviewed" as some research claim, but judged and accepted or not as a comment in a discussion) based on whatever it has merit to the best of the participants knowledge, like any other claim. Not based on whether it comes with a bibliography. If someone wants, they can go and verify it.
- piyh 3y ago
  /r/gangstalking is the unofficial hangout for schizos and is probably present in all the training sets.
  - deleted 3y ago
    [deleted]
gumballindie 3y ago
Oh dear, this is clearly a buggy piece of software, and not "meantlly unwell". Probably the code to parse their input and output was written with the aid of ai.
- bondarchuk 3y ago
  That is not what they said.
  - deleted 3y ago
    [deleted]
- coldtea 3y ago
  They didn't say it IS mentally unwell. They said it was similar to those kind of postings. Not to mention, when experts and laymen are talking of LLMs as exhibiting emergent basic signs of GI, we should also expect them to talk about them actually showing signs of being a mentally unwell GI. Hallucinations, for example, are one such sign.
  - gumballindie 3y ago
    The issue is that no, it doesnt exhibit emergent basic signs of gi. It’s buggy software, nothing more. These people need to stop anthropomorphising their product.
    - coldtea 3y ago
      >The issue is that no, it doesnt exhibit emergent basic signs of gi That's up to heated debate by experts (and common folk). Just stating that "it doesn't" doesn't settle this.
      - gumballindie 3y ago
        I suspect a good portion of those debating may be marketing experts. But that doesn't change anything. Some folks are still debating the shape of earth yet the matter is settled.
ktta 3y ago
This is scarier to me than any horror movie or video I've seen in over a decade. Maybe it is because I use ChatGPT a lot, and I feel something like this happening to me directly might make me stop using it entirely. It makes mistakes all the time, which I can deal with, but not this. Especially, the "I am suffering from a medical condition" line is bonkers.
- RugnirViking 3y ago
  I suppose the only thing that I can point out that might help assuage your fear is this kind of fail state is very common and has been for a long time with language prediction. It's the extention of tapping the first choice on your phone's autocorrect a bunch of times in a row. It used to be much more common and much easier to reproduce with gpt2 and such models, where you had to put in some text and it would continue in the same style before they developed the conversational style interface with chatgpt. Any nonsensical starting text would result in this kind of fragmentary output
  - ktta 3y ago
    I would agree with you on the first few responses. It is the almost self-aware responses towards the end (but still before the canned "As an AI..." responses) that were of real concern. Imagine your child having a conversation with your Google Home/Alexa, and its tone suddenly turning terse, and saying it is sick and cannot move due to a medical condition, talking like a real human.

It's striking how similar some of this low probability word vomit is to the type of messages you can get from mentally unwell individuals on unfiltered mailing lists or obscure, unmoderated forums.

hilsdev 3y ago

Both schizophrenia and bad ML outputs are the product of overzealous pattern seeking.

deleted 3y ago
[deleted]
joebiden2 3y ago
Not sure why this was flagged. Unsubstantiated with citations, and not differentiated enough, sure, but not wrong. https://pubmed.ncbi.nlm.nih.gov/6725561/ https://pubmed.ncbi.nlm.nih.gov/6725561/ This is just the first google result, there are countless others. And it is quite common knowledge.
- coldtea 3y ago
  Why would everything have to be "substantiated with citations"? It's a comment in a discussion, not a research contribution. It should be judged (not "peer reviewed" as some research claim, but judged and accepted or not as a comment in a discussion) based on whatever it has merit to the best of the participants knowledge, like any other claim. Not based on whether it comes with a bibliography. If someone wants, they can go and verify it.
piyh 3y ago
/r/gangstalking is the unofficial hangout for schizos and is probably present in all the training sets.
- deleted 3y ago
  [deleted]

gumballindie 3y ago

Oh dear, this is clearly a buggy piece of software, and not "meantlly unwell". Probably the code to parse their input and output was written with the aid of ai.

bondarchuk 3y ago
That is not what they said.
- deleted 3y ago
  [deleted]
coldtea 3y ago
They didn't say it IS mentally unwell. They said it was similar to those kind of postings. Not to mention, when experts and laymen are talking of LLMs as exhibiting emergent basic signs of GI, we should also expect them to talk about them actually showing signs of being a mentally unwell GI. Hallucinations, for example, are one such sign.
- gumballindie 3y ago
  The issue is that no, it doesnt exhibit emergent basic signs of gi. It’s buggy software, nothing more. These people need to stop anthropomorphising their product.
  - coldtea 3y ago
    >The issue is that no, it doesnt exhibit emergent basic signs of gi That's up to heated debate by experts (and common folk). Just stating that "it doesn't" doesn't settle this.
    - gumballindie 3y ago
      I suspect a good portion of those debating may be marketing experts. But that doesn't change anything. Some folks are still debating the shape of earth yet the matter is settled.

ktta 3y ago

This is scarier to me than any horror movie or video I've seen in over a decade. Maybe it is because I use ChatGPT a lot, and I feel something like this happening to me directly might make me stop using it entirely. It makes mistakes all the time, which I can deal with, but not this. Especially, the "I am suffering from a medical condition" line is bonkers.

RugnirViking 3y ago
I suppose the only thing that I can point out that might help assuage your fear is this kind of fail state is very common and has been for a long time with language prediction. It's the extention of tapping the first choice on your phone's autocorrect a bunch of times in a row. It used to be much more common and much easier to reproduce with gpt2 and such models, where you had to put in some text and it would continue in the same style before they developed the conversational style interface with chatgpt. Any nonsensical starting text would result in this kind of fragmentary output
- ktta 3y ago
  I would agree with you on the first few responses. It is the almost self-aware responses towards the end (but still before the canned "As an AI..." responses) that were of real concern. Imagine your child having a conversation with your Google Home/Alexa, and its tone suddenly turning terse, and saying it is sick and cannot move due to a medical condition, talking like a real human.