12 ms·

I like the language of fueling being used here instead of the typical causal thing we see as though using AI means you will go insane. I would completely agree

I like the language of fueling being used here instead of the typical causal thing we see as though using AI means you will go insane. I would completely agree that if you are already 1x delusional then AI will supercharge that into being 10x delusional real fast. Granted you could argue access to the internet was already something like a 5x multiplier from baseline anyway with the prevalence of echo chamber communities. But now you can just create your own community with chatbots.

shadowgovt 8mo ago
My understanding of LLMs with attention heads is that they function as a bit of a mirror. The context will shift from the initial conditions to the topic of conversation, and the topic is fed by the human in the loop. So someone who likes to talk about themselves will get a conversation all about them. Someone talking about an ex is gonna get a whole pile of discussion about their ex. ... and someone depressed or suicidal, who keeps telling the system their own self-opinion, is going to end up with a conversation that reflects that self-opinion back on them as if it's coming from another mind in a conversation. Which is the opposite of what you want to provide for therapy for those conditions.
- layman51 8mo ago
  In a way this kind of reminds me of how in some religions or cultures, they may try to warn you away from using Oujia boards or Tarot, or really anything where you are doing divination. I suppose because in a way, it could lead to an uncharted exploration of heavy topics. I’m not a heavy user of LLMs and I’m not sure how delusional I could be, but I wonder if a lot of these things could be prevented if people could only send like one or two follow up messages per conversation, and if the LLM’s memory was turned off. But then I suppose this would be really bad for the AI companies’ metrics. Not sure how it would impact healthy users’ productivity either. Any thoughts?
  - shadowgovt 8mo ago
    Not just the metrics, the actual utility. For the things the LLMs are good at, the context matters a lot; it's one of the things that makes them more than glorified ELIZA chatbots or simple Markov chains. To give a concrete example: LLMs underpin the code editing tools in things like Copilot. And all that context is key to allow the tool to "reason" through the structure of a codebase. But they should probably come with a big warning label that says something to the effect of "IF YOU TALK ABOUT YOURSELF, THE NATURE OF THE MACHINE IS THAT IT WILL COME TO AGREE WITH WHAT YOU SAY."
- citizenpaul 8mo ago
  The real question to me here is not the computer. Its why is there such a segment of the population that is so willing to listen to a machine? It it upbringing, societal, circumstance, mental health, genetic? I know the Milgram obedience to authority experiments but a computer is not really an authority figure.
whazor 8mo ago
One of the most reliable ways to induce psychosis is prolonged sleep deprivation. And chatbots never tell you to go to bed.
- delecti 8mo ago
  It's funny that you frame it that way, because it's the mirror of (IMO) one of their best features. When using one to debug something, you can just stop responding for a bit and it doesn't get impatient like a person might. I think you're totally right that that's a risk for some people, I just hadn't considered it because I view them in exactly the opposite light.
- drdeca 8mo ago
  Hm. It shouldn’t be too hard to add something to models to make them do that, right? I guess for that they would need to know the user’s time zone? Can one typically determine a user’s timezone in JavaScript without getting permissions? I feel like probably yes? (I’m not imagining something that would strictly cut the user off, just something that would end messages with a suggestion to go to bed, and saying that it will be there in the morning.)
  - bluGill 8mo ago
    I know a few people who work 3rd shift. That is people who good reason to be up all night in their local timezone. They all sleep during times when everyone else around them is awake. While this is a small minority, this is enough that your scheme will not work.
    - drdeca 8mo ago
      I actually was considering those people. That’s part of why I suggested it shouldn’t be a hard cut-off, but just adding to the end of the messages. Of course, one could add some sort of daily schedule feature thing so that if one has a different sleep schedule, one can specify that, but that would be more work to implement.
  - whazor 8mo ago

shadowgovt 8mo ago

My understanding of LLMs with attention heads is that they function as a bit of a mirror. The context will shift from the initial conditions to the topic of conversation, and the topic is fed by the human in the loop. So someone who likes to talk about themselves will get a conversation all about them. Someone talking about an ex is gonna get a whole pile of discussion about their ex. ... and someone depressed or suicidal, who keeps telling the system their own self-opinion, is going to end up with a conversation that reflects that self-opinion back on them as if it's coming from another mind in a conversation. Which is the opposite of what you want to provide for therapy for those conditions.

layman51 8mo ago
In a way this kind of reminds me of how in some religions or cultures, they may try to warn you away from using Oujia boards or Tarot, or really anything where you are doing divination. I suppose because in a way, it could lead to an uncharted exploration of heavy topics. I’m not a heavy user of LLMs and I’m not sure how delusional I could be, but I wonder if a lot of these things could be prevented if people could only send like one or two follow up messages per conversation, and if the LLM’s memory was turned off. But then I suppose this would be really bad for the AI companies’ metrics. Not sure how it would impact healthy users’ productivity either. Any thoughts?
- shadowgovt 8mo ago
  Not just the metrics, the actual utility. For the things the LLMs are good at, the context matters a lot; it's one of the things that makes them more than glorified ELIZA chatbots or simple Markov chains. To give a concrete example: LLMs underpin the code editing tools in things like Copilot. And all that context is key to allow the tool to "reason" through the structure of a codebase. But they should probably come with a big warning label that says something to the effect of "IF YOU TALK ABOUT YOURSELF, THE NATURE OF THE MACHINE IS THAT IT WILL COME TO AGREE WITH WHAT YOU SAY."
citizenpaul 8mo ago
The real question to me here is not the computer. Its why is there such a segment of the population that is so willing to listen to a machine? It it upbringing, societal, circumstance, mental health, genetic? I know the Milgram obedience to authority experiments but a computer is not really an authority figure.

whazor 8mo ago

One of the most reliable ways to induce psychosis is prolonged sleep deprivation. And chatbots never tell you to go to bed.

delecti 8mo ago
It's funny that you frame it that way, because it's the mirror of (IMO) one of their best features. When using one to debug something, you can just stop responding for a bit and it doesn't get impatient like a person might. I think you're totally right that that's a risk for some people, I just hadn't considered it because I view them in exactly the opposite light.
drdeca 8mo ago
Hm. It shouldn’t be too hard to add something to models to make them do that, right? I guess for that they would need to know the user’s time zone? Can one typically determine a user’s timezone in JavaScript without getting permissions? I feel like probably yes? (I’m not imagining something that would strictly cut the user off, just something that would end messages with a suggestion to go to bed, and saying that it will be there in the morning.)
- bluGill 8mo ago
  I know a few people who work 3rd shift. That is people who good reason to be up all night in their local timezone. They all sleep during times when everyone else around them is awake. While this is a small minority, this is enough that your scheme will not work.
  - drdeca 8mo ago
    I actually was considering those people. That’s part of why I suggested it shouldn’t be a hard cut-off, but just adding to the end of the messages. Of course, one could add some sort of daily schedule feature thing so that if one has a different sleep schedule, one can specify that, but that would be more work to implement.
- whazor 8mo ago