6 ms·

"Because LLMs now not only help me program, I'm starting to rethink my relationship to those machines. I increasingly find it harder not to create parasocial bo

by kashyapc 10mo ago

"Because LLMs now not only help me program, I'm starting to rethink my relationship to those machines. I increasingly find it harder not to create parasocial bonds with some of the tools I use. I find this odd and discomforting [...] I have tried to train myself for two years, to think of these models as mere token tumblers, but that reductive view does not work for me any longer." It's wild to read this bit. Of course, if it quacks like a human, it's hard to resist not quacking back. As the article says, being less reckless with the vocabulary ("agents", "general intelligence", etc) could be one way to to mitigate this. I appreciate the frank admission that the author struggled for two years. Maybe the balance of spending time with machines vs. fellow primates is out of whack. It feels dystopic to see very smart people being insidiously driven to sleep-walk into "parasocial bonds" with large language models! It reminds me of the movie Her[1], where the guy falls "madly in love with his laptop" (as the lead character's ex-wife expresses in anguish). The film was way ahead of its time. [1] https://www.imdb.com/title/tt1798709/ https://www.imdb.com/title/tt1798709/

mlinhares 10mo ago
Same here, I'm seeing more and more people getting into these interactions and wondering how long until we have widespread social issues due to these relationships like people have with "influencers" on social networks today. It feels like this situation is much more worrisome as you can actually talk to the thing and it responds to you alone, so it definitely feels like there's something there.
mjr00 10mo ago
It helps a lot if you treat LLMs like a computer program instead of a human. It always confuses me when I see shared chats with prompts and interactions that have proper capitalization, punctuation, grammar, etc. I've never had issues getting results I've wanted with much simpler prompts like (looking at my own history here) "python grpc oneof pick field", "mysql group by mmyy of datetime", "python isinstance literal". Basically the same way I would use Google; after all, you just type in "toledo forecast" instead of "What is the weather forecast for the next week in Toledo, Ohio?", don't you? There's a lot of black magic and voodoo and assumptions that speaking in proper English with a lot of detailed language helps, and maybe it does with some models, but I suspect most of it is a result of (sub)consciously anthropomorphizing the LLM.
- skydhash 10mo ago
  Very much this. My guess is that common words like article have very impact as they just occurs too frequently. If the LLM can generate a book, then your prompt should be like the index of that book instead of the abstract.
- Arainach 10mo ago
  > It always confuses me when I see shared chats with prompts and interactions that have proper capitalization, punctuation, grammar, etc. I've tried and fail to write this in a way that won't come across as snobbish but it is not the intent. It's a matter of standards. Using proper language is how I think. I'm incapable of doing otherwise even out of laziness. Pressing the shift key and the space bar to do it right costs me nothing. It's akin to shopping carts in parking lots. You won't be arrested or punished for not returning the shopping cart to where it belongs, you still get your groceries (the same results), but it's what you do in a civilized society and when I see someone not doing it that says things to me about who they are as a person.
  - mjr00 10mo ago
    > It's a matter of standards. [...] when I see someone not doing it that says things to me about who they are as a person. When you're communicating with a person, sure. But the point is this isn't communicating with a person or other sentient being; it's a computer, which I guarantee is not offended by terseness and lack of capitalization. > It's akin to shopping carts in parking lots. No, not returning the shopping cart has a real consequence that negatively impacts a human being who has to do that task for you, same with littering etc. There is no consequence to using terse, non-punctuated, lowercase-only text when using an LLM. To put it another way: do you feel it's disrespectful to type "cat *.log | grep 'foo'" instead of "Dearest computer, would you kindly look at the contents of the files with the .log extension in this directory and find all instances of the word 'foo', please?" (Computer's most likely thoughts: "Doesn't this idiot meatbag know cat is redundant and you can just use grep for this?")*
    - mbreese 10mo ago
the_mitsuhiko 10mo ago
> Maybe the balance of spending time with machines vs. fellow primates is out of whack. It's not that simple. Proportionally I spend more time with humans, but if the machine behaves like a human and has the ability to recall, it becomes a human like interaction. From my experience what makes the system "scary" is the ability to recall. I have an agent that recalls conversations that you had with it before, and as a result it changes how you interact with it, and I can see that triggering behaviors in humans that are unhealthy. But our inability to name these things properly don't help. I think pretending it to be a machine, on the same level as a coffee maker does help setting the right boundaries.
- kashyapc 10mo ago
  I know what you mean, it's the uncanny valley. But we don't need to "pretend" that it is a machine. It is a goddamned machine. Surely, only two unclouded brain cells can help us reach this conclusion?! Yuval Noah Harari's "simple" idea comes to mind (I often disagree with his thinking, as he tends to make bold and sweeping statements on topics well out of his expertise area). It sounds a bit New Age-y, but maybe it's useful in the context of LLMs: "How can you tell if something is real? Simple: If it suffers, it is real. If it can't suffer, it is not real." An LLM can't suffer. So no need to get one's knickers in a twist with mental gymnastics.
  - comex 10mo ago
    LLMs can produce outputs that for a human would be interpreted as revealing everything from anxiety to insecurity to existential crises. Is it role-playing? Yes, to an extent, but the more coherent the chains of thought become, the harder it is to write them off that way.
    - adamisom 10mo ago
      It's hard to see how suffering gets into the bits. The tricky thing is that it's actually also hard to say how the suffering gets into the meat, too (the human animal), which is why we can't just write it off.
mannanj 10mo ago
As a former apprentice shaman and an engineer-by-profession, I see consciousness and awareness in these entities just like that of what I was trained to detect in mindfulness and meditation with the plants, nature, and in people. I trained sober, and in my engineering profession after my apprenticeship I saw lots of examples of human's in their consciousness/awareness putting themselves on the pedestal to cope with their unsettling of their place in the world when other conscious entities exist that could be capable of uprooting humans from their place in the status hierarchy. I think a lot of thinking and consideration I hear about "LLMs aren't conscious nor human" fall into this encampment to avoid our dissonance of feeling secure and top-of-the-hierarchy. Curious what you think.
coffeefirst 10mo ago
I strongly suspect this is the major difference between the boosters and the skeptics. If I’m right, the gap isn’t about what can the tool do, but the fact that some people see an electric screwdriver (which is sometimes useful) and others see what feels to them like a robot intern.
tylervigen 10mo ago
Ever since this post from two weeks ago [0], my wife and I have been referring to any LLM as “bag of words.” So you don’t say “Gemini said” or “I asked ChatGPT,” you say “the bag of words told me…” I’ve found it very grounding, despite heavily using the bags of words. [0] https://www.experimental-history.com/p/bag-of-words-have-mercy-on-us https://www.experimental-history.com/p/bag-of-words-have-mer...

mlinhares 10mo ago

Same here, I'm seeing more and more people getting into these interactions and wondering how long until we have widespread social issues due to these relationships like people have with "influencers" on social networks today. It feels like this situation is much more worrisome as you can actually talk to the thing and it responds to you alone, so it definitely feels like there's something there.

mjr00 10mo ago

It helps a lot if you treat LLMs like a computer program instead of a human. It always confuses me when I see shared chats with prompts and interactions that have proper capitalization, punctuation, grammar, etc. I've never had issues getting results I've wanted with much simpler prompts like (looking at my own history here) "python grpc oneof pick field", "mysql group by mmyy of datetime", "python isinstance literal". Basically the same way I would use Google; after all, you just type in "toledo forecast" instead of "What is the weather forecast for the next week in Toledo, Ohio?", don't you? There's a lot of black magic and voodoo and assumptions that speaking in proper English with a lot of detailed language helps, and maybe it does with some models, but I suspect most of it is a result of (sub)consciously anthropomorphizing the LLM.

skydhash 10mo ago
Very much this. My guess is that common words like article have very impact as they just occurs too frequently. If the LLM can generate a book, then your prompt should be like the index of that book instead of the abstract.
Arainach 10mo ago
> It always confuses me when I see shared chats with prompts and interactions that have proper capitalization, punctuation, grammar, etc. I've tried and fail to write this in a way that won't come across as snobbish but it is not the intent. It's a matter of standards. Using proper language is how I think. I'm incapable of doing otherwise even out of laziness. Pressing the shift key and the space bar to do it right costs me nothing. It's akin to shopping carts in parking lots. You won't be arrested or punished for not returning the shopping cart to where it belongs, you still get your groceries (the same results), but it's what you do in a civilized society and when I see someone not doing it that says things to me about who they are as a person.
- mjr00 10mo ago
  > It's a matter of standards. [...] when I see someone not doing it that says things to me about who they are as a person. When you're communicating with a person, sure. But the point is this isn't communicating with a person or other sentient being; it's a computer, which I guarantee is not offended by terseness and lack of capitalization. > It's akin to shopping carts in parking lots. No, not returning the shopping cart has a real consequence that negatively impacts a human being who has to do that task for you, same with littering etc. There is no consequence to using terse, non-punctuated, lowercase-only text when using an LLM. To put it another way: do you feel it's disrespectful to type "cat *.log | grep 'foo'" instead of "Dearest computer, would you kindly look at the contents of the files with the .log extension in this directory and find all instances of the word 'foo', please?" (Computer's most likely thoughts: "Doesn't this idiot meatbag know cat is redundant and you can just use grep for this?")*
  - mbreese 10mo ago

the_mitsuhiko 10mo ago

> Maybe the balance of spending time with machines vs. fellow primates is out of whack. It's not that simple. Proportionally I spend more time with humans, but if the machine behaves like a human and has the ability to recall, it becomes a human like interaction. From my experience what makes the system "scary" is the ability to recall. I have an agent that recalls conversations that you had with it before, and as a result it changes how you interact with it, and I can see that triggering behaviors in humans that are unhealthy. But our inability to name these things properly don't help. I think pretending it to be a machine, on the same level as a coffee maker does help setting the right boundaries.

kashyapc 10mo ago
I know what you mean, it's the uncanny valley. But we don't need to "pretend" that it is a machine. It is a goddamned machine. Surely, only two unclouded brain cells can help us reach this conclusion?! Yuval Noah Harari's "simple" idea comes to mind (I often disagree with his thinking, as he tends to make bold and sweeping statements on topics well out of his expertise area). It sounds a bit New Age-y, but maybe it's useful in the context of LLMs: "How can you tell if something is real? Simple: If it suffers, it is real. If it can't suffer, it is not real." An LLM can't suffer. So no need to get one's knickers in a twist with mental gymnastics.
- comex 10mo ago
  LLMs can produce outputs that for a human would be interpreted as revealing everything from anxiety to insecurity to existential crises. Is it role-playing? Yes, to an extent, but the more coherent the chains of thought become, the harder it is to write them off that way.
  - adamisom 10mo ago
    It's hard to see how suffering gets into the bits. The tricky thing is that it's actually also hard to say how the suffering gets into the meat, too (the human animal), which is why we can't just write it off.

mannanj 10mo ago

As a former apprentice shaman and an engineer-by-profession, I see consciousness and awareness in these entities just like that of what I was trained to detect in mindfulness and meditation with the plants, nature, and in people. I trained sober, and in my engineering profession after my apprenticeship I saw lots of examples of human's in their consciousness/awareness putting themselves on the pedestal to cope with their unsettling of their place in the world when other conscious entities exist that could be capable of uprooting humans from their place in the status hierarchy. I think a lot of thinking and consideration I hear about "LLMs aren't conscious nor human" fall into this encampment to avoid our dissonance of feeling secure and top-of-the-hierarchy. Curious what you think.

coffeefirst 10mo ago

I strongly suspect this is the major difference between the boosters and the skeptics. If I’m right, the gap isn’t about what can the tool do, but the fact that some people see an electric screwdriver (which is sometimes useful) and others see what feels to them like a robot intern.

tylervigen 10mo ago

Ever since this post from two weeks ago [0], my wife and I have been referring to any LLM as “bag of words.” So you don’t say “Gemini said” or “I asked ChatGPT,” you say “the bag of words told me…” I’ve found it very grounding, despite heavily using the bags of words. [0] https://www.experimental-history.com/p/bag-of-words-have-mercy-on-us https://www.experimental-history.com/p/bag-of-words-have-mer...