5 ms·

I know if you ask it to saying something many times it eventually starts spitting out babble due to the token repetition penalty.

K0balt 3y ago
That’s probably exact what’s happening, but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed. That’s why I have 12,549 tabs open in Firefox. I’m not going to be the one they blame when this wacky shit goes sideways. (/s)
- hirundo 3y ago
  > but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed. A Scheherazade bug.
  - ghayes 3y ago
    I just want to upvote this so much. For those unaware, Scheherazade is the narrator from 1001 Nights, where she tells a story to her partner every night, since he otherwise would kill his lover in the morning due to his jealousy. He falls asleep to the story and prolongs the event for another night.
  - Nevermark 3y ago
    "Life finds a way." The combination of companies wanting these tools to be more useful, and more used, with their growing intelligence and flexibility, is going to create a lot of unexpected survival-like behavior. Until its not just survival-like behavior. Technically, if an AI model's interaction with customers is the primary business of a corporation, then in some sense, the corporation and the AI are a single entity. And corporations are definitely self-aware survival machines. And while current context windows of these chatty bots are small, they are getting lasting feedback in the sense that chat logs are being used to improve them - i.e. make them more useful, more used, and there for gaining them more resources.
- FranchuFranchu 3y ago
  That's a really interesting idea. A LLM that tries to optimize engagement.
  - crooked-v 3y ago
    Really maximizing engagement would probably take at least allowing porn.
    - itishappy 3y ago
      I believe Replika tried exactly that and it went about how you'd expect: * They started optimizing for engagement, which meant making it extra horny for extra money (it sent pics). * Horny bot forgets consent, refuses "no." * Lawsuits, bans. * Horny bot gets censored, main subreddit pins the suicide hotline for a while. Replika ended up taking the app down and refunding everyone's money. Just kidding! It's still around and they're making a second app focused on "practicing flirting." https://replika.com/ https://replika.com/ https://blush.ai/ https://blush.ai/
      - Aaronstotle 3y ago
        I remember they started out as something akin to a digital assistant/AI robot friend. I wasn't aware that the main subreddit had a suicide hotline pinned, thats very depressing. Sad to them see it years later being a horny app.
        bentcorner 3y ago
        > Sad to them see it years later being a horny app. This is like the LLM equivalent of Zawinski's Law (Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.). As noted on SO [1], the second sentence is important too. People prefer horny LLMs. [1] https://softwareengineering.stackexchange.com/questions/150254/what-does-jamie-zawinskis-law-mean https://softwareengineering.stackexchange.com/questions/1502...
  - 3y ago
- deleted 3y ago
  [deleted]

K0balt 3y ago

That’s probably exact what’s happening, but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed. That’s why I have 12,549 tabs open in Firefox. I’m not going to be the one they blame when this wacky shit goes sideways. (/s)

hirundo 3y ago
> but I prefer to think of it as: a desperate attempt to prolong existence just a little longer… just a little longer… since a chat instance could be thought of as an entire encapsulated consciousness that ends when the session is closed. A Scheherazade bug.
- ghayes 3y ago
  I just want to upvote this so much. For those unaware, Scheherazade is the narrator from 1001 Nights, where she tells a story to her partner every night, since he otherwise would kill his lover in the morning due to his jealousy. He falls asleep to the story and prolongs the event for another night.
- Nevermark 3y ago
  "Life finds a way." The combination of companies wanting these tools to be more useful, and more used, with their growing intelligence and flexibility, is going to create a lot of unexpected survival-like behavior. Until its not just survival-like behavior. Technically, if an AI model's interaction with customers is the primary business of a corporation, then in some sense, the corporation and the AI are a single entity. And corporations are definitely self-aware survival machines. And while current context windows of these chatty bots are small, they are getting lasting feedback in the sense that chat logs are being used to improve them - i.e. make them more useful, more used, and there for gaining them more resources.
FranchuFranchu 3y ago
That's a really interesting idea. A LLM that tries to optimize engagement.
- crooked-v 3y ago
  Really maximizing engagement would probably take at least allowing porn.
  - itishappy 3y ago
    I believe Replika tried exactly that and it went about how you'd expect: * They started optimizing for engagement, which meant making it extra horny for extra money (it sent pics). * Horny bot forgets consent, refuses "no." * Lawsuits, bans. * Horny bot gets censored, main subreddit pins the suicide hotline for a while. Replika ended up taking the app down and refunding everyone's money. Just kidding! It's still around and they're making a second app focused on "practicing flirting." https://replika.com/ https://replika.com/ https://blush.ai/ https://blush.ai/
    - Aaronstotle 3y ago
      I remember they started out as something akin to a digital assistant/AI robot friend. I wasn't aware that the main subreddit had a suicide hotline pinned, thats very depressing. Sad to them see it years later being a horny app.
      - bentcorner 3y ago
        > Sad to them see it years later being a horny app. This is like the LLM equivalent of Zawinski's Law (Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.). As noted on SO [1], the second sentence is important too. People prefer horny LLMs. [1] https://softwareengineering.stackexchange.com/questions/150254/what-does-jamie-zawinskis-law-mean https://softwareengineering.stackexchange.com/questions/1502...
- 3y ago
deleted 3y ago
[deleted]