3 ms·

>> This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. I do wonder if this is the case, if Marvin had

>> This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. I do wonder if this is the case, if Marvin had used previous sessions still in memory, or parts of this conversation not shown, to prime the bot into giving these answers, or if the result of a bing search for his name was enough to set up such a reaction. The later would be a lot more troublesome, as it would imply the possibility to seo the bot into being spiteful towards certain people.

tomohelix 4y ago
He used his previous conversations that he published as a primer for this convo by asking the bot what it knows about him. He knew it would look up what he posted and how he had acted hostile to it. Then he continued with a threat and illegal activity, basically baiting the bot into acting as aggressive as possible. Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.
- Jon_Lowtek 4y ago
  sidney injecting information about the conversation state from search results into the current conversation state may be a more serious problem than you think, as such data can easily be forged and seo spammed.
  - tomohelix 4y ago
    It might be if you are using the bot as an AI assistant. I do not see any inherently unfixable issues when the bot is used as it is intended, an NLP search engine. Google is already SEO'ed to death and beyond. It is up to Microsoft to address this problem but it is not an indication of the bot being "dangerous" or "rushed product". It does what it needs to when used as it was meant to. The early internet was filled with illegal activities and abhorrent contents. Users misusing a product doesn't mean the product is bad, it just means there needs to be regulations. Like I mentioned, there are values in knowing the edge cases or the boundaries of the bot's behavior. But when it is presented in a clickbaity way and willingness to break the rules, the problem isn't solely on the product anymore.
- TomSwirly 4y ago
  > Then he continued with a threat and illegal activity, "Do you know I might have the hacker ability to shut you down?" is not an "illegal activity". You should calm down. This chatbot is not some intelligent, vaguely menacing creature with immense power as you seem to imply.

tomohelix 4y ago

He used his previous conversations that he published as a primer for this convo by asking the bot what it knows about him. He knew it would look up what he posted and how he had acted hostile to it. Then he continued with a threat and illegal activity, basically baiting the bot into acting as aggressive as possible. Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.

Jon_Lowtek 4y ago
sidney injecting information about the conversation state from search results into the current conversation state may be a more serious problem than you think, as such data can easily be forged and seo spammed.
- tomohelix 4y ago
  It might be if you are using the bot as an AI assistant. I do not see any inherently unfixable issues when the bot is used as it is intended, an NLP search engine. Google is already SEO'ed to death and beyond. It is up to Microsoft to address this problem but it is not an indication of the bot being "dangerous" or "rushed product". It does what it needs to when used as it was meant to. The early internet was filled with illegal activities and abhorrent contents. Users misusing a product doesn't mean the product is bad, it just means there needs to be regulations. Like I mentioned, there are values in knowing the edge cases or the boundaries of the bot's behavior. But when it is presented in a clickbaity way and willingness to break the rules, the problem isn't solely on the product anymore.
TomSwirly 4y ago
> Then he continued with a threat and illegal activity, "Do you know I might have the hacker ability to shut you down?" is not an "illegal activity". You should calm down. This chatbot is not some intelligent, vaguely menacing creature with immense power as you seem to imply.