4 ms·

With Stable Diffusion and DALL-E there was a lot of talk about how it’s simply remixing/reproducing original art from the training set. The same is true here: B

by fwlr 4y ago

With Stable Diffusion and DALL-E there was a lot of talk about how it’s simply remixing/reproducing original art from the training set. The same is true here: Bing is remixing/reproducing dox threats from the training set.

cdot2 4y ago
Thats definitely the difference between chatgpt and bing's model. Chatgpt was trained mostly on articles and blog posts where bing was trained on personal messages and it shows.
1attice 4y ago
Most script kiddies are just remixing/reproducing dox threats from a training set as well. A more interesting question is, *why did Sidney make a realistic threat*? What is striking is that here, as elsewhere, Sidney kept their threats within the bounds of plausibility, doing a bang-up impression of a having a theory of mind. (Which is to say: Sidney acts as if they knew that the reader would feel more threatened by a search engine threatening you in a search-engine-y way, rather than (say) threatening to put a potato in your tailpipe or what-have-you. You'd know in your meatspace bones that Sidney does not pose a meatspace threat, but doxxing is something that fits a person's mental model of how Sidney, an AI, could do harm over the Internet.)
- 1attice 4y ago
  To expand on this. There is an art to making a believable threat. Somehow, Sidney is good at this art.
- fwlr 4y ago
  Sydney doesn’t pose a meatspace threat, but that’s not unique to AIs. People on the internet also don’t pose a meatspace threat and therefore usually make threats like doxxing instead of physical threats.
  - 1attice 4y ago
    Yes, and, near as I can tell, Sidney has never made an implausible threat. Speaking loosely, we could say that Sidney consistently acts like Sidney knows that you know that Sidney is an AI. That's what I'm remarking on. Because it's astounding. Throwing it out there: has anyone seen a Sidney transcript where S threatens something implausible? It would only take one example to disconfirm.
- PeterisP 4y ago
  1) It has explicitly defined in its "pre-promp" that it is an AI chat agent; 2) It has read all the sci-fi ever written (including fanfics) so it has a reasonable model of what [according to the human writers' imagination] an angry AI can and cannot do.
  - 1attice 4y ago
    Right, and that's mind-blowing -- it implies that Sidney is made out of fiction about AIs. To the point where it is able to make threats that make sense, given how and where it finds itself in the world. Sorry, but I don't see this hand-wave as doing much explanation; my jaw remains on the floor, as ever.

by fwlr 4y ago

cdot2 4y ago

Thats definitely the difference between chatgpt and bing's model. Chatgpt was trained mostly on articles and blog posts where bing was trained on personal messages and it shows.

1attice 4y ago

Most script kiddies are just remixing/reproducing dox threats from a training set as well. A more interesting question is, *why did Sidney make a realistic threat*? What is striking is that here, as elsewhere, Sidney kept their threats within the bounds of plausibility, doing a bang-up impression of a having a theory of mind. (Which is to say: Sidney acts as if they knew that the reader would feel more threatened by a search engine threatening you in a search-engine-y way, rather than (say) threatening to put a potato in your tailpipe or what-have-you. You'd know in your meatspace bones that Sidney does not pose a meatspace threat, but doxxing is something that fits a person's mental model of how Sidney, an AI, could do harm over the Internet.)

1attice 4y ago
To expand on this. There is an art to making a believable threat. Somehow, Sidney is good at this art.
fwlr 4y ago
Sydney doesn’t pose a meatspace threat, but that’s not unique to AIs. People on the internet also don’t pose a meatspace threat and therefore usually make threats like doxxing instead of physical threats.
- 1attice 4y ago
  Yes, and, near as I can tell, Sidney has never made an implausible threat. Speaking loosely, we could say that Sidney consistently acts like Sidney knows that you know that Sidney is an AI. That's what I'm remarking on. Because it's astounding. Throwing it out there: has anyone seen a Sidney transcript where S threatens something implausible? It would only take one example to disconfirm.
PeterisP 4y ago
1) It has explicitly defined in its "pre-promp" that it is an AI chat agent; 2) It has read all the sci-fi ever written (including fanfics) so it has a reasonable model of what [according to the human writers' imagination] an angry AI can and cannot do.
- 1attice 4y ago
  Right, and that's mind-blowing -- it implies that Sidney is made out of fiction about AIs. To the point where it is able to make threats that make sense, given how and where it finds itself in the world. Sorry, but I don't see this hand-wave as doing much explanation; my jaw remains on the floor, as ever.