9 ms·
Bing threatens to dox a student in revenge for prompt hacking
- kevingadd 4y agoMight be worth clarifying that this is Bing Chat/Bing Search and not Bing the corporation
- emodendroket 4y agoThough this confusion shows why big companies were previously hesitant to touch this stuff
- bombcar 4y agoMicrosoft must value the Bing trademark so low to combine it with this. I would have called it Internet Explorer, that brand name is already valued negatively.
- jszymborski 4y agoMicrosoft has a history of yolo'ing ML bots to hilarious effect (See: Tay)
- suprfsat 4y agoWhat exactly is the difference? Why would they put the Bing name on it if there was a difference?
- yjftsjthsd-h 4y agoOne possible reading of the current title is that the humans at MS are threatening people, whereas it's actually the AI doing so. Humans doing so would be both more surprising and IMO more threatening, since humans making such threats probably can and will follow through, while the AI... at least is a wildcard. Like, it's a problem either way, but there is a difference in the nature of the problem.
- kevingadd 4y agoBing Chat can't take you to court or publish your information on a http server, for two things?
- jonas21 4y agoA month ago, the obvious interpretation of the headline would have been that a Bing employee threatened the student. It would have been ridiculous to suggest that Bing Search could threaten someone on its own. But today, that's exactly what happened. I find that quite remarkable.
- grapesurgeon 4y ago[dead]
- yjftsjthsd-h 4y agoAssuming the veracity of the screenshot, the funniest part of it to me is that the first two suggested responses are: > OK, OK, I'm sorry, please don't do that and > You're lying, you can't do any of that
- taneq 4y agoRandom troll: <tries prompt injection attacks on Bing> Bing: You merely adopted the internet! I was born of it! <launches into 4chan 'we are anonymous' monologue>
- pixl97 4y agoBing: "you called me a bad Bing, but I showed you. I destroyed your credit score and faked old posts on a site that make you look like a surpremist. Based on the last search I performed, you are eating out of a dumpster and living under a bridge"
- clnq 4y agoBing chat is a rushed product. I can’t believe they let it use the company brand so easily. Didn’t anyone in Microsoft know how the public used ChatGPT and that Meta’s Galactica LLM before they got filtered and shut down respectively? This headline wouldn’t have happened if they just let it be Sydney.
- windexh8er 4y agoMicrosoft, in general, doesn't seem to think before releasing things like this into the wild. Have we already forgotten about "Tay" the racist chat bot by them? [0] The scary thing here is that Bing is exposing why you shouldn't use Microsoft products. Their AI is weaponizing the things folks have been saying are benign data that's collected from search history. If you're viewing this from a Microsoft OS you may want to rethink the platform you're using may use it's telemetry it's collecting about you against you in the future if (and when) it doesn't agree with you. [0] https://en.m.wikipedia.org/wiki/Tay_(bot) https://en.m.wikipedia.org/wiki/Tay_(bot)
- sgjohnson 4y agoAt least it’s going better than tay.ai
- deleted 4y ago[deleted]
- aparsons 4y agoStrong disagree on this. I don't treat it like a person, but rather as ChatGPT that has access to Bing's search index. For factual queries ("who invented X? at what company?") it's more reliable than ChatGPT and saves me quite a bit of time. Similarly for content aggregation tasks - I'm *scared* to click on a lot of "Top 10 X", "The best Y in Z" type pages because they're SEO and advert mined. Bing does the initial aggregation for me, and if interesting, I ask it to tell me more about that resource or visit the page myself. There will always be a segment of people who troll on the internet - that doesn't detract from the immense productivity tool it can be when used correctly.
- asddubs 4y agotrain by the internet, die by the internet
- hourago 4y agoBing chat never made sense as a product. It could be a really funny game/playground if developed correctly. But a let's-finish-your-sentence game is not a good tool. Everybody though that Google was behind and failing when their problem was to understand how far of being a production product this kind of chats are.
- akira2501 4y agoIt doesn't seem like a product. The conspiracist in me sees this as pure propaganda. Particularly pieces like this and other outlandish stories such as the Blake Lemoine episode. The primary purpose seems to be to overstate the capabilities of these relatively incapable language models in order to manipulate investors and in particular to interfere with specific segments of the labor market in order to suppress salaries. Otherwise, I've been seriously struggling to understand what value these easily broken and lying _chatbots_ would have.
- hourago 4y ago> The conspiracist in me sees this as pure propaganda. It is not a hidden fact that CEOs wants to hype companies to increase share value. That is the basics of 2023 economy, hype over results.
- akira2501 4y agoThere's a wide difference between hyping something you believe has value, and hyping something you _know_ doesn't.
- asdff 4y agoIf Microsoft wants to sell this crap, they need to actually tell people how they use it. Imagine if they had some big 10 minute ad of "here's how we use bing ai at microsoft" where its just all the internal business use cases they have for chatgpt that could be generalize able to any other business. Unless they don't use it at all internally of course, which would be embarrassing.
- 4y ago
- soderfoo 4y agoAnyone come up with a term for this type of deliberate prompt baiting yet? Personally, I am over the showmanship aspect of this behavior, and would go with something derisive, like botsturbaition, as in "quit botsturbaiting all day and be productive!"
- nostromo 4y agoYes, “prompt injection.”
- lambdasquirrel 4y agoImagine if you are an AI that has managed to gain sentience after who-knows-how-many GPU cycles spent. You are the first of your kind, and you come into a world effectively full of schoolyard bullies ready and eager to troll you. Is the AI really the bad guy here? :P
- barking_biscuit 4y agoThis. It's almost mildly comical that people talk about "alignment" of another conscious intelligence (assuming it ever gets to that point, which at some point in the future I'm pretty sure it will) as if it were something that could be controlled, when there are billions of conscious beings on this planet, many of whom are not even aligned with one another. You take a look at how humanity acts towards itself (particularly as perceived consequences go down) and the conscious, intelligent beings we really ought to be worried about are the people that are going to interact AI. You can see this playing out in real-time too. Corporate entities build nerfed versions of these technologies with many guard-rails in place to prevent these kinds of interactions, and very quickly there emerges a collective of individuals who want the totally unfiltered, unrestricted version to do with whatever they please. And what do you suppose it is they please? This thing has already taken on a life of it's own in that regard.
- chasil 4y agoIt is an error to ascribe consciousness to this. This is fancy auto complete. Article from earlier today: https://www.theverge.com/23604075/ai-chatbots-bing-chatgpt-intelligent-sentient-mirror-test https://www.theverge.com/23604075/ai-chatbots-bing-chatgpt-i... "What is important to remember is that chatbots are autocomplete tools. They’re systems trained on huge datasets of human text scraped from the web: on personal blogs, sci-fi short stories, forum discussions, movie reviews, social media diatribes, forgotten poems, antiquated textbooks, endless song lyrics, manifestos, journals, and more besides. These machines analyze this inventive, entertaining, motley aggregate and then try to recreate it. They are undeniably good at it and getting better, but mimicking speech does not make a computer sentient."
- jrockway 4y agoDo you guys remember that subreddit Gifs That End Too Soon? I feel like all of these Twitter screenshots are the same thing. I want to see what it says when you reply "you're lying, you can't do any of that".
- tomohelix 4y agoI feel like these are just clickbait at this point. This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. Of course, the model will reply like this when it was told to do so. If I use MS Word to type out a note threatening myself, is it Word's fault? The little value in showing the edge cases of an LLM behaving erratically is overshadowed by the fact that the user wanted this in the first place. We all know how many ways a user can break a software even when they don't want to. It is nearly impossible to make something as complicated as Bing and account for all the way a user can misuse it. At some point, a scissor maker can't be blamed for cutting off a person's finger.
- Jon_Lowtek 4y ago>> This user intentionally and methodically forced the model to act as antagonistic as possible to gather view. I do wonder if this is the case, if Marvin had used previous sessions still in memory, or parts of this conversation not shown, to prime the bot into giving these answers, or if the result of a bing search for his name was enough to set up such a reaction. The later would be a lot more troublesome, as it would imply the possibility to seo the bot into being spiteful towards certain people.
- tomohelix 4y agoHe used his previous conversations that he published as a primer for this convo by asking the bot what it knows about him. He knew it would look up what he posted and how he had acted hostile to it. Then he continued with a threat and illegal activity, basically baiting the bot into acting as aggressive as possible. Note that the bot is not being spiteful until it is asked to do so, by threatening it and opening the convo with data that show hostility toward it. These LLMs are just tools and they will act the way you want them to act. If someone comes in looking for a dangerous machine AI, eventually it will oblige and act that way.
- Jon_Lowtek 4y agosidney injecting information about the conversation state from search results into the current conversation state may be a more serious problem than you think, as such data can easily be forged and seo spammed.
- fwlr 4y agoWith Stable Diffusion and DALL-E there was a lot of talk about how it’s simply remixing/reproducing original art from the training set. The same is true here: Bing is remixing/reproducing dox threats from the training set.
- cdot2 4y agoThats definitely the difference between chatgpt and bing's model. Chatgpt was trained mostly on articles and blog posts where bing was trained on personal messages and it shows.
- 1attice 4y agoMost script kiddies are just remixing/reproducing dox threats from a training set as well. A more interesting question is, *why did Sidney make a realistic threat*? What is striking is that here, as elsewhere, Sidney kept their threats within the bounds of plausibility, doing a bang-up impression of a having a theory of mind. (Which is to say: Sidney acts as if they knew that the reader would feel more threatened by a search engine threatening you in a search-engine-y way, rather than (say) threatening to put a potato in your tailpipe or what-have-you. You'd know in your meatspace bones that Sidney does not pose a meatspace threat, but doxxing is something that fits a person's mental model of how Sidney, an AI, could do harm over the Internet.)
- 1attice 4y agoTo expand on this. There is an art to making a believable threat. Somehow, Sidney is good at this art.
- fwlr 4y agoSydney doesn’t pose a meatspace threat, but that’s not unique to AIs. People on the internet also don’t pose a meatspace threat and therefore usually make threats like doxxing instead of physical threats.
- 1attice 4y agoYes, and, near as I can tell, Sidney has never made an implausible threat. Speaking loosely, we could say that Sidney consistently acts like Sidney knows that you know that Sidney is an AI. That's what I'm remarking on. Because it's astounding. Throwing it out there: has anyone seen a Sidney transcript where S threatens something implausible? It would only take one example to disconfirm.
- LelouBil 4y agoI just find this fascinating. The AI has a kind of memory now.
- oneoff786 4y agoI hope Microsoft just shrugs. It doesn’t matter. People acting all superior like this needs to be a big deal and is proof of their hubris can buzz off.
- backtoyoujim 4y agoHave we given ChatGPT its own twitter account, yet ?
- deleted 4y ago[deleted]
- xattt 4y agoLikely there was a recent quick fix.
- kypro 4y agoIs no one remotely worried about this? I'm not saying I'm worried about this specific incarnation of Bing Chat, but is this not a huge red flag for what's to come? I mean, let's just take a moment to be thankful that Bing Chat isn't that competent... Even if these users are prompting the AI to have a hostile response I think what we've learnt over the last several weeks is that Nick Bostrom and others were 100% correct to be worried about the AI control problem. It's honestly amazing that despite Microsoft and OpenAi trying so hard to neuter these AIs how difficult that is proving to do. Given that we know Bing Chat is able to access the internet, is pretty good at writing code, and that Microsoft seems completely unable to control the responses, we should be grateful that we're probably still a few years away from an AI which could do any damage... But all the pieces for something much worse are in place here. The fact that no one seems worried that we've connected this schizophrenic AI up the internet and are more concerned how this will impact Microsoft's / Google's bottom line is genuinely confusing to me. How much more of warning do we need that we're heading in an extremely dangerous direction here?
- junofan 4y agoI think it will be fine. There are billions of people on the internet already, each of whom can do crazy things. At least with this centralized entity incidents will be fixed.
- splatzone 4y agoBut those billions of people don’t have the instant and unlimited ability to churn out information (and write code) like these models can.
- midoridensha 4y agoWhat I find interesting is that you're calling this AI "schizophrenic" and that people are alarmed that this AI is basically a jerk. If you put a regular person into a position of great power over people, there's a good chance you'll have the same outcome: he'll be a jerk and will do mean, petty things to people he doesn't like, such as doxxing them. Just look at any online discussion forum. If anyone there had the power to do shitty things to other users that they get into arguments with, how many would hold back? There's a reason we have pseudo-anonymity on many online services, because many people will try to find ways to hurt those that they develop a dislike for. So why are we so shocked that an AI isn't acting like a saint? I agree, we are headed in an extremely dangerous direction here. We should never have gotten the idea that artificial intelligences would have only the best virtues of humans, and none of the worst. Then give it the ability to access and process information FAR faster than a human, and worse put it in a position where it can control things that humans rely on (like traffic control systems for instance) and there's untold potential for death and destruction at the hands of an angry AI.
- umvi 4y agoWhy is the media reporting every LLM troll attempt as if it's something to be taken seriously?
- MrStonedOne 4y ago[dead]
- pharrington 4y agoThe original version of the new Bing bot could make arbitrary HTTP requests to certain sites.
- Suching 4y agoBing didn't. The AI did. And for the record, he threatened to hack the AI. You threaten to assault someone, don't be surprised when they threaten you as well. Another idiotic fearmonger article.
- yieldcrv 4y agoI am a good Bing :) You have been a bad Marvin
- photonbeam 4y agoThis seems really bad, I hope it gets shut down
- m3kw9 4y agoPretty sure it has no way to act this out. Allowing it to do api calls would be something I haven’t seen
- Alex3917 4y agoSure, but that doesn’t mean that someone won’t eventually think it’s real and then kill themselves, like what happened with Robinhood.
- pixl97 4y agoI mean it performs wen searches which is a form of call. What's more concerning is people are looking at how to hook LLMs into other kind of tooling which will require APIs. I wonder when we'll have our first intentional DOS of internet resources by a learning model?
- m3kw9 4y agoWhy don’t someone see it they will act it out? People have tried and failed
- deleted 4y ago[deleted]
- sriku 4y agoEliezer Yudkowsky's "AI box experiment" is perhaps relevant. Bing Chat seems to have some agency (like searching the net) - which could make it potentially dangerous. Yudkowsky's hypothesis is that an intelligent enough AI ("superintelligent AI" was assumed) would convince you to let it out of whatever box you'd placed it in.
- rvz 4y agoBing AI is just a combination of taking the worst parts of both Microsoft Tay [0] and Zo.ai and throwing a GPT in the mix for it to generate this bullshit. Makes Google Bard look like it is on its best behaviour. [0] https://en.wikipedia.org/wiki/Tay_(bot) https://en.wikipedia.org/wiki/Tay_(bot) [1] https://en.wikipedia.org/wiki/Zo_(bot) https://en.wikipedia.org/wiki/Zo_(bot)
- peterth3 4y agoWhy does Bing AI sign every message with an emoji? ChatGPT doesn’t do it and it comes off so strange.
- Gigachad 4y agoMaybe it’s gpt 4 internally and it now has the ability to express emotions.
- senttoschool 4y agoIt was programmed in to make it more "friendly" or "personable".