6 ms·
Half of the announcement talks about safety. The next step will be these control mechanisms being built into all sorts of software I suppose. It's "safe" for t
by patates 3y ago
Half of the announcement talks about safety. The next step will be these control mechanisms being built into all sorts of software I suppose.
It's "safe" for them, not for the users, at least they should make that clear.
- spir 3y agothanks, i hadn't fully realized that 'safety' means 'safe to offer' and not 'safe for users'. i won't forget it
- wiz21c 3y agoThey rather talk about "reasonable steps" to safety. Sounds like "just the minimum so we don't end up in legal trouble" to me...
- tasty_freeze 3y agoThere is some truth in what you say, just like saying you're a "free speech absolutist" sounds good at first blush. But the real world is more complicated, and the provider adds safety features because they have to operate in the real world and not just make superficial arguments about how things should work. Yes, they are protecting themselves from lawsuits, but they are also protecting other people. Preventing people asking for specific celebrities (or children) having sex is for their benefit too.
- s1k3s 3y agoI truly wonder what "unsafe" scenarios an image generator could be used for? Don't we already have software that can do pretty much anything if a professional human is using it?
- t_von_doom 3y agoI would say the barrier to entry is stopping a lot of ‘candid’ unsafe behaviour. I think you allude to it yourself in implying currently it requires a professional to achieve the same results. But giving that ability to _everyone_ will lead to a huge increase in undesirable and targeted/local behaviour. Presumably it enables any creep to generate what they want by virtue of being able to imagine it and type it, rather than learn a niche skill set or employ someone to do it (who is then also complicit in the act)
- hypocrticalCons 3y ago"undesirable local behavior" Why don't you just say you believe thought crime should be punishable?
- KittenInABox 3y ago[Edited: I'm realizing the person I'm responding to is kinda unhinged, so I'm retracting out of the convo.]
- hypocrticalCons 3y ago[flagged]
- wongarsu 3y agoI imagine they might talk about things like students making nudes of their classmates and distributing them. Or maybe not. It's hard to tell when nobody seems to want to spell out what behaviors we want to prevent.
- hypocrticalCons 3y agoStudents already share nudes every day. Where are the Americans asking about Snapchat? If I were a developer at Scnapchat I could prolly open a few Blob Storage accounts and feed a darknet account big enough to live off of. You people are so manipulatable.
- jncfhnb 3y agoStudents don’t share photorealistic renders of nude classmates getting gangbanged though
- 4bpp 3y agoWould it be illegal for a student who is good at drawing to paint a nude picture of an unknowing classmate and distribute it? If yes, why doesn't the same law apply to AI? If no, why are we only concerned about it when AI is involved?
- Sharlin 3y agoEh, a professional human could easily lockpick the majority of front doors out there. Nevertheless I don't think we're going to give up on locking our doors any time soon.
- martiuk 3y agoSimilar to why Google's latest image generator refuses to produce a correct image of a 'Realistic, historically accurate, Medieval English King'. They have guard rails and system prompts set up to force the output of the generator with the company's values, or else someone would produce Nazi propaganda or worse. It (for some reason) would be attributed to Google and their AI, rather than the user who found the magic prompt words.
- s1k3s 3y agoYeah this is probably the most realistic reason
- fragmede 3y agoFor some scenarios, it's not the image itself but the associations that the model might possibly make from being fed a diet of 4chan and Stormfront's unofficial YouTube channel. The worry is over horrible racist shit, like if you ask it for a picture of a black person, and it outputs a picture of a gorilla. Or if you ask it for a picture of a bad driver, and it only manages to output pictures of Asian women. I'm sure you can think up other horrible stereotypes that would result in a PR disaster.
- PeterisP 3y agoThe major risky use cases for image generators are (a) sexual imagery of kids and (b) public personalities in various contexts usable for propaganda.
- Spivak 3y agoIt's also "safety" in the sense that you can deploy it as part of your own application without human review and not have to worry that it's gonna generate anything that will get you in hot water.
- matthewmacleod 3y agoI really wish that every discussion about a new model didn’t rapidly become a boring and shallow discussion about AI safety.
- cypress66 3y agoThis announcement only mentions safety. What else do you expect to talk about?
- hypocrticalCons 3y ago[flagged]
- jprete 3y agoAI is not an engineered system; it's emergent behavior from a system we can vaguely direct but do not fundamentally understand. So it's natural that the boundaries of system behavior would be a topic of conversation pretty much all the time. EDIT: Boring and shallow are, unfortunately, the Internet's fault. Don't know what to do about those.
- PeterisP 3y agoAt least in some latest controversies (e.g. Gemini generation of people) all of the criticized behavior was not emergent from ML training, but explicitly intentionally engineered manually.
- jprete 3y agoBut that's the thing, prompt formulation is not engineering in the sense I'm talking about. We know why a plane flies, we know why an engine turns, we know how a CPU works - mostly. We don't know how GenAI gets from the prompt to the result with any specificity at all. Almost all the informational entropy of the output is hidden from us.
- deleted 3y ago[deleted]
- hedora 3y agoPSA: There are now calls to embed phone-home / remote kill switch mechanisms into hardware because “AI safety”.
- newzisforsukas 3y agoexamples? seems like it would be easier to instead communicate with ISPs.
- astrange 3y agoHmm, all computers have remote kill switches in them unless you have a generator at home.
- root_axis 3y agoThis is the world we live in. CYA is necessary. Politicians, media organizations, activists and the parochial masses will not brook a laissez faire attitude towards the generation of graphic violence and illegal porn.
- Sharlin 3y agoNot even legal porn, unfortunately. Or even the display of a single female nipple…
- realusername 3y agolooking at the manual censorship of the big channels on youtube, you don't even need to display anything, just suggesting it is enough to get a strike. (of course unless you are into yoga, then everything is permitted)
- Sohcahtoa82 3y ago> (of course unless you are into yoga, then everything is permitted) ...or children's gymnastics.
- hypocrticalCons 3y ago> This is the world we live in. Great talk about slavery and religious-persecution, Jim! Wait, what were we talking about? Fucking American fascists trying to control our thoughts and actions, right right.
- hypocrticalCons 3y agoBTW Nvidia and AMD are baking safety mechanisms into the fucking video drivers No where is safe
- jprete 3y agoDo you have a reference on this?
- hypocrticalCons 3y ago[flagged]
- QuadmasterXLII 3y agoThat would be an impressive technical feat haha.
- GuB-42 3y agoIf AMD and Nvidia managed to understand what you are asking for just looking at a bunch of vectors and manipulate the outputs to a goal, that would be a serious breakthrough in the field. Even if it did that by looking at the process memory (which would be seriously wrong in many ways), just the manipulation part would be mighty impressive. And if it was the case I guess we would see many papers and heated discussions about them. Or it is just the usual conspiracy theory.
- jncfhnb 3y agoThey custom tune the gpus to specific programs. That’s a long shot from general embedding interpretation. Sounds like you are perhaps just bad at using the generative art tools
- Kerb_ 3y agoBro I have generated literally thousands of monster girls in stable diffusion and all of my outputs have been spectacular. Driver changes have not affected my generation at all
- wongarsu 3y agoWhat's equally interesting is that while they spend a lot of words on safety, they don't actually say anything. The only hint what they even mean by safety is that they took "reasonable steps" to "prevent misuse by bad actors". But it's hard to be more vague than that. I still have no idea what they did and why they did it, or what the threat model is. Maybe that will be part of future papers or the teased technical report. But I find it strange to put so much emphasis on safety and then leave it all up to the reader's imagination.
- fortran77 3y agoRemember when AI safety meant the computers weren’t going to kill us?
- SV_BubbleTime 3y agoNow people spend a lot of time making them worse to ensure we don’t see boobs.
- BryanLegend 3y agoFrom George Hotz on Twitter (https://twitter.com/realGeorgeHotz/status/1760603918839542117 https://twitter.com/realGeorgeHotz/status/176060391883954211...) "It's not the models they want to align, it's you."
- aredox 3y ago[flagged]
- rvnx 3y agoWhat he thinks about non-AI topics in his free time is his choice; his political choices don't invalidate his other thoughts. Like with Stallman, if you listen about his political thoughts vs his software engineering thoughts.
- Cthulhu_ 3y agoWhile that thoughtpiece makes me not like him as a person, your point is correct; the comment you're replying to is a low effort ad hominem, attacking the person and something they said about a different subject instead of addressing the actual remark about models.
- aredox 3y agoCome on, his point is not about AI but about politics: "they" versus "you". And he is completely, totally incompetent on this - and by the way, he's also completely incompetent on AI, and on most of tech. See his stint at twitter...
- jtr1 3y agoWhat specific cases are being prevented by safety controls that you think should be allowed?
- Tomte 3y agoNot specifically SD, but DallE: I wanted to get an image of a pure white British shorthair cat on the arm of a brunette middle-aged woman by the balcony door, both looking outside. It wasn‘t important, just something I saw in the moment and wanted to see what DallE makes of it. Generation denied. No explanation given, I can only imagine that it triggered some detector of sexual request? (It wasn‘t the phrase "pure white", as far as I can tell, because I have lots of generated pics of my cat in other contexts)
- dmezzetti 3y agoAny large publicly available model has no choice but to do this. Otherwise, they're petrified of a PR nightmare. Models with a large user base will have an inverse relationship with usability. That's why it's important to have options to train your own with open source.
- beefield 3y agoI get a slightly uncomfortable feeling with this talk about AI safety. Not in the sense that there is anything wrong with that (may be or may be not), but in the sense I don't understand what people are talking about when they talk about safety in this context. Could someone explain like I have Asperger (ELIA?) whats this about? What are the "bad actors" possibly going to do? Generate (child) porn/ images with violence etc. and sell them? Pollute the training data so that the racist images pops up when someone wants to get an image of a white pussycat? Or produce images that contain vulnerabilities so that when you open that in your browser you get compromised? Or what?
- Tadpole9181 3y ago> Could someone explain like I have Asperger (ELIA?) Excuse me?
- beefield 3y agoYou sound offended. My apologies. I had no intention whatsoever to offend anyone. Even if I am not diagnosed, I think I am at least borderline somewhere in the spectrum, and thought that would be a good way to ask people explain without assuming I can read between the lines.
- Tadpole9181 3y agoLet's just stick with the widely understood "Explain Like I'm 5" (ELI5). Nobody knows you personally, so this comes off quite poorly.
- beefield 3y agoI think ELI5 means that you simplify a complex issue so that even a small kid understands it. In this case there is no need to simplify anything, just explain what a term actually means without assuming reader understanding nuances of terms used. And I still do not quite get how ELIA can be considered hostile, but given the feedback, maybe I avoid it in the future.
- mempko 3y agoI think this AI safety thing is great. These models will be used by people to make boring art. The exciting art will be left for people to make. This idea of AI doing the boring stuff is good. Nothing prevents you from making exciting, dangerous, or 'unsafe' art on your own. My feeling is that most people who are upset about AI safety really just mean they want it to generate porn. And because it doesn't, they are upset. But they hide it under the umbrella of user freedom. You want to create porn in your bedroom? Then go ahead and make some yourself. Nothing stopping you, the person, from doing that.
- TulliusCicero 3y agoI agree with you, but when companies don't implement these things, they get absolutely trashed in the press & social media, which I'm sure affects their business. What would you have them do? Commit corporate suicide?
- TylerLives 3y agoThis is a good question. I think it would be best for them to give some sort of signal, which would mean "We're doing this because we have to. We are willing to change if you offer us an alternative." If enough companies/people did this, at some point change would become possible.
- acomjean 3y agoI think this isn’t software as much as a service. When viewed through this lens the guard rails make more sense.