4 ms·
>It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning mo
by some_random 3y ago
>It works in a similar way to Nightshade: by changing the pixels of images in subtle ways that are invisible to the human eye but manipulate machine-learning models to interpret the image as something different from what it actually shows.
>Nightshade exploits a security vulnerability in generative AI models, one arising from the fact that they are trained on vast amounts of data—in this case, images that have been hoovered from the internet. Nightshade messes with those images.
So, this will not work at all as soon as the vulnerability is patched? Adversarial attacks are super cool, but they flat out do not generalize afaik. If there's been some breakthrough I'd love to hear about it, but the idea doesn't pass the sniff test to me.
- dpflan 3y agoYes, pretty sure that is how it will always be with such attacks. cat-and-mouse forever.
- lawlessone 3y agoIt does put these companies in an awkward position though. If you're implementing defenses against this, it's conceding not all your sources of data consented, or at least that you're not 100% on their provenance.
- EnergyAmy 3y agoNot really. "We're just protecting ourselves from the 4chan trolls"
- londons_explore 3y agoMore importantly, you can detect this and filter out those images pretty easily - which means you have implicit consent from the users of all other images because they chose not to protect their images technically.
- zmgsabst 3y agoThat’s not how the law works. You can’t take things from a house because they didn’t lock the front door.
- altairprime 3y agoYou can absolutely take things from a house with an unlocked front door, if there's no jail time and the fine is one [insert local unit of currency here] and that theft doesn't hurt your reputation. That's how AI works right now: "No one sued us before we made it to market with our models trained on stolen property, and now we're entrenched with a billion dollars of legal defense funds and budgeted for the fines in case we end up convicted of robbing you — and now our customers pay us to externalize and disregard the robberies we committed to reduce their effort, so they have the moral high ground and we wallow in our wealth."
- gruez 3y ago>"No one sued us before we made it to market with our models trained on stolen property, and now we're entrenched with a billion dollars of legal defense funds and budgeted for the fines in case we end up convicted of robbing you — and now our customers pay us to externalize and disregard the robberies we committed to reduce their effort, so they have the moral high ground and we wallow in our wealth." What about "all art is derivative to some extent, and training a machine artist based on existing art is no different than training a machine artist based on existing art"?
- altairprime 3y agoThat is for the courts and the lawmakers to decide, and in any case I don’t plan on convincing anyone to change their mind about ML training ethics.
- hooverd 3y agoThen the Stable Diffusion model should hold the copyright. And using generative models doesn't make you any more of an artist that commissioning a human does.
- blibble 3y agoit's also likely to be a DRM circumvention device under the DMCA
- extraduder_ire 3y agoI don't think watermarking counts as an effective technological countermeasure.
- blibble 3y agoit's designed to break/degrade use contrary to the license, not simply point back to it ala macrovision (circumvention of which was ruled to be unlawful under the DMCA)
- omneity 3y agoThis is a great point, it doesn’t have to be a perfect technical countermeasure. Being able to prove opt-out using a watermark tech like this one, maybe even encoding provenance, could make for very interesting legal battles.
- gruez 3y ago>Being able to prove opt-out using a watermark tech like this one, maybe even encoding provenance, could make for very interesting legal battles. That's... not how copyright works? If you're infringing on copyright, "the author didn't opt out" isn't a valid defense. The valid defenses like fair use can not be opt out by authors. In other words I don't really see any circumstance where opt outs are a valid factor.
- pixl97 3y agoEven that isn't how copyright works.... If I never look at your work and make a work that looks like your work I've still violated copyright. If I look at all your works and make something that looks kinda but not close enough, most likely I've not violated your work. You cannot both put your work in public and then yell at people|things not to look at it.