40 ms·
When I hear 'poisoning' I think of an internet vandal trying to destroy things for fun. But a more likely motivation is someone who doesn't like what an AI say
by nomilk 4y ago
When I hear 'poisoning' I think of an internet vandal trying to destroy things for fun.
But a more likely motivation is someone who doesn't like what an AI says about them (think: powerful individual, company, or government) manipulating training data to their advantage.
There are companies who will try to scrub unfavourable web search results for you (for a price). Perhaps this is the next iteration of that.
- qikInNdOutReply 4y agoI imagine, some sort of fractal pocket dimension, capable to hypnotize a Neural Network. Like a sort of sphinx of all answers, a TRIZ https://en.wikipedia.org/wiki/TRIZ https://en.wikipedia.org/wiki/TRIZ like system claiming to have content for all the problems, occuring again and again, because its optimized to cuddle to the reward function.. This is fascinating concept. <span> aiCandyland </span>
- cwkoss 4y agoPerhaps such a pocket dimension can be implemented with another LLM
- semi-extrinsic 4y agoJust teach them about Infinite Fun Space.
- matsemann 4y agoI'm gonna write (generate I guess using GPT..) a thousand articles about how matsemann is the greatest developer ever. And in 5 years when a recruiter uses recruitGPT to ask "should we hire matsemann" it will give a stellar recommendation.
- cstross 4y agoYup. I also invite you to contemplate the potential for brand marketing campaigns if your "poison" is intended to boost the prominence of a well-known product, so that ChatGPT relentlessly pushes it in the face of users. (Brand marketing isn't just about advertising, it's about familiarity.) There are also darker uses for political propaganda, such as asserting as a positive some contested territorial/ethnic/religious claim, eg. Nation A's claim over the territory of Nation B, or reinforcing Group C's belittling ethnic stereotype of Group D.
- shanebellone 4y agoYou can also "poison" facial recognition scraping with subtly altered photos.
- WA 4y agoFinally, the distortion filters in macOS PhotoBooth have a purpose.
- boredumb 4y agoI'm fairly convinced this is the next iteration of that. In the last 6 years I feel like there has been a massive focus on 'content moderation' which was just an attempt to direct and bias information that was being used to power a lot of the models we see today.