15 ms·
Facebook Discovers Fake AI-Generated Profiles
- neonate 7y agohttp://archive.md/abR53 http://archive.md/abR53
- joeblubaugh 7y ago“Discovers” People have been talking about this group for months and Facebook hemmed and hawed and delayed and delayed
- jethro_tell 7y agolol, I came here to put that word in quotes.
- Barrin92 7y agoit's actually hard to overstate how significant the effort of the epoch times is. they gathered billions of views on facebook, more than any other news organisation. "In April, at the height of its ad spending, videos from the Epoch Media Group, which includes The Epoch Times and digital video outlet New Tang Dynasty, or NTD, combined for around 3 billion views on Facebook, YouTube and Twitter, ranking 11th among all video creators across platforms and outranking every other traditional news publisher, according to data from the social media analytics company Tubular." https://www.nbcnews.com/tech/tech-news/trump-qanon-impending-judgment-day-behind-facebook-fueled-rise-epoch-n1044121 https://www.nbcnews.com/tech/tech-news/trump-qanon-impending...
- mtnGoat 7y agobut were those views REAL?
- scarejunba 7y agoPeople have also been talking about every other group. The sensitivity of "people talking" may be 100% but the specificity is near zero.
- jedberg 7y agoIt's ironic that as we move into a world where anyone can be a "journalist" and anyone can have a voice, we need professional journalists more than ever. Now a journalist will not only need training in writing well, conveying a story with facts, and investigating, but they will also need to become experts in identifying fake source material. In a world where anyone can make a video or photo of anyone doing anything, we'll need to rely on reputable journalists to vet that material for us. Saw a video on YouTube of the President saying something bad? No idea if it's real anymore. Sure, a lot of this technology has existed for a while, but it was usually in the fringes (see The National Enquirer). Now it's going mainstream. And the worst part is, there are a whole group of people who will not trust a professional journalist (sometimes with good reason) but will trust anyone online who happens to provide evidence that boosters their existing opinions.
- dv_dt 7y agoIn part, imho, the distrust comes from the the standards of the highest tiers of professional journalists being on a low right now too.
- nsomaru 7y agoJournalists are following clicks instead of the other way around
- phkahler 7y agoEvery day the NYT sends me a summary email. Every day there is a status on impeachment. They dont even call him president, its Mr Trump. The bias is obvious. They would like me to pay for more!
- mtnGoat 7y agoI agree, I will happily pay for news, but most of these "news" outlets are just a bunch of opinions and usually they are all one way or the other depending on the editor.
- rvz 7y agoAnd so it appears that the inevitable has 'finally happened'. Mark Zuckerberg's website is now a place not to be trusted as a source of news, finding friends and meeting others if there's a possibility of AI generated profiles being proliferated on the website. I've heard that meeting real humans outdoors makes it less likely to be deceived. Their authenticity is vouched via word of mouth and experience by other humans, rather than robots on the internet.
- MuffinFlavored 7y agoRequire manual profile validation for new Facebook accounts? Probably would get downvoted if I said SSN + a credit check. Phone number required + needs to be validated + can not be reused + can not be Twilio. Prove your address? Prove your ISP cable bill? I don't know. Everything I guess, people here are going to tell me it's an invasion of privacy and they don't trust Facebook to do it. But they'll let CreditKarma have every bit of financial information related to them possible. People will apply for a credit card online (employer, address, social, income)... make a universal login process for that. Maybe like blue Twitter verified checkmarks next to all profile names. "Verified human"
- jethro_tell 7y agoThe problem is that a lot of people can't even fathom the idea of automating something like account generation. It takes them two support calls and an 1 1/2 hour to log into their email. They don't even understand what's possible, and a blue checkmark won't/doesn't solve that.
- bart_spoon 7y agoSeems like an incredible amount of work for something that at the end of the day doesn't actually provide much value to users or the world.
- Barrin92 7y agoor simply charge a few bucks for account creation.
- deleted 7y ago[deleted]
- chrstphrhrt 7y ago"Discovers", yeah sure.
- username4567 7y agoI look forward to this block of fake accounts registering to vote and voting by mail. Will any accounts go rogue and defy their creators? How does a candidate appeal to this influential group?
- tschellenbach 7y agoFor many use cases it would be great to have a real/verified oauth service. Dutch government does something like that with digid. Imagine how much fraud could be prevented if there was something like that in the US.
- jyu 7y agoThe cynic in me thinks Facebook let this slide to hide the anemic "growth" of their userbase and activity. Bot accounts will still click on ads, like pages, and fake interactions that generate revenue for Facebook.
- CM30 7y agoAka the usual problem these social media services have; bad behaviour from bots and users likely looks great on the stats spreadsheet, even if it's at the expense of the quality of the site/community.
- scarejunba 7y agoThe most fascinating thing to watch is the democratization of propaganda. Now you no longer need the printing press. You can speak with a million voices without being a Getty or Rockefeller. The NYT no longer has a monopoly on convincing lies. These are interesting times. I think we're close to the emergence of trustworthy celebrity journalists.
- noxer 7y agoI just leave this here https://www.thispersondoesnotexist.com/ https://www.thispersondoesnotexist.com/ Probably the source of these images.
- dwd 7y agoThis other article linked here, discusses it: https://news.ycombinator.com/item?id=21847583 https://news.ycombinator.com/item?id=21847583 Interesting that the eyes on every TPDNE image have a specific position that could (for now) be used to detect them.
- mmagic 7y agoIf anyone's interested in neural networks and how they work, you should check out victoR. VictoR is a powerful tool which utilizes file storage size to modify specific neuron mapping components and optimize processing power. It's still quite experimental, but here is a download link for an installation wizard if you want to check it out: https://bit.ly/35mtZZj https://bit.ly/35mtZZj
- avivo 7y agoFYI, I actually just published a guide on how you can help ensure that ("deepfake") tools that you make are less likely to end up misused like this: https://www.technologyreview.com/s/614912/ethical-deepfake-t https://www.technologyreview.com/s/614912/ethical-deepfake-t ... (Also at /newest; 3rd page now, and the other submission. Ideally those topics at least would be merged?) Summary of mitigations: # Limiting users or use cases - User Vetting: Carefully vet those who can use a tool (Example: Synthesia only working with vetted enterprise clients.) - Usage Constraints: Limit the synthesis possibilities. (Example: Humen is providing only a limited set of movements for generated videos.) # Discouraging malicious use of flexible and public tools - Clear disclosure: Ensuring and requiring that manipulated outputs are clearly labeled (including via metadata). - Consent protection: Requiring the consent of those being impersonated. - Detection friendliness: Supporting those working on detection and not seeking to fool detection systems. - Hidden watermarks: Embedding data through robust watermarks. - Usage logs: Storing information about usage and media outputs. - Use restrictions: Enforcing contracts that prohibit forms of violative or malicious use. Not all apply to every system of course. The article has more detail!
- eivarv 7y agoGood article, but for the sake of argument: Don't you think any technical "solutions" to this issue will also eliminate the possibility to use these tools for ethical goals (let's say the roles were reversed or whatever) – in addition to all the other gray-area, and inevitable arms-race of detection/avoidance?
- avivo 7y agoDefinitely. I mean I think e.g. you could have incredible art projects and experiences created out e.g. of giant undetectable fake crowds. There are real tradeoffs in terms of what we value or need in a society.
- Lammy 7y agoI think technical limitations like this are the worst possible solution. The genie can't be put back in the bottle with any particular technology, and the lack of widespread access and awareness means the damage from any particular deepfake that _does_ get released can be that much higher. There is zero chance this tech won't be in the hands of every government-sponsored group on the planet, not to mention all the other independent bad guys out there online.
- mgiannopoulos 7y agoReported by Snopes for months now, not “discovered” by Facebook https://www.snopes.com/news/2019/10/11/pro-trump-outlet-linked-epoch-times/ https://www.snopes.com/news/2019/10/11/pro-trump-outlet-link...
- tempodox 7y agoWould anybody expect the truth from FB? No way they wouldn't spin this to their best possible advantage.
- mgiannopoulos 7y agoI would expect the NYT to do some proper reporting though instead of copy/pasting Facebook’s press releases.
- joshspankit 7y agoWhat leads you to expect that?
- wyxuan 7y agoThis isn’t any the bl but rather accounts that helped disseminate the bl’s stuff, which snopes doesn’t discuss
- cloudking 7y agoWhy doesn't Facebook have an identity verification process? They literally allow anyone to sign up as any person. There is a show on MTV called Catfish that is dedicated to people who create fake profiles to trick people into relationships.
- throw55362 7y agoIt’s interesting that “The Epoch Media Group” is funded by Falun Gong.
- qznc 7y agoRecently I played around with the newest iPhone filters. The pictures had weird shadows and distorted backgrounds. Are distorted backgrounds really an indicator for fake profiles? Smartphones do all kind of weird stuff to "real" images.
- kortex 7y agoGANs tend to leave a very distinctive kind of artifact which is hard to describe but I think anyone who goes through a lot of GAN imagery knows what I am talking about. It's a particular kind of distortion which is a liquidy psychedelic weirdness, that you don't really see in other media fakery. If you are interested, media forensics is an entire field about this sort of thing.
- akerro 7y agoSeriously is it what nytimes is now? https://gfycat.com/maturefarflunghagfish https://gfycat.com/maturefarflunghagfish
- nullbull 7y agoWhy take them down? That's someone's free speech right? This is what we're told about false information and political ads with outright lies in them, aren't we? Why would this be any different? Rather predictable that the "line" for taking something down suddenly appears when you're lying to Facebook. Lie to other users and it's fine... as long as you are a real commodity, ahem, I mean "person." Cease to lie in a way that allows Facebook to make money (not much point serving ads to bots, is there?), and all of the sudden you're taken down. Funny that.
- sbarre 7y agoFacebook is a private enterprise. Free speech doesn't apply. Compliance with government/social pressure is what applies (for better or worse). That's the thing that so many people don't get.. Facebook is not a public space. You have no inherent rights there, and they can do whatever they want with your acccount/content/etc (or at least whatever their TOS says)..
- colejohnson66 7y agoNo. The First Amendment doesn’t apply. The concept of free speech can be applied anywhere.
- sbarre 7y agoThat's extremely pedantic and doesn't really add to the discussion. My point was that these private companies police their content based on the impact it has on their bottom line and any laws they have to respect, and that's it. They don't do it to protect any kind of speech, groups or individuals, and no one has any rights (related to speech - yes AODA and other such legal rights apply), in the civic sense, when using Facebook or other social media platforms.
- elorant 7y agoBecause it breaks ad targeting, or to be more precise makes ads obsolete because you're promoting products to someone who isn't who they say they are. Advertising is the essence of Facebook's existence. And they'll go a long way to ensure it works as a clock.
- tomaszs 7y agoKeep calm. It is false positive. It was only Facebook customer service bots
- yellow_lead 7y agoYou better believe black hat SEOs are using AI for spam like this.
- hooande 7y agoThis is just using GANs to generate fake profiles. It does scale better than making up fake people by hand, but to what end? I'm not sure if there's a direct relationship between the ease of creating fake accounts and their cultural influence. And it seems like facebook doesn't have trouble detecting them. Convince me that this isn't just "add AI to the headline" clickbait
- deleted 7y ago[deleted]
- narrator 7y agoI have said for a while now that the future is their AI vs our AI and nobody can turn the AI off because then we'd lose.
- GarrisonPrime 7y agoIronic, since Facebook itself uses a sort of AI to create fake shadow profiles.
- EsssM7QVMehFPAs 7y agoSource?
- willart4food 7y ago"Discovers"
- snowedin 7y agoThere's been fake generated profiles on Facebook for over a decade. I know this because I had some, I was targeted by some, and I had "security friends" who generated some. Over the years I've seen several campaigns of groups of fake profiles trying to collect access (by "friending into" friend circles). At university my security club had a member who gave a presentation on algorithms for detecting and generating fake profile posts for Facebook, through which he created sock puppets and also found a ton of others belonging to "who knows which group". That was around 10 years ago.
- ddmma 7y agoPeople are faking their profile pictures for sometime using filters or AI cameras, bamby eyes. Already a digital ego fairytale now deepfakers would open new positions as ’ghost busters’
- ansmithz42 7y agoA fascinating thought on this for me is the basic American belief that "bigger is better". What I mean here is that the companies generating these fake ids will generate lots of the fake ids, under the belief that the more they have the more influence they have. At some point, the fake ids will reach a tipping point and outnumber the real ids thus making Facebook itself irrelevant. Sort of a "DOS" using fake ids, in this case a denial of relevance. There are some interesting paths that this could take, but I don't see any incentive in Facebook really doing much about this because their power actually lies in also "Bigger is better". So, generation of fake ids could be a blessing by overloading Facebook and making it irrelevant.
- SubiculumCode 7y agoIt makes me reconsider the early RealID efforts in social media. Perhaps perhaps.
- aaron695 7y ago> Facebook Discovers Fake AI-Generated Profiles I think the profiles are human generated. I see no evidence otherwise. If they have a GAN why did they repeat the photo? I'd think they got the photo from some open GAN project. [edit] Researchers actual say they might be from thispersondoesnotexist.com [3.14] But I guess that spoils a good HN story. [3.14] https://graphika.com/uploads/Graphika%20Report%20-%20OperationFFS_Fake_Face_Storm.pdf https://graphika.com/uploads/Graphika%20Report%20-%20Operati...
- tempodox 7y agoThis gives me flights of fancy. Imagine (something like) FB (re-)built as a fake AI-generated company (with fake AI-generated users). It would feel like a circle being closed, something reaching its ultimate consequence.
- fit2rule 7y agoCool. Now it needs to work on algorithms which detect whether a journalist is working for the CIA. That'd be progress.
- plerpin 7y agoWhat's the purpose? Making an army of bots for Shen Yun astroturfing?