10 ms·

I think pay-for-verification is going to be dominant on all real-identity social media. Reason is simple: AI 1.You can now trivially create an avatar photo, o

I think pay-for-verification is going to be dominant on all real-identity social media. Reason is simple: AI 1.You can now trivially create an avatar photo, of any level of attractiveness, of any race, of any age. You can even reproduce the same character reliably in different environments/costumes via stable diffusion + LORA. 2. You can now easily create a comment history on that account, thanks to ChatGPT. 3. You can even produce voices reliably with just a few samples. There's no real defense against AI impersonation at scale, except for charging-for-verification. The money drastically increases the cost for impersonators and scammers/catfishers, and provides resources on the other end to moderate impersonation.

notahacker 4y ago
I think the pay-for-verification might solve those problems ship sailed when Twitter decided that a symbol which had previously meant they'd attempted verification could be given to pretty much anyone who paid them $8 a month... Impersonation-at-scale depends on scale, not verification of individual accounts as authentic people with authentic notability and opinions, and impersonation not at scale sometimes feels more motivated to pay the $8 than the person or position being impersonated.
- anonuser123456 4y ago
  Once/if consumers value identity this might change. If everyone on twitter were paying for verification, I think twitter would be much more interested in defending the sanctity of said system. As it stands, they are currently just trying to get the feature out there and get people to pay. Getting it deployed trumps making it perfect.
  - notahacker 4y ago
    They already had a verification feature deployed which wasn't perfect but was reasonably strict. Then they debased it for a new revenue stream. I don't see them making the now that we've got a bunch of people paying for it, lets reduce our revenue by suddenly becoming strict on it again decision. Or people taking the badge more seriously now it just means someone subscribed to Twitter Blue
- halJordan 4y ago
  I get the "i hate Twitter/elon" attitude, but the truth is that the CA system has been dependent on pay-to-verify for some time now and despite whatever grievances you have against verisign or whoever it's brought us to today safely enough.
  - commandersaki 4y ago
    The pay-to-verify is the reason why a significant chunk of the Internet didn't have HTTPS. It was the inception of free CAs like Lets Encrypt that made 90+% HTTPS penetration.
    - xeromal 4y ago
      I'd argue laziness is the reason most of the internet didn't have HTTPS. Browsers putting a scary checkmark next to the site is what made it happen.
  - joe_the_user 4y ago
    And the problem with pay for identity is you can buy other people's identities. The news was full impersonations right after Twitter initiated it's pay for blue check marks. That we don't here about it now may just be a matter that the news stopped caring. URL squatting was a problem for a long time - the decision that trademark holder could seize domains helped but a web address today isn't considered a solid identity at this point so "the system working" is a bit of an exaggeration. Also, I'd note the gp didn't bad-mouth Elon or Verisign so your comment is a kind of riding a trolly false accusation.
  - giaour 4y ago
    I think the problem was not that Twitter started charging $8/month ago much as that they gave up the expensive-to-perform ID verification workflow that had been in place. The CA system, on the other hand, has more or less abandoned the expensive, manual "extended verification" process in favor of the zero marginal cost "domain verification" process. This switchover drove the cost of a new cert to $0 for almost all users. So I'm really not following your analogy, since Twitter gave up the workflow and started charging (aka, a cash grab), whereas CAs gave up charging and adopted a more efficient workflow (aka the expected behavior of efficient markets).
- pessimizer 4y ago
  > a symbol which had previously meant they'd attempted verification That's not what it meant. It was an award for being notable. $8 is a fee that you pay to get verified.
  - notahacker 4y ago
    It was an award for being an authentic notable person or entity, on the basis that the notable people and entities were most likely to be parodied or faked. Now it's an award for people that pay up
    - 300bps 4y ago
      It was an award for being an authentic notable person or entity Using an arbitrary and biased determinant of who is notable. https://www.wbur.org/cognoscenti/2021/06/28/twitter-verification-blue-check-yashar-ali-kaivan-shroff https://www.wbur.org/cognoscenti/2021/06/28/twitter-verifica...
      - deleted 4y ago
        [deleted]
      - palata 4y ago
        Maybe, but at least the verification seemed meaningful. And it feels like a meaningful verification doesn't generalise well: it's easy to pick a few people (arbitrarily) and truly verify them. But verifying millions of people... that's hard, unless you don't truly verify (in which case it's meaningless).
    - bombcar 4y ago
      It started out as “this is who you expect it is” for things like Obama or Trump, then it became “this is a person who speaks with authority of some sort, listen to them” and now it is just “someone pays eightbux”
      - pessimizer 4y ago
- m3kw9 4y ago
  It helps because the cost of impersonating wont make sense for a lot of people
  - bombcar 4y ago
    This is a huge part of it. Shitposting for $0 is a lot different than shitposting for $8. Some will bust out the card to do so, but not many.
    - kube-system 4y ago
      The problem is that those who are willing to break out the $8 are probably the same ones who have the most to gain from it.
    - 8note 4y ago
      More importantly, $8 includes an identity check
- benced 4y ago
  Elon turning a $8 subscription into a culture war artifact doesn’t invalidate the idea of a subscribing for legitimacy altogether.
  - notahacker 4y ago
    Agreed it doesn't completely invalidate it as a concept, but Elon turning the highest profile somewhat reliable implementation of an authenticity badge into a culture war artifact doesn't exactly bode well for it being a social media must-pay-for. The fact Facebook once aspired to be the platform where everyone used their real name and now can't be bothered to deactivate friendspamming sexbots without most users caring suggests that ordinary people won't exactly be queueing up to pay them $144 per annum because of their inherent trustworthiness as a verifier either. It'd probably work a bit on LinkedIn because of the nature of the user base and lots of people already expensing Premium accounts, but funnily enough I'm not sure LinkedIn actually has that much of a fake account problem...
    - ericmay 4y ago
      > but funnily enough I'm not sure LinkedIn actually has that much of a fake account problem... Oh it does for sure, unfortunately many people are more than happy to connect with anyone who wants to connect with them because “LinkedIn doesn’t have a fake account problem”. And many people post resumes, entire employment history, all professional social connections, etc. and so all you need to do is pretend to be a student of your targets alma mater and now you have all of that personal info with the click of a Connect button. I’ve noticed trends where someone would try to connect with me and they’d be connected with some acquaintances of mine, but not once have I ever heard of that person. Those types of accounts exist, are fake, and steal information. If you haven’t 1-1 met someone before in person or via actual business relationship you should definitely not connect with them on LinkedIn.
      - notahacker 4y ago
        Most of those accounts are actual people registered with their actual name using spamming tools (some of them LinkedIn approved) to connect with as many people as possible though, usually to boost the appearance of their personal profile and have as wide a possible audience for promotional messages they share rather than to circumvent LinkedIn's information paywalls. Spamming people loosely connected with your industry with what you're doing at work is, after all, LinkedIn's intended purpose...
iLoveOncall 4y ago
Sending a photoshopped ID to Facebook to get the verified badge on your fake account is orders of magnitude easier than everything you mention here. Those verifications are 100% useless for non public figures.
- palata 4y ago
  Agreed. Either they get it right (and then congrats, the Facebook account is now a passport), or they don't, and it's completely meaningless because it still means "well don't trust it though".
- zeroonetwothree 4y ago
  Yeah but you also have to pay a bunch of money. That's going to discourage scammers at least some amount.
  - iLoveOncall 4y ago
    It won't. Actually the fact that it's paid makes it even more useless, because the vast majority of users will NOT be verified. I would never, ever, pay for this verification for example. Therefore a non-verified user will not look less trustable than a verified one.
stefan_ 4y ago
Who cares? Is the AI gonna RSVP to my event? Haggle over my 2nd hand item? Comment on whatever the local council has planned? All these schemes tend to fall apart at the first contact with reality. (Also, you are talking about "Facebook verifying users". They can't even verify who is paying them for political ads, and they certainly don't seem to care very much.)
- michaelt 4y ago
  Well, there clearly are bots on social media, doing things like reposting popular posts to farm reputation points. The idea these bots are doing that for a reason doesn't seem so crazy to me.
- urbandw311er 4y ago
  I think it’s still a relevant problem we will face as a society with regards to bot farms etc running social interference (eg election propaganda)
- oneoff786 4y ago
  > Comment on whatever the local council has planned? That seems plausible? Already achievable with some interns mind you. But still
  - blitzar 4y ago
    Having spent all of 5 minutes on "nextdoor" - it isnt hard to whip some locals into complete hysteria.
- IshKebab 4y ago
  They will definitely haggle over your second hand item. Ever tried selling an Apple product on Facebook marketplace? You get a ton of scammers and you can't tell with some of them (until they try the actual scam, by which point you've wasted time messaging them, packaging the item, etc.)
linuxftw 4y ago
Or, you know, we can just go back to circa 2000 and tell people not to trust everything they see on the internet as fact?
- woeirua 4y ago
  Propaganda works. Even when people know it’s probably fake.
  - kube-system 4y ago
    You can straight up tell people something is fake, and they will still entertain the thought enough to have an impact on their perspective. Consider popular novels which are fictional social commentaries. Or culturally influential Hollywood movies.
    - LadyCailin 4y ago
      God yes. I absolutely hate it when people cite 1984 as “what happens when you have large government.” As if no one could write a dystopian fiction novel based on whatever world view that person has.
- JasserInicide 4y ago
  Tell that to employers that now just Google you and look at the top search results
verisimi 4y ago
You're going to be tracked and traced everywhere, and you're going to pay for the privilege. Amazing what they can get us to do! This is the reason musk bought twitter too, of course.
- palata 4y ago
  I thought he bought it to prove what a moron and a jerk he is. Maybe that's just a side effect :)
shapefrog 4y ago
> I think pay-for-verification is going to be dominant on all real-identity social media. Bad actors are going to be happy to pay $10/20/30/40 a month to scam people, its their job and livelihood.
- ericd 4y ago
  It raises the cost of making a fake account by orders of magnitude, it's useful against bad actor bot accounts at scale.
  - palata 4y ago
    Shouldn't there be a difference between "verified" and "paid"? If bad actors can just pay to have a "verified" account, then it's completely meaningless, isn't it? Either they truly verify (which is probably hard) and I can pay to be verified, or they don't, and it's completely useless because nobody should trust the verification...
    - ericd 4y ago
      It’s not that black and white, most of these things are a mix. Some verification up front, and the normal ongoing monitoring of behavior. Putting it all up front usually presents too large a barrier to users, and causes conversion to suffer (and it’s harder to be sure than just catching bad behavior once they start using the service). Taking money makes getting banned a much larger penalty for bad actors. And that makes a lot of bad behavior unprofitable, and that subset will stop, and reduce load on their spam fighting efforts.
      - palata 4y ago
        Right, that's interesting! Thanks for the insights :) > and the normal ongoing monitoring of behavior Now I get another question: say you have millions of paid, verified users (that's the goal, right?). If you can monitor millions of account successfully, then it's most likely automated, isn't it? And then it should scale to all the users, maybe? In which case again, "verified" feels meaningless (because they just improved the bot situation overall). > And that makes a lot of bad behavior unprofitable I wonder if the bad behavior that can be reasonably well detected is the unprofitable one. Say I can leverage the "verified" badge to scam people more efficiently -> probably I can pay 8$ for an account that will scam a few people, right? So really paying 8$ would be a deterrent for bots that purposely do disinformation, but there it's hard to know if it's a bot, isn't it? I mean many people do disinformation just out of a lack of knowledge, and that's not a reason to qualify them as bots, right?
bogomipz 4y ago
>"There's no real defense against AI impersonation at scale, except for charging-for-verification." What's to stop a generative AI from generating a state issued drivers license or similar to go along with that avatar photo though? Also how different is that really though from all the existing non-AI generated fake accounts?
- oneoff786 4y ago
  Well that would be a felony to start. Which is more noteworthy than just against terms of service.
  - nebulousthree 4y ago
    If McDonald's required showing your driver's license to pick up your order, would it be a felony to show a fake license and pick up an order? You're not lying to a government organisation. You're just obscuring your identity to a private corporation. I fail to see how that can be any more than unethical.
    - ericbarrett 4y ago
      Showing a false driver's license in the U.S. is usually a crime (though not often a felony by itself) regardless of circumstance; e.g. Texas law: https://texas.public.law/statutes/tex._transp._code_section_521.451 https://texas.public.law/statutes/tex._transp._code_section_... Note that a "Class A misdeamenor" in Texas is punishable by a $4,000 fine and/or 1 year in jail. Is that going to happen because you used it to pick up a food order? Probably not, but stranger things have happened...
    - oneoff786 4y ago
      No it would not. And it would not be a felony to show one to Facebook either. But the question was about generating new IDs with AI. That is a felony.
  - bogomipz 4y ago
    Can you explain how an AI-generated fake ID for a fake social media account constitutes a felony? On what grounds exactly? Facebook despite attempting to normalize the handing over of government-issued identification to an advertising platform, is not a government entity.
schainks 4y ago
> provides resources on the other end to moderate impersonation. Eeeehhhhhh, I don't think so.
nip 4y ago
> There's no real defense against AI impersonation at scale, except for charging-for-verification. There is: have the state take care of that as you are a citizen that pay taxes (you already are paying for this "verification") – an ID card that contains a proof of you being... you. https://openid.ee/en/ https://openid.ee/en/
- ikesau 4y ago
  Similarly, New Zealand's government-provided SAML 2.0 authentication service called RealMe is really good. https://developers.realme.govt.nz https://developers.realme.govt.nz But the U.S. has a phobia of any form of national ID card, so I doubt that the USDS will be able to build anything soon. It's such a shame. At least having it as an option could allow society's to start experimenting with it as a solution to these problems.
  - schmorptron 4y ago
    I'd like a govt service where you can verify that you're a real person but not your identity or anything of the sort. Sort of like having the third party registering with the government and then you get a token per account you make so the service can verify there isn't more than one accounts per person but they can't identify you across the services. Would also be immensely helpful useful for stopping cheating in online video games, while not requiring you to give our your real name and identity to anything
wkat4242 4y ago
Impersonation or creating fake personas has always been possible. In the security community these are called "sockpuppets". It's just less work now.
tangjurine 4y ago
How about meeting people in person and then asking for their social media username? Most people use social media to keep track of people they already know.
- vineyardmike 4y ago
  > Most people use social media to keep track of people they already know. And most people who drive for Uber are just doing so to make a few bucks in their daily life. And most people on AirBnb are renting their spare room or couch. /s Most people follow a handful of a famous people or companies, and some friends. Especially on twitter, which was very "one to many" following relationships. Thats why verification was so impactful, it gave the "one" in "one to many" something special, and a way to establish authenticity.
jasmer 4y ago
"There's no real defense against AI impersonation at scale," Defamation liability. I'm not sure the laws have worked out yet, but I suggest that these companies may be in for some lawsuits if thing go awry, and they have not shown to be actively working against it. I'm looking forward to that one.
elcomet 4y ago
It's not about verification, it's about customer support. Verification could be provided for free with ID cards or something like this.

notahacker 4y ago

I think the pay-for-verification might solve those problems ship sailed when Twitter decided that a symbol which had previously meant they'd attempted verification could be given to pretty much anyone who paid them $8 a month... Impersonation-at-scale depends on scale, not verification of individual accounts as authentic people with authentic notability and opinions, and impersonation not at scale sometimes feels more motivated to pay the $8 than the person or position being impersonated.

anonuser123456 4y ago
Once/if consumers value identity this might change. If everyone on twitter were paying for verification, I think twitter would be much more interested in defending the sanctity of said system. As it stands, they are currently just trying to get the feature out there and get people to pay. Getting it deployed trumps making it perfect.
- notahacker 4y ago
  They already had a verification feature deployed which wasn't perfect but was reasonably strict. Then they debased it for a new revenue stream. I don't see them making the now that we've got a bunch of people paying for it, lets reduce our revenue by suddenly becoming strict on it again decision. Or people taking the badge more seriously now it just means someone subscribed to Twitter Blue
halJordan 4y ago
I get the "i hate Twitter/elon" attitude, but the truth is that the CA system has been dependent on pay-to-verify for some time now and despite whatever grievances you have against verisign or whoever it's brought us to today safely enough.
- commandersaki 4y ago
  The pay-to-verify is the reason why a significant chunk of the Internet didn't have HTTPS. It was the inception of free CAs like Lets Encrypt that made 90+% HTTPS penetration.
  - xeromal 4y ago
    I'd argue laziness is the reason most of the internet didn't have HTTPS. Browsers putting a scary checkmark next to the site is what made it happen.
- joe_the_user 4y ago
  And the problem with pay for identity is you can buy other people's identities. The news was full impersonations right after Twitter initiated it's pay for blue check marks. That we don't here about it now may just be a matter that the news stopped caring. URL squatting was a problem for a long time - the decision that trademark holder could seize domains helped but a web address today isn't considered a solid identity at this point so "the system working" is a bit of an exaggeration. Also, I'd note the gp didn't bad-mouth Elon or Verisign so your comment is a kind of riding a trolly false accusation.
- giaour 4y ago
  I think the problem was not that Twitter started charging $8/month ago much as that they gave up the expensive-to-perform ID verification workflow that had been in place. The CA system, on the other hand, has more or less abandoned the expensive, manual "extended verification" process in favor of the zero marginal cost "domain verification" process. This switchover drove the cost of a new cert to $0 for almost all users. So I'm really not following your analogy, since Twitter gave up the workflow and started charging (aka, a cash grab), whereas CAs gave up charging and adopted a more efficient workflow (aka the expected behavior of efficient markets).
pessimizer 4y ago
> a symbol which had previously meant they'd attempted verification That's not what it meant. It was an award for being notable. $8 is a fee that you pay to get verified.
- notahacker 4y ago
  It was an award for being an authentic notable person or entity, on the basis that the notable people and entities were most likely to be parodied or faked. Now it's an award for people that pay up
  - 300bps 4y ago
    It was an award for being an authentic notable person or entity Using an arbitrary and biased determinant of who is notable. https://www.wbur.org/cognoscenti/2021/06/28/twitter-verification-blue-check-yashar-ali-kaivan-shroff https://www.wbur.org/cognoscenti/2021/06/28/twitter-verifica...
    - deleted 4y ago
      [deleted]
    - palata 4y ago
      Maybe, but at least the verification seemed meaningful. And it feels like a meaningful verification doesn't generalise well: it's easy to pick a few people (arbitrarily) and truly verify them. But verifying millions of people... that's hard, unless you don't truly verify (in which case it's meaningless).
  - bombcar 4y ago
    It started out as “this is who you expect it is” for things like Obama or Trump, then it became “this is a person who speaks with authority of some sort, listen to them” and now it is just “someone pays eightbux”
    - pessimizer 4y ago
m3kw9 4y ago
It helps because the cost of impersonating wont make sense for a lot of people
- bombcar 4y ago
  This is a huge part of it. Shitposting for $0 is a lot different than shitposting for $8. Some will bust out the card to do so, but not many.
  - kube-system 4y ago
    The problem is that those who are willing to break out the $8 are probably the same ones who have the most to gain from it.
  - 8note 4y ago
    More importantly, $8 includes an identity check
benced 4y ago
Elon turning a $8 subscription into a culture war artifact doesn’t invalidate the idea of a subscribing for legitimacy altogether.
- notahacker 4y ago
  Agreed it doesn't completely invalidate it as a concept, but Elon turning the highest profile somewhat reliable implementation of an authenticity badge into a culture war artifact doesn't exactly bode well for it being a social media must-pay-for. The fact Facebook once aspired to be the platform where everyone used their real name and now can't be bothered to deactivate friendspamming sexbots without most users caring suggests that ordinary people won't exactly be queueing up to pay them $144 per annum because of their inherent trustworthiness as a verifier either. It'd probably work a bit on LinkedIn because of the nature of the user base and lots of people already expensing Premium accounts, but funnily enough I'm not sure LinkedIn actually has that much of a fake account problem...
  - ericmay 4y ago
    > but funnily enough I'm not sure LinkedIn actually has that much of a fake account problem... Oh it does for sure, unfortunately many people are more than happy to connect with anyone who wants to connect with them because “LinkedIn doesn’t have a fake account problem”. And many people post resumes, entire employment history, all professional social connections, etc. and so all you need to do is pretend to be a student of your targets alma mater and now you have all of that personal info with the click of a Connect button. I’ve noticed trends where someone would try to connect with me and they’d be connected with some acquaintances of mine, but not once have I ever heard of that person. Those types of accounts exist, are fake, and steal information. If you haven’t 1-1 met someone before in person or via actual business relationship you should definitely not connect with them on LinkedIn.
    - notahacker 4y ago
      Most of those accounts are actual people registered with their actual name using spamming tools (some of them LinkedIn approved) to connect with as many people as possible though, usually to boost the appearance of their personal profile and have as wide a possible audience for promotional messages they share rather than to circumvent LinkedIn's information paywalls. Spamming people loosely connected with your industry with what you're doing at work is, after all, LinkedIn's intended purpose...

iLoveOncall 4y ago

Sending a photoshopped ID to Facebook to get the verified badge on your fake account is orders of magnitude easier than everything you mention here. Those verifications are 100% useless for non public figures.

palata 4y ago
Agreed. Either they get it right (and then congrats, the Facebook account is now a passport), or they don't, and it's completely meaningless because it still means "well don't trust it though".
zeroonetwothree 4y ago
Yeah but you also have to pay a bunch of money. That's going to discourage scammers at least some amount.
- iLoveOncall 4y ago
  It won't. Actually the fact that it's paid makes it even more useless, because the vast majority of users will NOT be verified. I would never, ever, pay for this verification for example. Therefore a non-verified user will not look less trustable than a verified one.

stefan_ 4y ago

Who cares? Is the AI gonna RSVP to my event? Haggle over my 2nd hand item? Comment on whatever the local council has planned? All these schemes tend to fall apart at the first contact with reality. (Also, you are talking about "Facebook verifying users". They can't even verify who is paying them for political ads, and they certainly don't seem to care very much.)

michaelt 4y ago
Well, there clearly are bots on social media, doing things like reposting popular posts to farm reputation points. The idea these bots are doing that for a reason doesn't seem so crazy to me.
urbandw311er 4y ago
I think it’s still a relevant problem we will face as a society with regards to bot farms etc running social interference (eg election propaganda)
oneoff786 4y ago
> Comment on whatever the local council has planned? That seems plausible? Already achievable with some interns mind you. But still
- blitzar 4y ago
  Having spent all of 5 minutes on "nextdoor" - it isnt hard to whip some locals into complete hysteria.
IshKebab 4y ago
They will definitely haggle over your second hand item. Ever tried selling an Apple product on Facebook marketplace? You get a ton of scammers and you can't tell with some of them (until they try the actual scam, by which point you've wasted time messaging them, packaging the item, etc.)

linuxftw 4y ago

Or, you know, we can just go back to circa 2000 and tell people not to trust everything they see on the internet as fact?

woeirua 4y ago
Propaganda works. Even when people know it’s probably fake.
- kube-system 4y ago
  You can straight up tell people something is fake, and they will still entertain the thought enough to have an impact on their perspective. Consider popular novels which are fictional social commentaries. Or culturally influential Hollywood movies.
  - LadyCailin 4y ago
    God yes. I absolutely hate it when people cite 1984 as “what happens when you have large government.” As if no one could write a dystopian fiction novel based on whatever world view that person has.
JasserInicide 4y ago
Tell that to employers that now just Google you and look at the top search results

verisimi 4y ago

You're going to be tracked and traced everywhere, and you're going to pay for the privilege. Amazing what they can get us to do! This is the reason musk bought twitter too, of course.

palata 4y ago
I thought he bought it to prove what a moron and a jerk he is. Maybe that's just a side effect :)

shapefrog 4y ago

> I think pay-for-verification is going to be dominant on all real-identity social media. Bad actors are going to be happy to pay $10/20/30/40 a month to scam people, its their job and livelihood.

ericd 4y ago
It raises the cost of making a fake account by orders of magnitude, it's useful against bad actor bot accounts at scale.
- palata 4y ago
  Shouldn't there be a difference between "verified" and "paid"? If bad actors can just pay to have a "verified" account, then it's completely meaningless, isn't it? Either they truly verify (which is probably hard) and I can pay to be verified, or they don't, and it's completely useless because nobody should trust the verification...
  - ericd 4y ago
    It’s not that black and white, most of these things are a mix. Some verification up front, and the normal ongoing monitoring of behavior. Putting it all up front usually presents too large a barrier to users, and causes conversion to suffer (and it’s harder to be sure than just catching bad behavior once they start using the service). Taking money makes getting banned a much larger penalty for bad actors. And that makes a lot of bad behavior unprofitable, and that subset will stop, and reduce load on their spam fighting efforts.
    - palata 4y ago
      Right, that's interesting! Thanks for the insights :) > and the normal ongoing monitoring of behavior Now I get another question: say you have millions of paid, verified users (that's the goal, right?). If you can monitor millions of account successfully, then it's most likely automated, isn't it? And then it should scale to all the users, maybe? In which case again, "verified" feels meaningless (because they just improved the bot situation overall). > And that makes a lot of bad behavior unprofitable I wonder if the bad behavior that can be reasonably well detected is the unprofitable one. Say I can leverage the "verified" badge to scam people more efficiently -> probably I can pay 8$ for an account that will scam a few people, right? So really paying 8$ would be a deterrent for bots that purposely do disinformation, but there it's hard to know if it's a bot, isn't it? I mean many people do disinformation just out of a lack of knowledge, and that's not a reason to qualify them as bots, right?

bogomipz 4y ago

>"There's no real defense against AI impersonation at scale, except for charging-for-verification." What's to stop a generative AI from generating a state issued drivers license or similar to go along with that avatar photo though? Also how different is that really though from all the existing non-AI generated fake accounts?

oneoff786 4y ago
Well that would be a felony to start. Which is more noteworthy than just against terms of service.
- nebulousthree 4y ago
  If McDonald's required showing your driver's license to pick up your order, would it be a felony to show a fake license and pick up an order? You're not lying to a government organisation. You're just obscuring your identity to a private corporation. I fail to see how that can be any more than unethical.
  - ericbarrett 4y ago
    Showing a false driver's license in the U.S. is usually a crime (though not often a felony by itself) regardless of circumstance; e.g. Texas law: https://texas.public.law/statutes/tex._transp._code_section_521.451 https://texas.public.law/statutes/tex._transp._code_section_... Note that a "Class A misdeamenor" in Texas is punishable by a $4,000 fine and/or 1 year in jail. Is that going to happen because you used it to pick up a food order? Probably not, but stranger things have happened...
  - oneoff786 4y ago
    No it would not. And it would not be a felony to show one to Facebook either. But the question was about generating new IDs with AI. That is a felony.
- bogomipz 4y ago
  Can you explain how an AI-generated fake ID for a fake social media account constitutes a felony? On what grounds exactly? Facebook despite attempting to normalize the handing over of government-issued identification to an advertising platform, is not a government entity.

schainks 4y ago

> provides resources on the other end to moderate impersonation. Eeeehhhhhh, I don't think so.

nip 4y ago

> There's no real defense against AI impersonation at scale, except for charging-for-verification. There is: have the state take care of that as you are a citizen that pay taxes (you already are paying for this "verification") – an ID card that contains a proof of you being... you. https://openid.ee/en/ https://openid.ee/en/

ikesau 4y ago
Similarly, New Zealand's government-provided SAML 2.0 authentication service called RealMe is really good. https://developers.realme.govt.nz https://developers.realme.govt.nz But the U.S. has a phobia of any form of national ID card, so I doubt that the USDS will be able to build anything soon. It's such a shame. At least having it as an option could allow society's to start experimenting with it as a solution to these problems.
- schmorptron 4y ago
  I'd like a govt service where you can verify that you're a real person but not your identity or anything of the sort. Sort of like having the third party registering with the government and then you get a token per account you make so the service can verify there isn't more than one accounts per person but they can't identify you across the services. Would also be immensely helpful useful for stopping cheating in online video games, while not requiring you to give our your real name and identity to anything

wkat4242 4y ago

Impersonation or creating fake personas has always been possible. In the security community these are called "sockpuppets". It's just less work now.

tangjurine 4y ago

How about meeting people in person and then asking for their social media username? Most people use social media to keep track of people they already know.

vineyardmike 4y ago
> Most people use social media to keep track of people they already know. And most people who drive for Uber are just doing so to make a few bucks in their daily life. And most people on AirBnb are renting their spare room or couch. /s Most people follow a handful of a famous people or companies, and some friends. Especially on twitter, which was very "one to many" following relationships. Thats why verification was so impactful, it gave the "one" in "one to many" something special, and a way to establish authenticity.

jasmer 4y ago

"There's no real defense against AI impersonation at scale," Defamation liability. I'm not sure the laws have worked out yet, but I suggest that these companies may be in for some lawsuits if thing go awry, and they have not shown to be actively working against it. I'm looking forward to that one.

elcomet 4y ago

It's not about verification, it's about customer support. Verification could be provided for free with ID cards or something like this.