6 ms·

Authors making this argument (which we see often now) really need to sketch out why cryptographic solutions will fail. The entire thesis rests on this erroneou

by anselm 4y ago

Authors making this argument (which we see often now) really need to sketch out why cryptographic solutions will fail. The entire thesis rests on this erroneous sentence in the second point: “This can be done in two ways – just move to invite-only silos where you already know everybody, or big platforms where the owners do the vetting for you.” There are more options. We employ them when we are forced to; when it becomes cheaper than not. We can sign posts or sign that somebody is real or to sign that somebody has earned rep, or that somebody has burned their rep… and subjectively score every piece of content that crosses our phone against our social trust graph. We can rhizomatically scale social networks, can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter. There’s no specific reason why we cannot grow a kind of social nervous system that has a kind of myelin sheath against noise and spam. It doesn’t have to be specifically only people we’ve shaken hands with or our like 3 closest buddies.

awestroke 4y ago
So, people start signing their AI-generated messages. What have we gained?
- NoGravitas 4y ago
  Indeed. SPF and DKIM were supposed to reduce spam by ensuring that every sender was verified. Now, we have more spam than ever, and all the senders are verified (on short-lived garbage domains that are not yet on blocklists). The only DKIM failures I ever see are on legitimate mail from badly set-up lists.
  - kolinko 4y ago
    DKIM and SPF are more important for phishing than for spam. Without them, it’s very easy to spoof senders.
- treis 4y ago
  You can ban them and they'll stay banned. Today they'll come back or switch to one of the other dozen accounts they post crap on. The counter argument to that is that places like Facebook are mostly not pseudonymous and a lot of crap is posted there.
  - djmips 4y ago
    And when you get banned because someone stole your key - that's going to be awesome.
    - anselm 4y ago
      There are going to be a few cases where keys are lost or stolen. It may be possible to build multi-sig wallets that allow for you to migrate an identity to a new key. What we're looking for is some kind of statistical means of trying to reduce bad actors. Even if there was no recourse for you and you were totally screwed I'm not sure it totally invalidates the concept of having a key or a mechanism for trying to remove bad actors from social discourse. You could trip and die because you wore shoes. You could get locked out of your house... bad stuff does happen - it doesn't mean we shouldn't wear shoes or have keys.
  - awestroke 4y ago
    Ban them on what grounds?
  - HDThoreaun 4y ago
    AI generated content will likely be higher quality than what we have today. AI accounts will spend time building reputation with superior content and then burn some on an ad or spam campaign, but they’ll still have a better reputation than most people.
- anselm 4y ago
  It is gonna happen for sure. People will leverage powerful tools and claim it is their own voice. Me shrugs. I more want to at least have that individual emitter be accountable for what they post; to establish continuity. I want to know that that emitter is say 3 hops from me, and is trusted by 12 friends in between, and has a general trust score of say 7/10 overall as an overall rating by my extended trust graph. It is less that I want people to say good things, or be truthful or whatnot - I just want to know that they are real, that there is a human behind it, that that human has an opinion of some kind. The thing is that there are a ton of sock puppets that are not real - it's more about reducing the noise / spam rather than a perfect solution.
xg15 4y ago
The same problem we have with other networks of trust: Signing works alright if you already know the people you expect messages from (but in that case, it's also no better than the invite-only social networks the OP talked about). However, the real problem is getting to know new people or vetting messages from people you don't know. In the future the OP sketched, you can never be sure that an interesting new person isn't actually a bot. Knowing the public key of that person won't solve that problem.
rocqua 4y ago
One of the biggest problems in actually using cryptography in the real world is matching keys to identities. Using cryptography to solve the fake persona problem only works if the key-identity matching problem is solved. It would be great if the fake persona problem was the impetus that managed a solution to the key-identity matching problem. But I have my doubts. Notably the key-identity matching problem isn't technical. Its societal. "just do government provided keys" is technically easy, but society (rightfully so) is suspicious of this. Other solutions exist, with other trade-offs. SSL certificates have centralization, revocation, and weakest-link problems. PGP-keys have spoofing, verification, and usability problems (though I liked key-base's approach here). European E-id is an interesting step, one to watch, though I fear the EU bureaucratic system might make a crucial mis-step. I really like SSI based approaches, but SSI is mostly about using crypto when the key-identity matching problem has been solved, and less about solving the actual problem. Some technical aspects that need solutions that tend to be un-acceptable are handling key-revocation, key-theft, key-loss (as in forgetting), and key-duplication.
- px43 4y ago
  How about we let people generate their own keys, then use those keys to make identity claims, when people can generate on their own, or which can be generated by a third party. That gives multiple options to bind identities to arbitrary social media accounts etc without needing some monolithic root of trust. I also really like the idea of using the keys to hold some amount of value, such that if the keys ever get leaked, there is basically a built in bug bounty to alert the key holder (since the key thief has the option to take all the money). This also gives users the incentive to manage their keys in a sane way. Social key management schemes are also super interesting, and will likely be a part of future key management schemes. That is, basically, allowing some set of friends and family to re-roll or revoke identities that have been lost or stolen. Slowly but surely, I think this future is coming. Lots of good people are coming at it from different angles, but basically all converging on the same general concepts.
  - throw10920 4y ago
    > How about we let people generate their own keys, then use those keys to make identity claims, when people can generate on their own, or which can be generated by a third party. That gives multiple options to bind identities to arbitrary social media accounts etc without needing some monolithic root of trust. You hit the nail on the head. Matching keys to real people can be done in-person for direct friends, then through a web of trust for indirect friends. For accounts/keys/personas you find on the internet where you don't have a chain of friends, you can either rely on "trusted" third-party attestation ("holder of key 0xdeadbeef earned a degree from this university") - you may never know with complete certainty if that's a real human, a bot, or an alt account for someone you already know, and that's totally fine. The "problem" of matching keys 1-to-1 with identities for everyone globally (brought up by grandparent post) is a massive red herring that doesn't need to be "solved".
peteradio 4y ago
You can employ those techniques but real people will get blocked as spam. And the better AI can evade the closer to the bone you will have to cut. Then what? AI is interacting with your algorithm to silence voices.
- anselm 4y ago
  Even with some defects or imperfections anything is better than what we have now - which is basically nothing. I think the way I'd think about this is to imagine say a small community, such as a town of say 5000 people or so. While you cannot know each person individually, you can know of people by reputation. People do earn rep over time, and they can burn rep. It is true that some people will be unfairly downscored, or unfairly upscored - but I'm not really trying to argue for those fine grained situations. What I'm trying to argue for is simply distinguishing the very bad actors acting out of pure malice from injecting fake news, media and 'yellow journalism' into human conversations. True some real people will be downscored (I prefer to think of this as downscoring bad actors rather than 'blocking'). And true an AI can 'sound very human' - but an AI or a bad actor will struggle to build up a reputation over time. An AI can't shake hands with you, it is harder for it to prove it is human... Other bad actors will presumably burn their reputations if they spit out a series of offensive, misleading, false, inflammatory or toxic posts... Note I am not necessarily advocating for crypto per se as a way to establish social trust graphs (a'la PGP or say Keybase) but I am arguing that there are other options that the OP did not raise. I more want to see a wider discussion around ways to filter malicious media that either "centralized systems" OR "small social clubs". I'm not necessarily saying it has to be a cryptographic solution... but I do think there are more ways to have what we want.
vlovich123 4y ago
Because using technology to solve human problems rarely/never works. [1] was originally written for spam and bore mostly correct. Explain how replacing spam with “AI-generated spam” changes anything? You can try to fight this stuff but it will look more like AI to detect AI (similar to our current anti-spam tech). No reason to believe cryptography has some kind of magical bullet here as it’s an unrelated problem domain. And to the person claiming you get kicked off and that prevents you from coming back ignores a) we haven’t solved being able to tie disparate online personas for a unique offline one (despite Facebook ostensibly trying really hard) b) there are all sorts of secondary problems that pop up when you try to do that (eg ignores the concept of learning from your mistakes and redemption, key things that happen frequently with the young or anyone else testing boundaries). [1] https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wont-work-checklist/ https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wo...
- throw10920 4y ago
  > Because using technology to solve human problems rarely/never works. You're badly misunderstanding the parent post - it is not proposing a technological solution to a human problem, but a technological enforcement of a fundamentally human solution: > subjectively score every piece of content that crosses our phone against our social trust graph...can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter This is a social web of trust, where real people do the ranking and trust assignments - the cryptography and other technology just keeps track of bookkeeping.
  - vlovich123 4y ago
    Given that GPT is already difficult to distinguish from a person who’s confidently wrong, how does this web of trust system solve the problem? The belief that anything will “solve” this seems naive when there’s 20+ years of proof of this being an “unsolvable” problem despite repeated technological, social, and legislative attempts. There might be a new normal established with new battlegrounds drawn and we learn to “live” with it, but I’m willing to bet non-trivial domes of money against there being any true “solution” here.
wanderingbit 4y ago
Context: I've been working in the cryptocurrency space for a decade. One of the most interesting technologies the crypto space is working on are the SSI (self-sovereign identities). https://en.wikipedia.org/wiki/Self-sovereign_identity https://en.wikipedia.org/wiki/Self-sovereign_identity Even if the whole idea of cryptocurrencies is found to be a dead-end, the fact that it invigorated the research and development of SSI will change some important things about how we operate online. There exist prototypes of tech that allows you to prove you are indeed a unique human being online [1], and reveal nothing else about your identity. Most importantly, this tech is not owned or controlled by any FAANG or government, it's an open protocol just like email. I have listened to a podcast with an expert researcher in AI, and I remember him saying that he predicts some form of cryptographic identity will arise in order to help deal with the bot problem [1] https://worldcoin.org/ https://worldcoin.org/ (note, I don't work for them, and have no idea if this will be the tech that finally breaks out, I just think they're the furthest along of any of their competitors)

by anselm 4y ago

awestroke 4y ago

So, people start signing their AI-generated messages. What have we gained?

NoGravitas 4y ago
Indeed. SPF and DKIM were supposed to reduce spam by ensuring that every sender was verified. Now, we have more spam than ever, and all the senders are verified (on short-lived garbage domains that are not yet on blocklists). The only DKIM failures I ever see are on legitimate mail from badly set-up lists.
- kolinko 4y ago
  DKIM and SPF are more important for phishing than for spam. Without them, it’s very easy to spoof senders.
treis 4y ago
You can ban them and they'll stay banned. Today they'll come back or switch to one of the other dozen accounts they post crap on. The counter argument to that is that places like Facebook are mostly not pseudonymous and a lot of crap is posted there.
- djmips 4y ago
  And when you get banned because someone stole your key - that's going to be awesome.
  - anselm 4y ago
    There are going to be a few cases where keys are lost or stolen. It may be possible to build multi-sig wallets that allow for you to migrate an identity to a new key. What we're looking for is some kind of statistical means of trying to reduce bad actors. Even if there was no recourse for you and you were totally screwed I'm not sure it totally invalidates the concept of having a key or a mechanism for trying to remove bad actors from social discourse. You could trip and die because you wore shoes. You could get locked out of your house... bad stuff does happen - it doesn't mean we shouldn't wear shoes or have keys.
- awestroke 4y ago
  Ban them on what grounds?
- HDThoreaun 4y ago
  AI generated content will likely be higher quality than what we have today. AI accounts will spend time building reputation with superior content and then burn some on an ad or spam campaign, but they’ll still have a better reputation than most people.
anselm 4y ago
It is gonna happen for sure. People will leverage powerful tools and claim it is their own voice. Me shrugs. I more want to at least have that individual emitter be accountable for what they post; to establish continuity. I want to know that that emitter is say 3 hops from me, and is trusted by 12 friends in between, and has a general trust score of say 7/10 overall as an overall rating by my extended trust graph. It is less that I want people to say good things, or be truthful or whatnot - I just want to know that they are real, that there is a human behind it, that that human has an opinion of some kind. The thing is that there are a ton of sock puppets that are not real - it's more about reducing the noise / spam rather than a perfect solution.

xg15 4y ago

The same problem we have with other networks of trust: Signing works alright if you already know the people you expect messages from (but in that case, it's also no better than the invite-only social networks the OP talked about). However, the real problem is getting to know new people or vetting messages from people you don't know. In the future the OP sketched, you can never be sure that an interesting new person isn't actually a bot. Knowing the public key of that person won't solve that problem.

rocqua 4y ago

One of the biggest problems in actually using cryptography in the real world is matching keys to identities. Using cryptography to solve the fake persona problem only works if the key-identity matching problem is solved. It would be great if the fake persona problem was the impetus that managed a solution to the key-identity matching problem. But I have my doubts. Notably the key-identity matching problem isn't technical. Its societal. "just do government provided keys" is technically easy, but society (rightfully so) is suspicious of this. Other solutions exist, with other trade-offs. SSL certificates have centralization, revocation, and weakest-link problems. PGP-keys have spoofing, verification, and usability problems (though I liked key-base's approach here). European E-id is an interesting step, one to watch, though I fear the EU bureaucratic system might make a crucial mis-step. I really like SSI based approaches, but SSI is mostly about using crypto when the key-identity matching problem has been solved, and less about solving the actual problem. Some technical aspects that need solutions that tend to be un-acceptable are handling key-revocation, key-theft, key-loss (as in forgetting), and key-duplication.

px43 4y ago
How about we let people generate their own keys, then use those keys to make identity claims, when people can generate on their own, or which can be generated by a third party. That gives multiple options to bind identities to arbitrary social media accounts etc without needing some monolithic root of trust. I also really like the idea of using the keys to hold some amount of value, such that if the keys ever get leaked, there is basically a built in bug bounty to alert the key holder (since the key thief has the option to take all the money). This also gives users the incentive to manage their keys in a sane way. Social key management schemes are also super interesting, and will likely be a part of future key management schemes. That is, basically, allowing some set of friends and family to re-roll or revoke identities that have been lost or stolen. Slowly but surely, I think this future is coming. Lots of good people are coming at it from different angles, but basically all converging on the same general concepts.
- throw10920 4y ago
  > How about we let people generate their own keys, then use those keys to make identity claims, when people can generate on their own, or which can be generated by a third party. That gives multiple options to bind identities to arbitrary social media accounts etc without needing some monolithic root of trust. You hit the nail on the head. Matching keys to real people can be done in-person for direct friends, then through a web of trust for indirect friends. For accounts/keys/personas you find on the internet where you don't have a chain of friends, you can either rely on "trusted" third-party attestation ("holder of key 0xdeadbeef earned a degree from this university") - you may never know with complete certainty if that's a real human, a bot, or an alt account for someone you already know, and that's totally fine. The "problem" of matching keys 1-to-1 with identities for everyone globally (brought up by grandparent post) is a massive red herring that doesn't need to be "solved".

peteradio 4y ago

You can employ those techniques but real people will get blocked as spam. And the better AI can evade the closer to the bone you will have to cut. Then what? AI is interacting with your algorithm to silence voices.

anselm 4y ago
Even with some defects or imperfections anything is better than what we have now - which is basically nothing. I think the way I'd think about this is to imagine say a small community, such as a town of say 5000 people or so. While you cannot know each person individually, you can know of people by reputation. People do earn rep over time, and they can burn rep. It is true that some people will be unfairly downscored, or unfairly upscored - but I'm not really trying to argue for those fine grained situations. What I'm trying to argue for is simply distinguishing the very bad actors acting out of pure malice from injecting fake news, media and 'yellow journalism' into human conversations. True some real people will be downscored (I prefer to think of this as downscoring bad actors rather than 'blocking'). And true an AI can 'sound very human' - but an AI or a bad actor will struggle to build up a reputation over time. An AI can't shake hands with you, it is harder for it to prove it is human... Other bad actors will presumably burn their reputations if they spit out a series of offensive, misleading, false, inflammatory or toxic posts... Note I am not necessarily advocating for crypto per se as a way to establish social trust graphs (a'la PGP or say Keybase) but I am arguing that there are other options that the OP did not raise. I more want to see a wider discussion around ways to filter malicious media that either "centralized systems" OR "small social clubs". I'm not necessarily saying it has to be a cryptographic solution... but I do think there are more ways to have what we want.

vlovich123 4y ago

Because using technology to solve human problems rarely/never works. [1] was originally written for spam and bore mostly correct. Explain how replacing spam with “AI-generated spam” changes anything? You can try to fight this stuff but it will look more like AI to detect AI (similar to our current anti-spam tech). No reason to believe cryptography has some kind of magical bullet here as it’s an unrelated problem domain. And to the person claiming you get kicked off and that prevents you from coming back ignores a) we haven’t solved being able to tie disparate online personas for a unique offline one (despite Facebook ostensibly trying really hard) b) there are all sorts of secondary problems that pop up when you try to do that (eg ignores the concept of learning from your mistakes and redemption, key things that happen frequently with the young or anyone else testing boundaries). [1] https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wont-work-checklist/ https://trog.qgl.org/20081217/the-why-your-anti-spam-idea-wo...

throw10920 4y ago
> Because using technology to solve human problems rarely/never works. You're badly misunderstanding the parent post - it is not proposing a technological solution to a human problem, but a technological enforcement of a fundamentally human solution: > subjectively score every piece of content that crosses our phone against our social trust graph...can deputize our people in our extended social network to mark content as appropriate for kids or not, or otherwise filter This is a social web of trust, where real people do the ranking and trust assignments - the cryptography and other technology just keeps track of bookkeeping.
- vlovich123 4y ago
  Given that GPT is already difficult to distinguish from a person who’s confidently wrong, how does this web of trust system solve the problem? The belief that anything will “solve” this seems naive when there’s 20+ years of proof of this being an “unsolvable” problem despite repeated technological, social, and legislative attempts. There might be a new normal established with new battlegrounds drawn and we learn to “live” with it, but I’m willing to bet non-trivial domes of money against there being any true “solution” here.

wanderingbit 4y ago

Context: I've been working in the cryptocurrency space for a decade. One of the most interesting technologies the crypto space is working on are the SSI (self-sovereign identities). https://en.wikipedia.org/wiki/Self-sovereign_identity https://en.wikipedia.org/wiki/Self-sovereign_identity Even if the whole idea of cryptocurrencies is found to be a dead-end, the fact that it invigorated the research and development of SSI will change some important things about how we operate online. There exist prototypes of tech that allows you to prove you are indeed a unique human being online [1], and reveal nothing else about your identity. Most importantly, this tech is not owned or controlled by any FAANG or government, it's an open protocol just like email. I have listened to a podcast with an expert researcher in AI, and I remember him saying that he predicts some form of cryptographic identity will arise in order to help deal with the bot problem [1] https://worldcoin.org/ https://worldcoin.org/ (note, I don't work for them, and have no idea if this will be the tech that finally breaks out, I just think they're the furthest along of any of their competitors)