8 ms·

The inability to simultaneously verify sentience, location, and identity

106 points by DemiGuru 3y ago

BSEdlMMldESB 3y ago
we cannot live in a society where I must demonstrate to another human that I'm human with a piece of paper just because the other human is a bureaucrat with a computer. this is an 'online only' problem
- chayesfss 3y ago
  [dead]
- reilly3000 3y ago
  I think you described the process of purchasing any controlled substance at a corner store.
  - bena 3y ago
    No, the paper proves which human you are. That you are human is assumed.
RcouF1uZ4gsC 3y ago
Nit Pet Peeve: Confusing sapience and sentience.
- colechristensen 3y ago
  I'm starting to doubt the Internet nitpickers' distinction and definition of the two.
  - DougMerritt 3y ago
    Their distinction follows dictionaries that I've looked at, but common usage is clearly diverging from that. It's unclear to me at what point we should stop saying that lots of people are using words incorrectly, and start saying that lots of dictionaries are sticking with outmoded definitions, but looking at the past, transitions certainly occur sometimes.
    - greiskul 3y ago
      There are lots of people that have this view that there is "objectively correct" language, and that you can find it in grammar books and dictionaries. Any linguist worth their salt knows that is a completely outdated and classist way of studying language. Dictionaries are supposed to be updated acording to how people use language, not people be "corrected" to try to follow the dictionary. While language learners speak a language "wrong", any group of native language speakers has rights of ownership of their language as any other group, and as long as they are communicating in a way that they can understand each other, they are never wrong.
      - DougMerritt 3y ago
        As an avocational linguist I am sympathetic to all of that, but effective communication requires a large degree of mutual understanding of words, so it's not like anything goes, either; plus most people have not studied linguistics and do believe in "correct language" (prescriptive) and do not hesitate to correct people who use "incorrect language" -- so it's a bit complicated culturally, if not from the stance of descriptive linguistics. In this particular case, there's no danger of misunderstanding, so I agree that this was just a nitpick about dictionary definitions versus a fairly common usage.
- Scaevolus 3y ago
  On the internet, the problem is rarely dogs getting up to mischief.
  - rocketbop 3y ago
    That’s a nice line but could you tell me what it means?
    - ketzo 3y ago
      We consider many animals to be "sentient" -- we recognize that they are to some degree conscious, in this case meaning that they have the capability to sense, perceive their world, and have some kind of emotion about the things they perceive "Sapience" on the other hand, is essentially "human-level intelligence, consciousness, and self-awareness" (from homo sapiens)
    - willturman 3y ago
      I took it as a tongue in cheek reference to this: https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
  - rossdavidh 3y ago
    ...so far.
  - qingcharles 3y ago
    That's exactly what a dog would say.
someguy7250 3y ago
IMO, the meat of this paper is in section 4.3 and 4.4. And I cannot say for sure, but the formal proof of 4.4 basically summarizes the same points pointed out in 4.3. Most of these are not inherently mathematical problems but a social one. > Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡. Basically, people can sell identities ---- What really concerns me though, is how much and how often this paper discusses DRM, or in their own words, a "trust anchor" > With the assumed threat model in our case, the lack of inherent trust in the user only compounds the unreliability of the model without any trust anchor. > Assuming a proof of location is for a mobile device, rather than a particular human being, then associating the proof of uniqueness obtained under such a condition, i.e., without the involvement of a trust anchor, is unreliable. I know that the authors aren't directly calling for more centralized trust. But given recent development at Google, we all know how the readers would think
- MichaelZuo 3y ago
  > > Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡. > Basically, people can sell identities I can see why Sam Altman believes iris scans are the future, it's definitely much more cumbersome to 'sell off' your iris. Especially if it needs to be rescanned on a daily basis or sooner.
  - wmf 3y ago
    But Worldcoin isn't doing that. They scan your eyes once and issue you a private key that you can then sell. Maybe one day they can give away scanners CueCat-style but I haven't seen any discussion of that.
    - MichaelZuo 3y ago
version_five 3y ago
> Trolls, bots, and sybils distort online discourse and compromise the security of networked platforms. In some sense I think the authors' hypothesis is a good thing, ie that you can never fully verify someone online. It prevents wholesale algorithmic management of people, which is really what governments and companies would like to do, and forced some level of human contact or at least intervention. I expect it's inevitable that they'll find a way to offload the problem onto the citizen, for the most part they already have, but I'm personally glad it's impossible to assign me some kind of infallible identifier that will let me be The Castle style abused remotely and without recourse.
dcow 3y ago
I think a lot of people try to do identity things without understanding the fundamental nature of the problem they're attacking. For instance: I'm really really worried that governments are going to default into an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document. Or: I'm really annoyed that software services and web apps have clauses like "you can't use scripts or automation software to access our API" when a browser is just that. And they should really be enforcing rate limits and punishing abusive behavior whether the user clicked a button in a browser or a script did. These type of things are not rooted in a fundamental understanding of identity, they're sloppy stop-gaps. Despite all its faults, this is one of the reasons I'm super excited about WebAuthN. At least is make common the idea that an identity is a cryptographic secret and not a "possession of an email and phone". We really really need to dig out of this "email address identifies you" hole. Anyway it's exciting to see people discuss the topic more formally. It gives me hope that we can ultimately get to a better understanding of digital identity and not be trying to solve impossible problems by chasing an impossibly perfect solution that verifies all 3 tenets that actually doesn't exist and making a big mess of things because nobody stopped to ask or understand the scope of what we should be trying to do. Ultimately identity should be empowering not oppressive. And right now it feels more like services oppress people into all sorts of weird requirements like having an email, getting a phone verification code, running software on a device that has an integrity attestation framework, etc. rather than trust them and punish bad behavior. I want my government (and web services, but especially my government) to trust me and punish bad behavior, not treat me like an untrusted bot that needs to be managed and continuously verified.
- OfSanguineFire 3y ago
  > an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document. Buying a SIM card in a great many countries today already requires showing state-approved ID and then signing a form that the shop clerk prints out. So, ownership of a mobile phone number does mean being able to sign a document. Are you concerned about SIM-jacking? I admit, I find the thrust of your post difficult to follow.
Morizero 3y ago
Title should read "sentience, location, and uniqueness", which the paper states are the three key properties of identity
dontupvoteme 3y ago
Half the internet thinks troll means "person who disagrees with me or (dis)likes thing i (dis)like" - I'm distrustful of people who paint them as a big problem on the internet. Is this how the western Social Credit system begins?
- pphysch 3y ago
  Trolls, as in troll farms, astroturfing, organized influence campaigns, etc. are absolutely a serious problem for any society that pretends to care about democracy. Especially in the LLM era, where the marginal cost of adding another artificial "voice" approaches $0.
  - dontupvoteme 3y ago
    That's not what troll means. Astroturfing as you said is much better. They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they?
    - pphysch 3y ago
      If that is not one of the definitions of troll, then can you please explain what a "troll farm" is supposed to be? > They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they? It's not a bogeyman. It's a real, current, serious problem.
      - whatshisface 3y ago
        If the troll farms are responsible for online attacks on our civil liberties, why would they use themselves, troll farms, as the justification? I rather think the attacks on our freedom are being made by real people who are misguided about the tradeoffs between letting people speak when they are wrong, and making it easy for officials covering up misconduct.
      - ben0x539 3y ago
        "Troll farm" is also based on this new usage of the word.
jkhdigital 3y ago
It’s all the same problem that Ken Thompson identified in his classic Turing lecture: there is no way to reliably identify Trojans.
_xnmw 3y ago
What about Apple's FaceID? Secure, private, decentralized, and and verifies sentience (well atleast intactness of your physical head), location and identity.
kazinator 3y ago
The location element of the trilemma doesn't seem as important as the other two, and is a violation of privacy. Let it be hard to discover. A social site that discovers everyone's location, and reminds them it knows where they are, would probably drastically cut down on the trolling and abuse, but at what cost.
pontifier 3y ago
The philosophy of identity is quite interesting to me. I've been thinking about it for a long time. I like the idea of proof of personhood, and proof of location. These are both concepts that I have explored extensively. I looked at retina scans, and found it they're both not unique enough, and not secure enough to act as a basis for a currency. My proof of location was based on the 500 mile email story. Many millions of round trips between two devices could allow someone to prove on a blockchain that two secret keys were stored arbitrarily close to one another for some amount of time. The paper mentions key signing parties which I believe must be part of any web of trust that proves personhood. Proving uniqueness is more difficult. There's nothing that would stop me from having several different identities verified at different key signing parties, then using those to generate even more. Using genealogy seems to be the only way to truly verify a unique identity if the system becomes widespread enough that parents often verify their children at birth. Then the age of the identity matches the age of the individual. Farther into the future, the possibility of creating multiple copies of myself makes it much more difficult to tie an identity to a particular group of cells. There's nothing truly unique about a copy, especially if we have the ability to change our DNA. Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary.
- magic_hamster 3y ago
  > Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary. We are an arrangement of cells betrayed by the human character. If all we wanted was to live leisurely this could have been easily achieved after the industrial revolution. But it's our nature to be ambitious, to achieve things, to want more, to compete, to be greedy and 1up each other. Most people are never satisfied regardless of how big their house is or what they do for a living. So I don't think an idle society of abundance can exist as long as people are competing for the largest piece of the pie.
  - 3y ago

106 points by DemiGuru 3y ago

we cannot live in a society where I must demonstrate to another human that I'm human with a piece of paper just because the other human is a bureaucrat with a computer. this is an 'online only' problem

chayesfss 3y ago
[dead]
reilly3000 3y ago
I think you described the process of purchasing any controlled substance at a corner store.
- bena 3y ago
  No, the paper proves which human you are. That you are human is assumed.

RcouF1uZ4gsC 3y ago

Nit Pet Peeve: Confusing sapience and sentience.

colechristensen 3y ago
I'm starting to doubt the Internet nitpickers' distinction and definition of the two.
- DougMerritt 3y ago
  Their distinction follows dictionaries that I've looked at, but common usage is clearly diverging from that. It's unclear to me at what point we should stop saying that lots of people are using words incorrectly, and start saying that lots of dictionaries are sticking with outmoded definitions, but looking at the past, transitions certainly occur sometimes.
  - greiskul 3y ago
    There are lots of people that have this view that there is "objectively correct" language, and that you can find it in grammar books and dictionaries. Any linguist worth their salt knows that is a completely outdated and classist way of studying language. Dictionaries are supposed to be updated acording to how people use language, not people be "corrected" to try to follow the dictionary. While language learners speak a language "wrong", any group of native language speakers has rights of ownership of their language as any other group, and as long as they are communicating in a way that they can understand each other, they are never wrong.
    - DougMerritt 3y ago
      As an avocational linguist I am sympathetic to all of that, but effective communication requires a large degree of mutual understanding of words, so it's not like anything goes, either; plus most people have not studied linguistics and do believe in "correct language" (prescriptive) and do not hesitate to correct people who use "incorrect language" -- so it's a bit complicated culturally, if not from the stance of descriptive linguistics. In this particular case, there's no danger of misunderstanding, so I agree that this was just a nitpick about dictionary definitions versus a fairly common usage.
Scaevolus 3y ago
On the internet, the problem is rarely dogs getting up to mischief.
- rocketbop 3y ago
  That’s a nice line but could you tell me what it means?
  - ketzo 3y ago
    We consider many animals to be "sentient" -- we recognize that they are to some degree conscious, in this case meaning that they have the capability to sense, perceive their world, and have some kind of emotion about the things they perceive "Sapience" on the other hand, is essentially "human-level intelligence, consciousness, and self-awareness" (from homo sapiens)
  - willturman 3y ago
    I took it as a tongue in cheek reference to this: https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_you%27re_a_dog https://en.wikipedia.org/wiki/On_the_Internet,_nobody_knows_...
- rossdavidh 3y ago
  ...so far.
- qingcharles 3y ago
  That's exactly what a dog would say.

someguy7250 3y ago

IMO, the meat of this paper is in section 4.3 and 4.4. And I cannot say for sure, but the formal proof of 4.4 basically summarizes the same points pointed out in 4.3. Most of these are not inherently mathematical problems but a social one. > Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡. Basically, people can sell identities ---- What really concerns me though, is how much and how often this paper discusses DRM, or in their own words, a "trust anchor" > With the assumed threat model in our case, the lack of inherent trust in the user only compounds the unreliability of the model without any trust anchor. > Assuming a proof of location is for a mobile device, rather than a particular human being, then associating the proof of uniqueness obtained under such a condition, i.e., without the involvement of a trust anchor, is unreliable. I know that the authors aren't directly calling for more centralized trust. But given recent development at Google, we all know how the readers would think

MichaelZuo 3y ago
> > Verifying sentience is a fuzzy concept. While they can be bound together momentarily as we see in [66 ], the binding is very easily decoupled.The verified user might choose to sell off their uniqueness identifier at time period 𝑡 + 1 if the verification which binds sentience with uniqueness ends at 𝑡. > Basically, people can sell identities I can see why Sam Altman believes iris scans are the future, it's definitely much more cumbersome to 'sell off' your iris. Especially if it needs to be rescanned on a daily basis or sooner.
- wmf 3y ago
  But Worldcoin isn't doing that. They scan your eyes once and issue you a private key that you can then sell. Maybe one day they can give away scanners CueCat-style but I haven't seen any discussion of that.
  - MichaelZuo 3y ago

version_five 3y ago

> Trolls, bots, and sybils distort online discourse and compromise the security of networked platforms. In some sense I think the authors' hypothesis is a good thing, ie that you can never fully verify someone online. It prevents wholesale algorithmic management of people, which is really what governments and companies would like to do, and forced some level of human contact or at least intervention. I expect it's inevitable that they'll find a way to offload the problem onto the citizen, for the most part they already have, but I'm personally glad it's impossible to assign me some kind of infallible identifier that will let me be The Castle style abused remotely and without recourse.

dcow 3y ago

I think a lot of people try to do identity things without understanding the fundamental nature of the problem they're attacking. For instance: I'm really really worried that governments are going to default into an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document. Or: I'm really annoyed that software services and web apps have clauses like "you can't use scripts or automation software to access our API" when a browser is just that. And they should really be enforcing rate limits and punishing abusive behavior whether the user clicked a button in a browser or a script did. These type of things are not rooted in a fundamental understanding of identity, they're sloppy stop-gaps. Despite all its faults, this is one of the reasons I'm super excited about WebAuthN. At least is make common the idea that an identity is a cryptographic secret and not a "possession of an email and phone". We really really need to dig out of this "email address identifies you" hole. Anyway it's exciting to see people discuss the topic more formally. It gives me hope that we can ultimately get to a better understanding of digital identity and not be trying to solve impossible problems by chasing an impossibly perfect solution that verifies all 3 tenets that actually doesn't exist and making a big mess of things because nobody stopped to ask or understand the scope of what we should be trying to do. Ultimately identity should be empowering not oppressive. And right now it feels more like services oppress people into all sorts of weird requirements like having an email, getting a phone verification code, running software on a device that has an integrity attestation framework, etc. rather than trust them and punish bad behavior. I want my government (and web services, but especially my government) to trust me and punish bad behavior, not treat me like an untrusted bot that needs to be managed and continuously verified.

OfSanguineFire 3y ago
> an understanding of digital identity that involves ownership of an email address and mobile phone rather than ability to sign a document. Buying a SIM card in a great many countries today already requires showing state-approved ID and then signing a form that the shop clerk prints out. So, ownership of a mobile phone number does mean being able to sign a document. Are you concerned about SIM-jacking? I admit, I find the thrust of your post difficult to follow.

Morizero 3y ago

Title should read "sentience, location, and uniqueness", which the paper states are the three key properties of identity

dontupvoteme 3y ago

Half the internet thinks troll means "person who disagrees with me or (dis)likes thing i (dis)like" - I'm distrustful of people who paint them as a big problem on the internet. Is this how the western Social Credit system begins?

pphysch 3y ago
Trolls, as in troll farms, astroturfing, organized influence campaigns, etc. are absolutely a serious problem for any society that pretends to care about democracy. Especially in the LLM era, where the marginal cost of adding another artificial "voice" approaches $0.
- dontupvoteme 3y ago
  That's not what troll means. Astroturfing as you said is much better. They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they?
  - pphysch 3y ago
    If that is not one of the definitions of troll, then can you please explain what a "troll farm" is supposed to be? > They're going to be pushing for WEI/Attestation/Requiring easily doxxable accounts using this a bogeyman, aren't they? It's not a bogeyman. It's a real, current, serious problem.
    - whatshisface 3y ago
      If the troll farms are responsible for online attacks on our civil liberties, why would they use themselves, troll farms, as the justification? I rather think the attacks on our freedom are being made by real people who are misguided about the tradeoffs between letting people speak when they are wrong, and making it easy for officials covering up misconduct.
    - ben0x539 3y ago
      "Troll farm" is also based on this new usage of the word.

jkhdigital 3y ago

It’s all the same problem that Ken Thompson identified in his classic Turing lecture: there is no way to reliably identify Trojans.

_xnmw 3y ago

What about Apple's FaceID? Secure, private, decentralized, and and verifies sentience (well atleast intactness of your physical head), location and identity.

kazinator 3y ago

The location element of the trilemma doesn't seem as important as the other two, and is a violation of privacy. Let it be hard to discover. A social site that discovers everyone's location, and reminds them it knows where they are, would probably drastically cut down on the trolling and abuse, but at what cost.

pontifier 3y ago

The philosophy of identity is quite interesting to me. I've been thinking about it for a long time. I like the idea of proof of personhood, and proof of location. These are both concepts that I have explored extensively. I looked at retina scans, and found it they're both not unique enough, and not secure enough to act as a basis for a currency. My proof of location was based on the 500 mile email story. Many millions of round trips between two devices could allow someone to prove on a blockchain that two secret keys were stored arbitrarily close to one another for some amount of time. The paper mentions key signing parties which I believe must be part of any web of trust that proves personhood. Proving uniqueness is more difficult. There's nothing that would stop me from having several different identities verified at different key signing parties, then using those to generate even more. Using genealogy seems to be the only way to truly verify a unique identity if the system becomes widespread enough that parents often verify their children at birth. Then the age of the identity matches the age of the individual. Farther into the future, the possibility of creating multiple copies of myself makes it much more difficult to tie an identity to a particular group of cells. There's nothing truly unique about a copy, especially if we have the ability to change our DNA. Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary.

magic_hamster 3y ago
> Philosophically, I started to wonder if there's anything particularly special about any arrangement of cells. Maybe we should all just do our best, share resources, and create a society where abundance makes keeping track of all this stuff unnecessary. We are an arrangement of cells betrayed by the human character. If all we wanted was to live leisurely this could have been easily achieved after the industrial revolution. But it's our nature to be ambitious, to achieve things, to want more, to compete, to be greedy and 1up each other. Most people are never satisfied regardless of how big their house is or what they do for a living. So I don't think an idle society of abundance can exist as long as people are competing for the largest piece of the pie.
- 3y ago