5 ms·

The problem here is authenticating that you are talking to a human and not an AI. Certificates don't help with that. Criminals will register temporary busine

The problem here is authenticating that you are talking to a human and not an AI. Certificates don't help with that. Criminals will register temporary businesses and obtain certificates for them with no problem.

ericd 4y ago
But those certs can come with reputations attached, and it prevents people from claiming that they're representatives of well known companies.
- MattPalmer1086 4y ago
  Well, that would create strong financial incentives to compromise company certificates, which I'm sure would then happen. And you still don't know if your taking to a human or an AI.
  - lukevp 4y ago
    Like the strong financial incentives to compromise certificates owned by banking websites? Securing voice communications the same as website communications is an awesome idea, and the fact that it’s possible to steal some piece of data and compromise it shouldn’t prevent us from moving in that direction.
    - MattPalmer1086 4y ago
      It has happened, but there has to be a big pay off to make it worthwhile. Smaller companies would likely be more frequent targets given they would be easier to compromise. By the way, I'm not suggesting that the idea is useless. I'm just pointing out it isn't a panacea, and it still doesn't address the core problem raised in the article that you don't know if you are speaking to a human or not.
      - XorNot 4y ago
        But the point is that the pay off itself is limited. Smaller companies have a much smaller pool of people who would have any reason to accept unsolicited calls. This is evident in how scams work today: you either get explicitly targeted, or you dragnet using some service which everyone has i.e. a bank, the tax office, or a telecom company. Compromising "Joe's BBQ Emporium" might be easier, but it's still (1) something you have to do (and maybe get caught then) and (2) the number of people who are going to pick up, or not immediately blacklist unsolicited calls from "Joe's BBQ Emporium" is tiny.
  - ericd 4y ago
    Does it matter if you’re talking to a human or an AI? The main thing is its intent - is it helpful, or malicious? And attaching communications to persistent reputations via certs can help guess at that.
    - MattPalmer1086 4y ago
      It matters because an AI can contact many more potential victims that a human can. It makes the cost of attack much lower.
- pixl97 4y ago
  Where does the reputation come from? Is it going to work like our already questionable anti-spam systems?
  - XorNot 4y ago
    You don't need reputation systems: I can count on two hands the number of organizations that have any reason to be contacting me unsolicited. Add a minor tie-in with my bank, and my reputation list would essentially be "the government, my bank, my doctor, my insurer + any company I've done monetary transactions with recently". And realistically in a world with this system, this is all already implemented by my phone's contact list - no more phone numbers would mean that there's no reason to think that any organization would be contacting me from a totally unique (or anonymous) identity. Instead I'd just have a contact book whitelist entry for "bank.com.au" or whatever scheme we ended up with. As it is right now, actual government services call people from Caller ID blocked numbers, and don't widely publish allowable contact origins. Which is ridiculous.
vimax 4y ago
Why does it matter if you’re talking to an AI or a human? What can a criminal AI say that a criminal human cannot?
- MattPalmer1086 4y ago
  You can automate it and reach many more victims than doing it with humans. Edit: it reduces the cost of the attack and thus makes it more profitable.
- JeremyBanks 4y ago
  [dead]
- mrec 4y ago
  A criminal human probably can't do a perfect voice impersonation of your teenage kid or frail grandmother who desperately needs to be sent money for some urgent reason.
  - vimax 4y ago
    Sure, but that’s not too different from the current scams where there’s some excuse you can’t talk right now. In both cases it can be solved by checking if it is coming from a known number and calling the person in question to confirm. People you know typically won’t desperately need lots of money with some elaborate story that is impossible to be immediately confirmed. If AI suddenly leads to an increase in call spoofing, that’d be a problem of the phone network that we already face with robocalls, but wouldn’t be new.
- stri8ed 4y ago
  It's the cost factor. Using AI, it's economically feasible to contact a million people, even if only a fraction bite. Not unlike the current spam calls.
AnthonyMouse 4y ago
"Authenticating that you are talking to a human and not an AI" is just a proxy for something else, most commonly wanting to be able to rate limit something. Because otherwise, why does it matter? A lot of people are suggesting things like, have the government do it. But that has two big problems. First is privacy. If you have to prove your identity any time you want to do anything, nobody can be anonymous anymore, which is Very Bad. Second, it assumes the government has some magic incantation that nobody else can use, as if the Post Office knows who you are in a way that your bank doesn't. But they don't. To get a government ID, they just want you to show them some other existing ID. It has no way to bootstrap itself any better than anything else. And some of the IDs they accept are easy to get... without an ID. Because everybody has to start from somewhere. The system has to be set up in a way that it works for people who emigrate from a country with untrustworthy institutions as an adult or if your house burns down and you lose all your documents you can still get new ones. An AI is going to be able to BS its way into a government ID, even assuming criminals wouldn't be able to hack into any state's DMV (as they already have). It appears that going forward, telling the difference between a human and an AI is going to be hard. Maybe instead of trying to get better at that, we should find a different solution to the underlying problem. The simple answer is to make account creation cost something. Nothing big, so someone who needs one account isn't paying much, but a spammer who has 1000 accounts get banned every day is out of business. And that's not even hard -- it's finally something cryptocurrency would actually be good for. Because you want a way for people to pay for access to things, while still being anonymous. The real hard part is, how do you charge for account creation without deterring account creation?
- XorNot 4y ago
  > If you have to prove your identity any time you want to do anything, nobody can be anonymous anymore, which is Very Bad. No one would have to prove anything. But why would I accept calls from any entity claiming to be a legally registered business which doesn't present a government issued certificate proving that? I already verify businesses by looking up business number registries. But this should be automatic to the communication even happening. Same question with personal contacts: why would I blindly accept calls from people I don't know and who don't want to present any confirmation of their identity to me?

ericd 4y ago

But those certs can come with reputations attached, and it prevents people from claiming that they're representatives of well known companies.

MattPalmer1086 4y ago
Well, that would create strong financial incentives to compromise company certificates, which I'm sure would then happen. And you still don't know if your taking to a human or an AI.
- lukevp 4y ago
  Like the strong financial incentives to compromise certificates owned by banking websites? Securing voice communications the same as website communications is an awesome idea, and the fact that it’s possible to steal some piece of data and compromise it shouldn’t prevent us from moving in that direction.
  - MattPalmer1086 4y ago
    It has happened, but there has to be a big pay off to make it worthwhile. Smaller companies would likely be more frequent targets given they would be easier to compromise. By the way, I'm not suggesting that the idea is useless. I'm just pointing out it isn't a panacea, and it still doesn't address the core problem raised in the article that you don't know if you are speaking to a human or not.
    - XorNot 4y ago
      But the point is that the pay off itself is limited. Smaller companies have a much smaller pool of people who would have any reason to accept unsolicited calls. This is evident in how scams work today: you either get explicitly targeted, or you dragnet using some service which everyone has i.e. a bank, the tax office, or a telecom company. Compromising "Joe's BBQ Emporium" might be easier, but it's still (1) something you have to do (and maybe get caught then) and (2) the number of people who are going to pick up, or not immediately blacklist unsolicited calls from "Joe's BBQ Emporium" is tiny.
- ericd 4y ago
  Does it matter if you’re talking to a human or an AI? The main thing is its intent - is it helpful, or malicious? And attaching communications to persistent reputations via certs can help guess at that.
  - MattPalmer1086 4y ago
    It matters because an AI can contact many more potential victims that a human can. It makes the cost of attack much lower.
pixl97 4y ago
Where does the reputation come from? Is it going to work like our already questionable anti-spam systems?
- XorNot 4y ago
  You don't need reputation systems: I can count on two hands the number of organizations that have any reason to be contacting me unsolicited. Add a minor tie-in with my bank, and my reputation list would essentially be "the government, my bank, my doctor, my insurer + any company I've done monetary transactions with recently". And realistically in a world with this system, this is all already implemented by my phone's contact list - no more phone numbers would mean that there's no reason to think that any organization would be contacting me from a totally unique (or anonymous) identity. Instead I'd just have a contact book whitelist entry for "bank.com.au" or whatever scheme we ended up with. As it is right now, actual government services call people from Caller ID blocked numbers, and don't widely publish allowable contact origins. Which is ridiculous.

vimax 4y ago

Why does it matter if you’re talking to an AI or a human? What can a criminal AI say that a criminal human cannot?

MattPalmer1086 4y ago
You can automate it and reach many more victims than doing it with humans. Edit: it reduces the cost of the attack and thus makes it more profitable.
JeremyBanks 4y ago
[dead]
mrec 4y ago
A criminal human probably can't do a perfect voice impersonation of your teenage kid or frail grandmother who desperately needs to be sent money for some urgent reason.
- vimax 4y ago
  Sure, but that’s not too different from the current scams where there’s some excuse you can’t talk right now. In both cases it can be solved by checking if it is coming from a known number and calling the person in question to confirm. People you know typically won’t desperately need lots of money with some elaborate story that is impossible to be immediately confirmed. If AI suddenly leads to an increase in call spoofing, that’d be a problem of the phone network that we already face with robocalls, but wouldn’t be new.
stri8ed 4y ago
It's the cost factor. Using AI, it's economically feasible to contact a million people, even if only a fraction bite. Not unlike the current spam calls.

AnthonyMouse 4y ago

"Authenticating that you are talking to a human and not an AI" is just a proxy for something else, most commonly wanting to be able to rate limit something. Because otherwise, why does it matter? A lot of people are suggesting things like, have the government do it. But that has two big problems. First is privacy. If you have to prove your identity any time you want to do anything, nobody can be anonymous anymore, which is Very Bad. Second, it assumes the government has some magic incantation that nobody else can use, as if the Post Office knows who you are in a way that your bank doesn't. But they don't. To get a government ID, they just want you to show them some other existing ID. It has no way to bootstrap itself any better than anything else. And some of the IDs they accept are easy to get... without an ID. Because everybody has to start from somewhere. The system has to be set up in a way that it works for people who emigrate from a country with untrustworthy institutions as an adult or if your house burns down and you lose all your documents you can still get new ones. An AI is going to be able to BS its way into a government ID, even assuming criminals wouldn't be able to hack into any state's DMV (as they already have). It appears that going forward, telling the difference between a human and an AI is going to be hard. Maybe instead of trying to get better at that, we should find a different solution to the underlying problem. The simple answer is to make account creation cost something. Nothing big, so someone who needs one account isn't paying much, but a spammer who has 1000 accounts get banned every day is out of business. And that's not even hard -- it's finally something cryptocurrency would actually be good for. Because you want a way for people to pay for access to things, while still being anonymous. The real hard part is, how do you charge for account creation without deterring account creation?

XorNot 4y ago
> If you have to prove your identity any time you want to do anything, nobody can be anonymous anymore, which is Very Bad. No one would have to prove anything. But why would I accept calls from any entity claiming to be a legally registered business which doesn't present a government issued certificate proving that? I already verify businesses by looking up business number registries. But this should be automatic to the communication even happening. Same question with personal contacts: why would I blindly accept calls from people I don't know and who don't want to present any confirmation of their identity to me?