3 ms·

Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not,

by fwipsy 1mo ago

Read the rest of my comment please. Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Probably not, since if there's really all that much riding on it, people will point out it can be bypassed. Should we also abolish Pangram, because it's not 100% accurate? Someone might be convinced a text is not AI-generated when it actually is! We should get rid of it rather than fool people into thinking it can be determined accurately. What about antivirus? We should abolish it as well rather than fool people into thinking that their software is ever 100% safe. What about HTTPS? We shouldn't call it "secure" shell because the computer you're connecting to could be compromised! I could go on and on and on. The median instance of AI image generation isn't evidence in a court case. It's cyberbullying, or deepfakes, or fake news. It's called "slop" because there's a lot of it being churned out at low effort.

hypfer 1mo ago
You're missing all of the points that there could be by focussing on random people. While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level. This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.
- fwipsy 1mo ago
  It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors. I like being contrarian as much as the next guy, but "Actually, having security is worse for security" is taking it a little too far.
  - hypfer 1mo ago
    I am repeating myself, but this is about systems, and not about people. It is however in the interest of the people to keep the systems running in an untainted way. As said, on the individual level it's a tragedy, but one that can be absorbed somewhat. Democracy itself failing otoh is kinda hard to absorb. C2PA is not "having security". It is "having an illusion of security for compliance and CYA reasons, that can be fairly trivially exploited by nation state actors". Banality of evil. Again. ___ Actually, come to think of it, "security" is the wrong term there. Signatures don't secure anything. They attest. Those are different things. Argh and I ran with your term aah
    - fwipsy 1mo ago
      This is pascal's mugging. Democracy itself might fail! You're just inflating the stakes of your hypothetical bad outcome until it can overwhelm any positive upside. Not to mention, democracy is under just as much or more threat from "banal" fake news created by citizens. People gonna people. They don't need the DPRK lying to them to fool themselves.
  - Melatonic 1mo ago
    Yeah I think any additional security is good. At worst this could help in a lot of court cases. Someone presents photo evidence - it could be manipulated - it could be not. This happens already. Then someone produces an original higher quality version (like a raw photo - which I take even on my phone at all times now) and experts can verify that as the original. And I agree on state actors. If a major one is invested in something like this they might have well compromised the signing project itself, the verification process, or even the court system or media. That seems like a rare and extremely high bar to guard against.
    - fwipsy 1mo ago
      Exactly. It's just more information. It doesn't have to be 100% accurate in every case, to be useful.
Gormo 1mo ago
> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with. This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.
- anonreplier 1mo ago
  Why is this being framed as 2 types of users, lovable pranksters and fraudsters? There's a whole spectrum between these 2. Also I'd like to know if a "joke" is likely fake.
  - Gormo 1mo ago
    I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior. Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use. In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.
    - fwipsy 1mo ago
      Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally. Your argument applies to any imperfect security technology -- aka practically all of them.
treyd 1mo ago
You're conflating the effectiveness of the mechanism with its systemic impact. * Pangram: Yes we should really be discouraging people from putting trust in tools like this because they can't be made totally reliable. * Antivirus: We should be building application environments with robust security models so that malicious software has a limited blast radius (like we do on mobile, like the Linux ecosystem is trying to do with Flatpak, etc). * HTTPS: HTTPS is a strict upgrade from HTTP so we should be using it everywhere possible. The UI symbols to indicate to users the security expectations they're getting are good practice. * ssh: This is just an inappropriate comparison. The HTTPS comparison would make more sense if actually 0.1% of the time when their browser said they were using HTTPS it was just lying.
- fwipsy 1mo ago
  Pangram: I'm not arguing against encouraging skepticism; I'm arguing that the technology is not useless. It's good for people to know the limitations, but it's still evidence. Antivirus: "Actually, we should build this hypothetical better thing" is a cop-out. HTTPS: C2PA is a strict upgrade from unsigned photographs, so it should be used wherever possible. SSH: Totally appropriate, the entire point of the discussion is whether it's permissible to the user that they might be more secure.

by fwipsy 1mo ago

hypfer 1mo ago

You're missing all of the points that there could be by focussing on random people. While it is always an individual tragedy when people treat each other badly (e.g. through deepfakes and all), the real threat does not exist on that level. This is about misinformation and disinformation, so we're talking state actors. And with that, the 99.9% hypothesis does not hold true.

fwipsy 1mo ago
It's funny how people always say something is "a tragedy at the individual level" when they mean "it's not my problem." It's even crazier to dismiss the value of a security feature, just because it might make people feel more secure. That's true of every security feature! Very little of the technology that the web is built on is proof against state actors. I like being contrarian as much as the next guy, but "Actually, having security is worse for security" is taking it a little too far.
- hypfer 1mo ago
  I am repeating myself, but this is about systems, and not about people. It is however in the interest of the people to keep the systems running in an untainted way. As said, on the individual level it's a tragedy, but one that can be absorbed somewhat. Democracy itself failing otoh is kinda hard to absorb. C2PA is not "having security". It is "having an illusion of security for compliance and CYA reasons, that can be fairly trivially exploited by nation state actors". Banality of evil. Again. ___ Actually, come to think of it, "security" is the wrong term there. Signatures don't secure anything. They attest. Those are different things. Argh and I ran with your term aah
  - fwipsy 1mo ago
    This is pascal's mugging. Democracy itself might fail! You're just inflating the stakes of your hypothetical bad outcome until it can overwhelm any positive upside. Not to mention, democracy is under just as much or more threat from "banal" fake news created by citizens. People gonna people. They don't need the DPRK lying to them to fool themselves.
- Melatonic 1mo ago
  Yeah I think any additional security is good. At worst this could help in a lot of court cases. Someone presents photo evidence - it could be manipulated - it could be not. This happens already. Then someone produces an original higher quality version (like a raw photo - which I take even on my phone at all times now) and experts can verify that as the original. And I agree on state actors. If a major one is invested in something like this they might have well compromised the signing project itself, the verification process, or even the court system or media. That seems like a rare and extremely high bar to guard against.
  - fwipsy 1mo ago
    Exactly. It's just more information. It doesn't have to be 100% accurate in every case, to be useful.

Gormo 1mo ago

> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with. This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.

anonreplier 1mo ago
Why is this being framed as 2 types of users, lovable pranksters and fraudsters? There's a whole spectrum between these 2. Also I'd like to know if a "joke" is likely fake.
- Gormo 1mo ago
  I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior. Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use. In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.
  - fwipsy 1mo ago
    Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally. Your argument applies to any imperfect security technology -- aka practically all of them.

treyd 1mo ago

You're conflating the effectiveness of the mechanism with its systemic impact. * Pangram: Yes we should really be discouraging people from putting trust in tools like this because they can't be made totally reliable. * Antivirus: We should be building application environments with robust security models so that malicious software has a limited blast radius (like we do on mobile, like the Linux ecosystem is trying to do with Flatpak, etc). * HTTPS: HTTPS is a strict upgrade from HTTP so we should be using it everywhere possible. The UI symbols to indicate to users the security expectations they're getting are good practice. * ssh: This is just an inappropriate comparison. The HTTPS comparison would make more sense if actually 0.1% of the time when their browser said they were using HTTPS it was just lying.

fwipsy 1mo ago
Pangram: I'm not arguing against encouraging skepticism; I'm arguing that the technology is not useless. It's good for people to know the limitations, but it's still evidence. Antivirus: "Actually, we should build this hypothetical better thing" is a cop-out. HTTPS: C2PA is a strict upgrade from unsigned photographs, so it should be used wherever possible. SSH: Totally appropriate, the entire point of the discussion is whether it's permissible to the user that they might be more secure.