3 ms·

> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Yes, absolutely. Probably moreso. The whole p

by Gormo 1mo ago

> Is the single motivated malicious user able to do as much damage as all of the blocked attempts put together? Yes, absolutely. Probably moreso. The whole point of these proposals is to try to solve for the "motivated malicious user" who is engaging in actual high-stakes fraud. There is no point in applying techniques that suppress inconsequential pranks while making serious crimes easier to get away with. This really seems like a rehash of the perennial DRM argument: DRM restrictions provably do not reduce large-scale motivated copyright infringement, they just annoy legitimate paying users. This is the same class of solution, in that it is effective only where the stakes are low and the impact is minimal.

anonreplier 1mo ago
Why is this being framed as 2 types of users, lovable pranksters and fraudsters? There's a whole spectrum between these 2. Also I'd like to know if a "joke" is likely fake.
- Gormo 1mo ago
  I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior. Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use. In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.
  - fwipsy 1mo ago
    Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally. Your argument applies to any imperfect security technology -- aka practically all of them.
    - Gormo 1mo ago
      > Your idea of a criminal seems to be hypercompetent and think of everything. No, my idea of a criminal is someone who is motivated to commit crime, and I feel that we've already established in this thread that the approaches we're discussing are motivation gates far more than competence gates. > Smart, dedicated, technical people can typically make more money legally. Then who's been running all the botnets, writing cryptolocker malware, and running phishing scams for the past couple of decades? We've always had script kiddies, and now we have people using AI itself to do malicious things. Technical skill has never been an obstacle for sufficiently motivated scammers. > Your argument applies to any imperfect security technology -- aka practically all of them. Ultimately, everything has weaknesses, and with enough effort, most measures can be circumvented. But how much effort is enough varies wildly between solutions. There's a huge gulf between a "no trespassing" sign, on the one hand, and a concrete wall topped with barbed wire, on the other. The "no trespassing" sign only keeps out people willing to obey it; the concrete wall keeps out anyone who isn't willing and able to accept the time, effort, and risk necessary to climb over it or knock it down. And the point is that using digital signatures to distinguish AI-generated media from hand-made media is much closer to the "no trespassing" side of things than it is to the wall. Maybe it's analogous to a gate with a latch you can open from the other side if you reach over in just the right spot.
      - anonreplier 1mo ago
        Your no trespassing and concrete wall analogy again indicates the black and white thinking here.
        Gormo 1mo ago
        I'm afraid it doesn't. To the contrary, treating effective security in terms of how it influences the attacker's cost-benefit tradeoffs, and evaluating proposed measures on whether the effort threshold they create is enough, is quite literally the opposite of black-and-white thinking. And in this case we can clearly see that a solution that (a) does not substantially increase their costs -- and in fact, as I've pointed out above, only really filters by motivation, not by time, money or effort, and (b) doesn't target their potential benefits at all, is one that isn't likely to be effective.

by Gormo 1mo ago

anonreplier 1mo ago

Why is this being framed as 2 types of users, lovable pranksters and fraudsters? There's a whole spectrum between these 2. Also I'd like to know if a "joke" is likely fake.

Gormo 1mo ago
I don't suspect there is a uniform spectrum between those two. I think this is something that's going to be clinal, which we see in a lot of other comparable social contexts. The number of people actually willing to cross a moral threshold into outright crime is relatively small, but those are precisely the people who cause the most damage when they get away with their behavior. Bur I don't even really think that's really relevant anyway, because whatever the density of "malicious" motivations is, the point here is that the fact that it only is an effort/motivation threshold that allows this technique to "block" malicious uses, and the motivation to overcome that threshold correlates directly with the stakes involved in the malicious use. In other words, the more malicious the abuse is, the less effective this solution will be: the boundary of its usefulness will be wherever the line between pranksters and actual criminals happens to lie.
- fwipsy 1mo ago
  Your idea of a criminal seems to be hypercompetent and think of everything. These do exist, but most criminals are not very smart. Smart, dedicated, technical people can typically make more money legally. Your argument applies to any imperfect security technology -- aka practically all of them.
  - Gormo 1mo ago
    > Your idea of a criminal seems to be hypercompetent and think of everything. No, my idea of a criminal is someone who is motivated to commit crime, and I feel that we've already established in this thread that the approaches we're discussing are motivation gates far more than competence gates. > Smart, dedicated, technical people can typically make more money legally. Then who's been running all the botnets, writing cryptolocker malware, and running phishing scams for the past couple of decades? We've always had script kiddies, and now we have people using AI itself to do malicious things. Technical skill has never been an obstacle for sufficiently motivated scammers. > Your argument applies to any imperfect security technology -- aka practically all of them. Ultimately, everything has weaknesses, and with enough effort, most measures can be circumvented. But how much effort is enough varies wildly between solutions. There's a huge gulf between a "no trespassing" sign, on the one hand, and a concrete wall topped with barbed wire, on the other. The "no trespassing" sign only keeps out people willing to obey it; the concrete wall keeps out anyone who isn't willing and able to accept the time, effort, and risk necessary to climb over it or knock it down. And the point is that using digital signatures to distinguish AI-generated media from hand-made media is much closer to the "no trespassing" side of things than it is to the wall. Maybe it's analogous to a gate with a latch you can open from the other side if you reach over in just the right spot.
    - anonreplier 1mo ago
      Your no trespassing and concrete wall analogy again indicates the black and white thinking here.
      - Gormo 1mo ago
        I'm afraid it doesn't. To the contrary, treating effective security in terms of how it influences the attacker's cost-benefit tradeoffs, and evaluating proposed measures on whether the effort threshold they create is enough, is quite literally the opposite of black-and-white thinking. And in this case we can clearly see that a solution that (a) does not substantially increase their costs -- and in fact, as I've pointed out above, only really filters by motivation, not by time, money or effort, and (b) doesn't target their potential benefits at all, is one that isn't likely to be effective.