4 ms·

It's a good question, and unfortunately you're probably right. It boils down to this: If you can prevent [some organisation] from potentially destroying civili

It's a good question, and unfortunately you're probably right. It boils down to this: If you can prevent [some organisation] from potentially destroying civilisation, how much effort would be too much effort, and how much uncertainty is too much uncertainty? For most, there's a trade off. If someone believes that the technology is sufficient for any country to implement a brutal civil rights destruction campaign, and that this is 50% likely, and all you need to do is upload some harmless images to your icloud to thwart it, why wouldn't you? For example, maybe a certain political party could regain power in a massive way and start doing away with gay rights, locking up anyone with pictures of men kissing each other on their phones. Of course, other countries have other types of extremists that look like they might take over government, or have already taken over governments. In those countries, tools like this are already in place, and this new one could be very powerful. So if you could upload some innocuous images and save hundreds of lives in 20 or more countries in the world, even if you don't fear for your own country, would you? Assuming you said yes, the question is, could everyone who agrees give Apple enough false positives, force enough human moderators to have to inspect the images, that the whole scheme becomes financially un-viable on Apple's end? Of course, the only thing we could do here is slow this "progress" down. Maybe we can use that extra time to share the message that "this kind of technology is not ok" and that being naive about what this tech will be used for is almost guaranteed to kill more people than it saves. But while so few people are thinking of it in these terms, or if everyone believes the attempt to be futile, it can't work. It's like anything - by the time people realise there's a problem, it's often too late to fix the problem.

ThinBold 5y ago
save hundreds of lives, would you? Sure, why not. But let me ask some questions, because at this point I am not sure if people want Apple's system to be robust or jammable. If our fear is that Apple will tune the system to detect pictures of two men kissing, wouldn't an easily jammable system works in our favor because we can DDoS it or threaten to do so anytime we want?
- undecisive 5y ago
  It's a tricky one, certainly. There is probably a graph somewhere showing how much effort to fix is too much effort, vs how much bad will in the community this project is inducing, vs how much value this project has. So no, Apple's system being jammable is a great "booby prize" right up to the point where Apple fix the algorithms, or the chinese government start reporting these false positives as bugs and saying that Apple must fix the bugs before their devices can be sold in China. And so, one has to assume that if the algorithms can be fixed, they will be fixed. If we can DDoS Apple's human checking capability while it's still young, we might be able to prevent more resource being sunk into it - though I agree, that's unlikely. If we can do all that AND make it clear that this is going to cause nothing but bad will, and if we can get enough governments to regulate against it, then there may be hope. So your question was, why do people think this software isn't going to work, and the answer is we really hope it isn't going to work and really hope that enough people get onboard with the efforts to subvert it and really hope that the message sinks in to Apple that this was a doomed project that they should abandon for good. But with enough money and time, there is no problem that can't be overcome. So even if it is a currently jammable child protection system, in the future is will almost definitely become a robust human rights violation system. In the space between those two points there is hope. Slim hope, but hope nonetheless.

ThinBold 5y ago

save hundreds of lives, would you? Sure, why not. But let me ask some questions, because at this point I am not sure if people want Apple's system to be robust or jammable. If our fear is that Apple will tune the system to detect pictures of two men kissing, wouldn't an easily jammable system works in our favor because we can DDoS it or threaten to do so anytime we want?

undecisive 5y ago
It's a tricky one, certainly. There is probably a graph somewhere showing how much effort to fix is too much effort, vs how much bad will in the community this project is inducing, vs how much value this project has. So no, Apple's system being jammable is a great "booby prize" right up to the point where Apple fix the algorithms, or the chinese government start reporting these false positives as bugs and saying that Apple must fix the bugs before their devices can be sold in China. And so, one has to assume that if the algorithms can be fixed, they will be fixed. If we can DDoS Apple's human checking capability while it's still young, we might be able to prevent more resource being sunk into it - though I agree, that's unlikely. If we can do all that AND make it clear that this is going to cause nothing but bad will, and if we can get enough governments to regulate against it, then there may be hope. So your question was, why do people think this software isn't going to work, and the answer is we really hope it isn't going to work and really hope that enough people get onboard with the efforts to subvert it and really hope that the message sinks in to Apple that this was a doomed project that they should abandon for good. But with enough money and time, there is no problem that can't be overcome. So even if it is a currently jammable child protection system, in the future is will almost definitely become a robust human rights violation system. In the space between those two points there is hope. Slim hope, but hope nonetheless.