4 ms·

It would be better if the false positive rate was known to be nonzero. And what, in this context, is "your data?" Is a photo on an iPhone Apple's data? I'd ar

It would be better if the false positive rate was known to be nonzero. And what, in this context, is "your data?" Is a photo on an iPhone Apple's data? I'd argue that the thing you should do if you get a hit is to delete your data, unless you want to spend enormous amounts of time and money defending yourself from prosecution by people who believe that false positives are vanishingly rare.

Spivak 5y ago
I'm talking from the perspective of a service-owner who handles user-generated content. You typically set up PhotoDNA on two parts of your infrastructure. You scan client-side and refuse uploads to prevent the images from getting on your servers in the first place and then you scan on the backend to catch anything that slipped through. You do the first part so you don't have to call the FBI. I really don't care about the semantics of data ownership. If Apple wants to scan photos you upload to iCloud so they don't run into a scandal years down the road that "iCloud is being used to distribute CP" then that's their call. And to the letter of the law you can't just delete the data if you found CP that one of your customers uploaded without running afoul of mandatory reporting laws.
- noxer 5y ago
  In the real world the system delete the data, deletes the associated user, bans the user and no human will do anything about it. Most likely there isn't even a report made to a human it just goes under ToS violation or something so no one inside the companies aks questions. No FBI is called. Imagine the financial damage this would do if specialist come over and try to collect evidence and the company is forced to cooperated and work for them, for free ofc. Only the big tech does this. They have whole teams for this and automated system and what not because surely the FBI wont walk into googled data center ever to collect evidence. But all the smaller companies they cant afford to do this and the detection system is actually to protect them form going bankrupt because they accidentally stored some illegal data from a user. >And to the letter of the law you can't just delete the data if you found CP that one of your customers uploaded without running afoul of mandatory reporting laws. Yes such laws exist. So they use filter software from third parties. They intentionally offer some generic filter that includes "unwanted content" and doesn't specify what it is when it find something so the companies can wave away the liability as they have no way to know if the image was valuable evidence of a crime or just some LiveLeak gore. They can justify that no human watches it because that's to expensive.

Spivak 5y ago

I'm talking from the perspective of a service-owner who handles user-generated content. You typically set up PhotoDNA on two parts of your infrastructure. You scan client-side and refuse uploads to prevent the images from getting on your servers in the first place and then you scan on the backend to catch anything that slipped through. You do the first part so you don't have to call the FBI. I really don't care about the semantics of data ownership. If Apple wants to scan photos you upload to iCloud so they don't run into a scandal years down the road that "iCloud is being used to distribute CP" then that's their call. And to the letter of the law you can't just delete the data if you found CP that one of your customers uploaded without running afoul of mandatory reporting laws.

noxer 5y ago
In the real world the system delete the data, deletes the associated user, bans the user and no human will do anything about it. Most likely there isn't even a report made to a human it just goes under ToS violation or something so no one inside the companies aks questions. No FBI is called. Imagine the financial damage this would do if specialist come over and try to collect evidence and the company is forced to cooperated and work for them, for free ofc. Only the big tech does this. They have whole teams for this and automated system and what not because surely the FBI wont walk into googled data center ever to collect evidence. But all the smaller companies they cant afford to do this and the detection system is actually to protect them form going bankrupt because they accidentally stored some illegal data from a user. >And to the letter of the law you can't just delete the data if you found CP that one of your customers uploaded without running afoul of mandatory reporting laws. Yes such laws exist. So they use filter software from third parties. They intentionally offer some generic filter that includes "unwanted content" and doesn't specify what it is when it find something so the companies can wave away the liability as they have no way to know if the image was valuable evidence of a crime or just some LiveLeak gore. They can justify that no human watches it because that's to expensive.