7 ms·

We have yet to have a real discussion about what we expect the largest platforms to do about content like this. The nature of online content is that once it's

We have yet to have a real discussion about what we expect the largest platforms to do about content like this. The nature of online content is that once it's created it can be copied quite easily, if Facebook was able to block 80% of this content that should be commended. AI is not at the point where we can expect 100% of this content to be removed and if it was there would likely be some collateral damage that others would complain about. At it's current state we need a lot more humans screening this type of content and who are we to subject other people to that type of mental disturbance?

ben_w 8y ago
20% not being caught is surprisingly high to me. I would have assumed the majority of the upload attempts would be byte-for-byte identical, and that even if there were a dozen minor variations of the same content, they would be able to deal with >95% automatically.
- fizx 8y ago
  1.5 million videos uploaded implies the involvement of at least persistent script-kiddies, if not possibly state-level actors getting involved for propaganda. This seems likely to be an arms race.
  - ben_w 8y ago
    1.5 million uploads represents 0.1% of Facebook users making a single attempt each. While I expect some script efforts in general, I feel it’s an unnecessary complication in this case.
- chillacy 8y ago
  Are those numbers from data or are they just your gut feeling?
- hombre_fatal 8y ago
  80% of the videos probably did match a simple sha1 lookup and were trivially removed. Now what?
  - discreteevent 8y ago
    It's not an easy problem to solve if you want a system with an instant response for the poster. But if I am a journalist and I want to get something published it will take at least a few hours to get it through an editor of some kind. FB could do this even just for video. When a user pushes something up it takes a few hours to be published. It would cost FB because some users wouldn't use it without that instant hit.
  - PostOnce 8y ago
    Some kind of perceptual hashing and whatever additional magic YouTube uses to detect copyrighted stuff? Once a video is flagged, it can be added to the perceptual database in a fraction of a second and all future uploads checked against it, YouTube seems to have gotten good at it.
- mic47 8y ago
  Not really. There are 2 factors that play "against" FB having nice numbers. It takes some time between unique (or in some similarity cluster or whatever they have) content is uploaded, reported and taken down. During this time, exactly same content is not being taken down, because it's not known to be bad. So even if all the content was byte to byte identical, if 20% of it is uploaded before it is reported, there is no chance to have >95% block rate automatically. Once content is known to be bad, people (and bad actors) will usually stop uploading it (because it no longer works). Also people stop sharing it, because it's no longer possible (and FB delete all instances of such video from the platform anyways). If there is lame script kiddie that does not handle errors, then the number could be 95%, because it would continue with upload attempts. But normal people, or sophisticated actors will respond to errors and stop uploading exactly that video. 1 extreme example: with 1 sophisticated bad actor uploading exactly same video (through some fake/hacked accounts/pages, or whatever), the block rate will always look bad for Facebook: bad actor uploads N videos before being blocked. Once blocked (on video N+1), bad actor will stop. The block rate will be here 1/(N+1), which will be way less 80% (FB's current case) for any N > 0. What did I try to say? Percentage of what is being blocked does not make sense, since it also depend on how fast is attacker detecting that is being blocked. If fast, the number will always look bad. If attacker does not give a shit, the number will look awesome.
  - 43920 8y ago
    It seems like you could fix this (at least for exactly identical content) by generating a fingerprint for each file as it is uploaded. If a file with a given fingerprint is marked as bad, then all existing files with the same fingerprint would be removed (or maybe just recently-uploaded ones, in order to avoid removing too many legitimate files). That being said, it would probably be pretty simple to work around this by varying the content slightly on each upload.
    - mic47 8y ago
      What I said already assumes that you delete the all instances of that content when you find out it's bad. FB already does this. Problem is, that people see content before it is taken down. And once it's taken down, bad actors stop uploading it, and slightly change it (enough to make it pass existing upload filter).
- zxcb1 8y ago
  Face detection? It's Facebook after all
bitxbit 8y ago
80% of the content being blocked is effectively 0% of it being blocked. That’s not something to be commended. And you’re right. AI is nowhere near reliable yet companies such as FB rely on them.
thirdsun 8y ago
This discussion is also releveant for the upload filters the european union deems necessary. If Facebook and Google (ContentID) can't do it reliably, who can?

ben_w 8y ago

20% not being caught is surprisingly high to me. I would have assumed the majority of the upload attempts would be byte-for-byte identical, and that even if there were a dozen minor variations of the same content, they would be able to deal with >95% automatically.

fizx 8y ago
1.5 million videos uploaded implies the involvement of at least persistent script-kiddies, if not possibly state-level actors getting involved for propaganda. This seems likely to be an arms race.
- ben_w 8y ago
  1.5 million uploads represents 0.1% of Facebook users making a single attempt each. While I expect some script efforts in general, I feel it’s an unnecessary complication in this case.
chillacy 8y ago
Are those numbers from data or are they just your gut feeling?
hombre_fatal 8y ago
80% of the videos probably did match a simple sha1 lookup and were trivially removed. Now what?
- discreteevent 8y ago
  It's not an easy problem to solve if you want a system with an instant response for the poster. But if I am a journalist and I want to get something published it will take at least a few hours to get it through an editor of some kind. FB could do this even just for video. When a user pushes something up it takes a few hours to be published. It would cost FB because some users wouldn't use it without that instant hit.
- PostOnce 8y ago
  Some kind of perceptual hashing and whatever additional magic YouTube uses to detect copyrighted stuff? Once a video is flagged, it can be added to the perceptual database in a fraction of a second and all future uploads checked against it, YouTube seems to have gotten good at it.
mic47 8y ago
Not really. There are 2 factors that play "against" FB having nice numbers. It takes some time between unique (or in some similarity cluster or whatever they have) content is uploaded, reported and taken down. During this time, exactly same content is not being taken down, because it's not known to be bad. So even if all the content was byte to byte identical, if 20% of it is uploaded before it is reported, there is no chance to have >95% block rate automatically. Once content is known to be bad, people (and bad actors) will usually stop uploading it (because it no longer works). Also people stop sharing it, because it's no longer possible (and FB delete all instances of such video from the platform anyways). If there is lame script kiddie that does not handle errors, then the number could be 95%, because it would continue with upload attempts. But normal people, or sophisticated actors will respond to errors and stop uploading exactly that video. 1 extreme example: with 1 sophisticated bad actor uploading exactly same video (through some fake/hacked accounts/pages, or whatever), the block rate will always look bad for Facebook: bad actor uploads N videos before being blocked. Once blocked (on video N+1), bad actor will stop. The block rate will be here 1/(N+1), which will be way less 80% (FB's current case) for any N > 0. What did I try to say? Percentage of what is being blocked does not make sense, since it also depend on how fast is attacker detecting that is being blocked. If fast, the number will always look bad. If attacker does not give a shit, the number will look awesome.
- 43920 8y ago
  It seems like you could fix this (at least for exactly identical content) by generating a fingerprint for each file as it is uploaded. If a file with a given fingerprint is marked as bad, then all existing files with the same fingerprint would be removed (or maybe just recently-uploaded ones, in order to avoid removing too many legitimate files). That being said, it would probably be pretty simple to work around this by varying the content slightly on each upload.
  - mic47 8y ago
    What I said already assumes that you delete the all instances of that content when you find out it's bad. FB already does this. Problem is, that people see content before it is taken down. And once it's taken down, bad actors stop uploading it, and slightly change it (enough to make it pass existing upload filter).
zxcb1 8y ago
Face detection? It's Facebook after all

bitxbit 8y ago

80% of the content being blocked is effectively 0% of it being blocked. That’s not something to be commended. And you’re right. AI is nowhere near reliable yet companies such as FB rely on them.

thirdsun 8y ago

This discussion is also releveant for the upload filters the european union deems necessary. If Facebook and Google (ContentID) can't do it reliably, who can?