13 ms·
Facebook disqualifies leaders of Deepfake Detection Challenge for rule breach
- dx034 6y agoWhy would written consent be needed from people appearing on pictures with CC-BY licence? Was this just an overreaction or is there any legal risk for Facebook using those pictures without additional consent?
- weinzierl 6y agoCreative Commons is based on copyright law. In some countries the rights regarding ones likeness are covered by separate laws and Creative Commons has no influence on that. I suspect that even in the US there is more to it. For example, stock photo agencies treat copyright and model releases as separate things. For any non-editorial images that contain recognizable faces they are in my experience strict about about getting a proper model release.
- tsbinz 6y agoIn addition to the concern that copyright doesn't necessarily cover personality rights, you can not assume that just because someone uploaded an image to Flickr as CC-BY that they had the right to do so, in a lot of cases they do not (when people upload images found on the internet). Yes, it's their fault, but in the end you still don't have a valid license for the image.
- prepend 6y agoI think the poster is responsible for correctly releasing content under the right copyright. I think it’s reasonable to assume that because something is labeled CC-BY that it’s legal to use for situations allowed by CC-BY. Is there some copyright certification service? Is there a way to test copyrights? If I don’t go my the author’s copyright, how could I verify it? If I watch a movie, should I not do so until I verify the personality rights of every person who appears in the film? How can I trust Disney?
- marcinzm 6y agoLegally, you are responsible for ensuring YOUR work follows copyright. If your work is made of other works then you need to ensure you have a license for them all. Otherwise you can be sued and saying someone uploaded it to Flickr with a CC-BY won't save you. Presumably you'd track down the original author and have them sign a piece of paper where they claim it is their work. Then you can at least sue them for damages if it's not. edit: As for your example, copyright generally doesn't care about you consuming a work but about you sharing a work. As such, you don't have to check anything but Disney does have to check everything.
- prepend 6y agoThat’s what I meant. The youtube poster is responsible for the copyright. I don’t think it’s possible for me to check beyond the legal copyright. Disney, and random youtuber, does have to check everything, not me if I’m legally using it. However, if the copyright if wrong, I’m not sure what the liability is.
- KarlKemp 6y agoWhile I understand where you’re coming from, in terms of the current legal situation, you are liable for mistakenly believing some license information when that turns out to be wrong. The way that publishers, for example, avoid such pitfalls is by exclusively working with professional agencies that control the provenance of the work they license. Watching anything isn’t copying nor showing and therefore safe. Practically, if you want to use any CC-licensed photos, I would advise at least a reverse image search.
- tsbinz 6y agoOne way to protect yourself is buying licenses to images from a service that provides legal indemnification. Part of the reason why stock image websites can earn money is that they provide this, so (at least part of) the risk is with them.
- chillfox 6y agoCopyright only gets you editorial usage. For any commercial usage, you must also have a model release or property release.
- parliament32 6y agoIt's not about legality, it's about image. If they hadn't disqualified them, the headline would've been "Creepy Facebook Contest Scrapes YouTubers' Faces for AI 'Black Box'".
- teddyh 6y agoOne possible reason for the “no external datasets” rule might be that the data which Facebook uses to judge the competition is also taken from the same publicly available sources. If this is so, then if somebody uses those same datasets, they would have trained to the test, so to speak, which obviously would not lead to good outcomes when run against future data.
- amitport 6y agoThis competition should not be about scraping and tagging skills (impressive as they may). So maybe they'll get to win on lack of clarity in the specifications, but that will be unfortunate.
- Tenoke 6y ago>This competition should not be about scraping and tagging skills Competitions that allow external data are at least partially explicitly about that. If they weren't they wouldn't allow external data (which is not exactly uncommon on kaggle).
- amitport 6y agoThey're explicit that this data should be shared. Obviously aiming for it to not be a part of unique contribution of the contender. They may not provide a tight enough statement about that, but it is fairly clear.
- tomp 6y agoSo you think this task is learneable without any labeled data ("scraping and tagging skills")? After all, that's pretty much how FB, Google et. al. do it as well, throw more data & CPU power at the problem.
- amitport 6y agoThat is not what I said
- Traster 6y agoI think the issue here is that Kaggle's statement that the top teams broke the rules is just very opaque. They stated they broke the rules on external data. The article then goes on to talk about what data the teams used and what licenses it has, and what data the teams were asked to provide. But it really is almost impossible to know what the concerns of FB/Kaggle were without them specifically stating them. Clearly whatever the issue was it didn't effect every team - so it may be there were details of the licenses that the disqualified teams used that weren't good enough. As I say though, it's very difficult to say and it's kind of hard to think of a reason Facebook would arbitrarily disqualify teams for no good reason. It's perfectly possible FB were concerned about image rights or something else, but people seem to be perfectly happy just assuming some grand conspiracy.
- nl 6y agoFor those who aren't aware, many Kaggle competitions allow external data (this one did) but require disclosure, and often there is some back-and-forth to clarify the exact details of what is used. In this case the disqualified participants are well respected and haven't previously been involved in any dubious behavior. They properly disclosed what they were doing and despite there being other clarifications there was none that person releases for CC-BY data would be required. Obviously this is a ridiculous requirement. There's no way for that team to be able to do that, but they did take proper care to use data that Facebook could reasonably use. It's unreasonable for FB/Kaggle to expect participants in a data science competition to suddenly know what Facebook's data ethics department is demanding this week outside what is legally required.
- mkagenius 6y agoYes, this seems like a clear case of bullying by FB and in turn by Kaggle. As a contestant I would be furious too. If I shared what data I am using during the contest and to other participants, and then I won then its completely fair. After the match, suddenly the Facebook legal team wakes up, they should have been awake during the whole contest. Ideally, they should give the prizes and then do another contest where the legal team is involved throughout
- deleted 6y ago[deleted]
- kuu 6y agoIf they disclosed the data, I agree with you that the decision should have been done during the competition, not after it...
- Hendrikto 6y ago> There's no way for that team to be able to do that There is an extremely simple solution to that: Do not use the data. There are countries in which privacy is considered important. Data being public has no influence on whether is is considered personal in the EU. Also, copyright has nothing to do with the issue.
- 6y ago
- vivekkalyan 6y agoThe rules seem pretty clear that consent is required from any persons appearing in any external datasets that are required. The winners scraped data from Youtube videos so I am not not sure the issue is. The more worrying takeaway is that the winners scraped videos from people who clearly had no intention of their videos being used for a deepfake detection algorithm. Yet they did not think of the ethical considerations of using that data (did everyone in the video even have a say in the video being uploaded?). I think Kaggle disqualifying the team is the right move (even if it's a painful one for the winners).
- oars 6y agoIt's much better they learn now by being banned from a competition rather than having a lawsuit filed against them in the future. The correct decision was made.
- nostrebored 6y agoI'm with you here. There are ethical concerns, legal concerns for productization, and overall this defeats the purpose of creating novel algorithms rather than a better trained model. For instance, with the same scraping being used to train the deepfake GAN, would their model be more or less effective than a competitor model? It seems like they won from a disparity in data not an innovative technical approach.
- reedwolf 6y agoYeah, with $1 million at stake, I can't believe this team of really smart people made such an incredible blunder. The whole reason Facebook launched this challenge was to try and bury the bad PR over their data practices. If people in the external datasets had complained about the unauthorized use of their faces in the winning solution, it would've been pretty embarrassing for FB.
- TheAdamAndChe 6y agoThe article states the videos used a Creative Commons license that allowed for commercial use. It is an extremely liberal license that does not state "free for commercial use except for when used with facial recognition."
- phh 6y agoIMO the real issue is that Facebook wanted a commercially-usable product, and they thought Kaggle had all the safeguards for that, but no: Because of RGPD and friends, Facebook can't store those photos, even though their license are permissive, and respects Kaggle's rules. This clearly shows what Kaggle is: It is a way to get very cheap and high quality data-science work. It's not for hiring people, not for truly helping the research community, not for to help people learn. Nope, just cheap workers. It really feels like Facebook have there whole deepfake detection strategy here! They put like 2M$ on the table to solve an issue that will(?) plague their whole multi-billion platform.
- Majestic121 6y agoDo Facebook need to store the photos to use the model commercialy ? I would expect the algo + weight of the model to be enough, and that it is all fine RGPD-wise since you definitely can't identify people from the weight of a model trained with your picture.
- willcipriano 6y agoPresumably you'd want to retrain the model from time to time and that would require the original material.
- mnky9800n 6y agoI think this is the main takeaway. Kaggle, corporate sponsored hack-a-thons, etc. are all very cheap ways for companies to get a lot of work that is typically very expensive done for very cheap. Who benefits from these situations except for companies? Even if you get a job out of your kaggle contributions, what value was created for a company by kaggle in comparison to the value that was created for you?
- KarlKemp 6y agoWe would arguably all benefit from a robust method to detect deepfakes, and from its availability, as open source, to all platforms large and small.
- deleted 6y ago[deleted]
- UweSchmidt 6y agoIt's unfortunate that the title leads with the "backlash" from the thing that happened, not the thing itself ("Kaggle disqualifies participants over usage of external data usage"). This suggests that the decision about the case has already been made by a plurality and with fervor. In reality this article is the first time many HN readers learn about this for the first time. While I'm sure it's just accidental in this case, I see this all over the news and suspect attempts to steer public opinion by condemning people or institions in the headline, before the news is actually reported on. A forum of independent thinkers should insist on not having the news presented to them in a potentially manipulative manner.
- RMPR 6y agoI feel the same, and it was the right move imho because if they didn't disqualified the winners, another header could have been "Facebook, Kaggle rewarding unauthorized use of people data"
- pulse7 6y agoWe are in the Zeitgeist of "...all over the news and suspect attempts to steer public opinion"... One of the reasons is newspapers fighting for survival (and therefore prioritizing "what people like to hear" to boost ad-clicks over "what people should know"). There are also other reasons like newspaper editors favoring one political party other the other - and therefore they publish mostly negative news about one party and mostly positive news about the other one...
- ig1 6y agoAs someone who previously competed on Kaggle, this seems a reasonable decision. In previous contests it was pretty clear if you wanted to do something that used third party data you should get pre-clearance for it from Kaggle/contest organizers. The disqualified competitors here seem to have assumed that CC-BY meant you can do whatever you want with data, when actually that's far from true. CC-BY is solely about copyright and doesn't address other rights (e.g. model release, gdpr, etc.)
- sillysaurusx 6y agoAs a machine learning researcher, where exactly am I supposed to get a dataset that complies with Facebook's/Kaggle's rules in this case? No one is disputing that the team was disqualified fair and square. But this rule – where you must get consent from every single person appearing in your training data – seems neither standard nor sensible. Firstly, as someone else pointed out, copyright doesn't apply here at all. You can use whatever training data you want as long as your model is sufficiently transformative. OpenAI used terabytes of copyrighted music in their training for OpenAI Jukebox; they certainly didn't get a license from every musician. Beyond that – big companies don't play by this rule! If a BigCo wants to train on some data, you bet they'll be using it. When's the last time Google sent you an email like "Are you ok with us using your flickr photos to help improve Google Image Search?" So my question is simple: in the context of this competition, where should I go to get a decent dataset? The winners were disqualified doing exactly what I would have done. What's the alternative? Also, yes, ethics are a concern. If you're concerned about ethics, aim it at big companies, not us small fries that are merely trying to win some cash. Again, no one disputes that they were disqualified for valid reasons. But it has nothing to do with ethics and everything to do with the artificial constraints imposed by this competition.
- tsbinz 6y agoBig companies definitely care a lot more about these things than universities or "small-time researchers". I've been in long meetings with corporate lawyers at multiple big companies, and they are a lot more careful than you imply (because they have more to lose). It's always a risk vs reward thing. Things like "copyright doesn't apply here at all. You can use whatever training data you want as long as your model is sufficiently transformative." are not certain until they have been tested in courts, and will be different for each case (it's a lot easier to argue that a simple classifier is transformative than a GAN that might reproduce one of its training inputs, for example). An additional complication is that if you're a company that's operating globally, you have to pay attention to a lot more laws. Where I work now we work a lot with synthetic training data, because dealing with "real" data is complicated.
- Hendrikto 6y ago> As a machine learning researcher, where exactly am I supposed to get a dataset that complies with Facebook's/Kaggle's rules in this case? From Kaggle… There is a datasat provided for each challenge.
- RNCTX 6y agoIronic rules from Facebook considering Facebook has been caught multiple times harvesting data from other apps on phones without permission.
- withinboredom 6y ago> and each individual participant further waives all rights to have damages multiplied or increased. What about divided? by a fraction? :trollface: Does that fall under "increased"?