10 ms·

I see your point. And I understand that the term is understood differently by different people. But there is a substantial fraction of people (myself included

I see your point. And I understand that the term is understood differently by different people. But there is a substantial fraction of people (myself included) who understands it the way OSI defines it. My criticism is not that Facebook chooses to release the software under any particular terms (they are well within their rights not to release it at all). But the ambiguity of the term remains, and my belief is that Facebook is well aware of the ambiguity and nevertheless chooses to use it in bad faith, and that is what frustrates me. A less-ambiguous press release might instead say "source released under a non-commercial research license" or more succinctly "model code available for researchers," which sounds just fine.

godelski 3y ago
All I'm trying to get is people to change their phrasing to something like > Many may not be aware, because of the naming, but OSS is actually defined as having no commerce restrictions <link to OSD and DoD definitions>. Facebook should clarify that this is "source available". I'm glad the source is available, but this is extremely frustrating as we've had this discussion several times and I believe that Facebook should know better by now. I think such a phrasing could have led to better outcomes. For example, if I saw that -- instead of a bunch of comments about OSS -- I would have commented about how there is a lack of a training script and that this makes the work difficult to reproduce. While the training looks standard sometimes there are subtle differences that make this difficult. Though I'll admit that Facebook has a decent track record for reproduction within my own experience (MSR is the worst and Google is okay but uses proprietary datasets[0]). > or more succinctly "model code available for researchers," Just to be clear, the checkpoint is also available. But I should also mention that this is only the state dict and does not have other information, as is common practice. [0] Google needs to be not allowed to use this dataset in conference publications (at least in review phase). Not only does it make the work not reproducible but it breaks double blind reviewing, thus violating conference guidelines. Any paper which mentions JFT is automatically known as a Google work and biases the reviewer. I am ashamed that both reviewers and ACs have not flagged these works.

godelski 3y ago

All I'm trying to get is people to change their phrasing to something like > Many may not be aware, because of the naming, but OSS is actually defined as having no commerce restrictions <link to OSD and DoD definitions>. Facebook should clarify that this is "source available". I'm glad the source is available, but this is extremely frustrating as we've had this discussion several times and I believe that Facebook should know better by now. I think such a phrasing could have led to better outcomes. For example, if I saw that -- instead of a bunch of comments about OSS -- I would have commented about how there is a lack of a training script and that this makes the work difficult to reproduce. While the training looks standard sometimes there are subtle differences that make this difficult. Though I'll admit that Facebook has a decent track record for reproduction within my own experience (MSR is the worst and Google is okay but uses proprietary datasets[0]). > or more succinctly "model code available for researchers," Just to be clear, the checkpoint is also available. But I should also mention that this is only the state dict and does not have other information, as is common practice. [0] Google needs to be not allowed to use this dataset in conference publications (at least in review phase). Not only does it make the work not reproducible but it breaks double blind reviewing, thus violating conference guidelines. Any paper which mentions JFT is automatically known as a Google work and biases the reviewer. I am ashamed that both reviewers and ACs have not flagged these works.