7 ms·

So lame they focus on gatekeeping the exams and crying about permission as opposed to challenging the paper’s shit methodology.

by az226 3y ago

centmot 3y ago
Precisely.
11101010001100 3y ago
You realize this is only one part of the response?
- bluepod4 3y ago
  > We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. This is the conclusion of the memo. The problems with methodology are clearly a secondary concern. They also seem to imply that issues with consent are solely or mainly the cause of the methodology problems. This could be true, but idk. This is my opinion as an MIT grad who still views OCW content and similar content from other universities from time-to-time. (Complete speculation: I almost feel as if certain professors or lecturers are “embarrassed” about their content.)
  - dkqmduems 3y ago
    Working hard does not necessarily mean getting it right. (Also an mit grad).
    - bluepod4 3y ago
      No one said it did. Did you mean to reply to someone else? This thread was about whether the response focused on the consent or methodology problems and it’s my opinion that they are focusing on the consent issues more. I only copied a quote from the memo.
      - quadrifoliate 3y ago
        > He did so without the consent of many of his co-authors and despite having been told of problems that should be corrected before publication. I think they are point out consent issues in multiple cases, but also talk about "problems that should be corrected before publication". I can't really see how there would problems to be corrected before publication. If it's only a consent issue but the rest of the methodology is sound, they should just say "Drori should have waited for the consent to go ahead with publication, but the paper is fine"?
        bluepod4 3y ago
        To clarify, when I said this was the _conclusion_ of the memo, I meant that it was the literal closing statement (besides the one-liner/quip about GPT-4 not being able to earn an MIT degree). I didn’t mean that it was the be-all and end-all of those CSAIL profs’ perspective(s). The wording is still very suspect to me and seems like the Writing Center would have some constructive feedback for them (unless the meaning of words doesn’t matter).
      - dkqmduems 3y ago
        I take the stance that getting it right is implicitly understood to be primary, as it should be in science
        bluepod4 3y ago
        I agree but not everyone thinks like this even at MIT, as you should know since you said that you’re an alum.
23B1 3y ago
You... think it's lame that an academic institution would focus on protecting their students and professors?
- az226 3y ago
  Paper didn’t include the exams themselves, please correct me if my understanding is incorrect.
  - anonymouskimmer 3y ago
    Figure 4 on the last page of the PDF might include a single question.
  - dr_kretyn 3y ago
    Based on what i watched by Yannick on YouTube and haven't actually read... Questions weren't part of the paper but authors made a goof and included all of them in GitHub. Shortly after they deleted these questions but as part of a new commit so one could see the change, thus all the questions.
    - az226 3y ago
      But then this dumb statement should have been taking about the upload to GitHub and not the paper and how collaborators weren’t aware, didn’t get permission to run the analysis, etc. Sure, they’re all connected to the paper but they’re separate issues.
caddemon 3y ago
I mean if the dude submitted the work without the permission of the actual lead authors that's a pretty big violation, which also could explain in part why the methodology was so bad.
- az226 3y ago
  But that’s something they should deal with internally. It is quite strange actually. The statement should also come from MIT’s public affairs office and not the professors themselves. So it looks like they are big mad and acting on their own to save face.
  - deleted 3y ago
    [deleted]
- rsfern 3y ago
  The person who submitted to arxiv (and promoted the paper on Twitter) is one of the senior authors (listed last in the author block), so I don’t think that’s an excuse for publishing to arxiv before doing due diligence on methodology Apparently some of the other profs were not in the loop about the arxiv submission though
  - stefan_ 3y ago
    Senior author is a euphemism for "happens to run the lab". Explains why the other "senior authors" are lashing out, they never expected having to know what a paper that bears their name is all about.
    - rsfern 3y ago
      Sure - the point is that the preprint wasn’t posted by a student without permission of their advisor, it was posted by a professor without (apparently) permission of their collaborators, who aren’t part of a monolithic lab for what it’s worth. However, I generally agree with your take, and the response seems to me to kind of dodge the responsibility for methodology issues. that responsibility is not really compatible with the “hey we’re submitting to arxiv in a few days, last chance for comments” approach to collaboration that seems to be the minimum expected bar for signing off on the submission form that all authors are aware and agree to publication
YeGoblynQueenne 3y ago
The signatories are some of the paper's authors. It's not common for authors to criticise their own methodology as shitty. Anyway it's clear they didn't think the paper was ready to be made public so what is there to criticise about its methodology? It's a work in progress.

So lame they focus on gatekeeping the exams and crying about permission as opposed to challenging the paper’s shit methodology.

by az226 3y ago

So lame they focus on gatekeeping the exams and crying about permission as opposed to challenging the paper’s shit methodology.

centmot 3y ago

Precisely.

11101010001100 3y ago

You realize this is only one part of the response?

bluepod4 3y ago
> We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. This is the conclusion of the memo. The problems with methodology are clearly a secondary concern. They also seem to imply that issues with consent are solely or mainly the cause of the methodology problems. This could be true, but idk. This is my opinion as an MIT grad who still views OCW content and similar content from other universities from time-to-time. (Complete speculation: I almost feel as if certain professors or lecturers are “embarrassed” about their content.)
- dkqmduems 3y ago
  Working hard does not necessarily mean getting it right. (Also an mit grad).
  - bluepod4 3y ago
    No one said it did. Did you mean to reply to someone else? This thread was about whether the response focused on the consent or methodology problems and it’s my opinion that they are focusing on the consent issues more. I only copied a quote from the memo.
    - quadrifoliate 3y ago
      > He did so without the consent of many of his co-authors and despite having been told of problems that should be corrected before publication. I think they are point out consent issues in multiple cases, but also talk about "problems that should be corrected before publication". I can't really see how there would problems to be corrected before publication. If it's only a consent issue but the rest of the methodology is sound, they should just say "Drori should have waited for the consent to go ahead with publication, but the paper is fine"?
      - bluepod4 3y ago
        To clarify, when I said this was the _conclusion_ of the memo, I meant that it was the literal closing statement (besides the one-liner/quip about GPT-4 not being able to earn an MIT degree). I didn’t mean that it was the be-all and end-all of those CSAIL profs’ perspective(s). The wording is still very suspect to me and seems like the Writing Center would have some constructive feedback for them (unless the meaning of words doesn’t matter).
    - dkqmduems 3y ago
      I take the stance that getting it right is implicitly understood to be primary, as it should be in science
      - bluepod4 3y ago
        I agree but not everyone thinks like this even at MIT, as you should know since you said that you’re an alum.

23B1 3y ago

You... think it's lame that an academic institution would focus on protecting their students and professors?

az226 3y ago
Paper didn’t include the exams themselves, please correct me if my understanding is incorrect.
- anonymouskimmer 3y ago
  Figure 4 on the last page of the PDF might include a single question.
- dr_kretyn 3y ago
  Based on what i watched by Yannick on YouTube and haven't actually read... Questions weren't part of the paper but authors made a goof and included all of them in GitHub. Shortly after they deleted these questions but as part of a new commit so one could see the change, thus all the questions.
  - az226 3y ago
    But then this dumb statement should have been taking about the upload to GitHub and not the paper and how collaborators weren’t aware, didn’t get permission to run the analysis, etc. Sure, they’re all connected to the paper but they’re separate issues.

caddemon 3y ago

I mean if the dude submitted the work without the permission of the actual lead authors that's a pretty big violation, which also could explain in part why the methodology was so bad.

az226 3y ago
But that’s something they should deal with internally. It is quite strange actually. The statement should also come from MIT’s public affairs office and not the professors themselves. So it looks like they are big mad and acting on their own to save face.
- deleted 3y ago
  [deleted]
rsfern 3y ago
The person who submitted to arxiv (and promoted the paper on Twitter) is one of the senior authors (listed last in the author block), so I don’t think that’s an excuse for publishing to arxiv before doing due diligence on methodology Apparently some of the other profs were not in the loop about the arxiv submission though
- stefan_ 3y ago
  Senior author is a euphemism for "happens to run the lab". Explains why the other "senior authors" are lashing out, they never expected having to know what a paper that bears their name is all about.
  - rsfern 3y ago
    Sure - the point is that the preprint wasn’t posted by a student without permission of their advisor, it was posted by a professor without (apparently) permission of their collaborators, who aren’t part of a monolithic lab for what it’s worth. However, I generally agree with your take, and the response seems to me to kind of dodge the responsibility for methodology issues. that responsibility is not really compatible with the “hey we’re submitting to arxiv in a few days, last chance for comments” approach to collaboration that seems to be the minimum expected bar for signing off on the submission form that all authors are aware and agree to publication

YeGoblynQueenne 3y ago

The signatories are some of the paper's authors. It's not common for authors to criticise their own methodology as shitty. Anyway it's clear they didn't think the paper was ready to be made public so what is there to criticise about its methodology? It's a work in progress.