9 ms·

On the paper “Exploring the MIT Mathematics and EECS Curriculum Using LLMs” [pdf]

160 points by jlaneve 3y ago

az226 3y ago
So lame they focus on gatekeeping the exams and crying about permission as opposed to challenging the paper’s shit methodology.
- centmot 3y ago
  Precisely.
- 11101010001100 3y ago
  You realize this is only one part of the response?
  - bluepod4 3y ago
    > We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. This is the conclusion of the memo. The problems with methodology are clearly a secondary concern. They also seem to imply that issues with consent are solely or mainly the cause of the methodology problems. This could be true, but idk. This is my opinion as an MIT grad who still views OCW content and similar content from other universities from time-to-time. (Complete speculation: I almost feel as if certain professors or lecturers are “embarrassed” about their content.)
    - dkqmduems 3y ago
      Working hard does not necessarily mean getting it right. (Also an mit grad).
      - bluepod4 3y ago
        No one said it did. Did you mean to reply to someone else? This thread was about whether the response focused on the consent or methodology problems and it’s my opinion that they are focusing on the consent issues more. I only copied a quote from the memo.
        quadrifoliate 3y ago
        > He did so without the consent of many of his co-authors and despite having been told of problems that should be corrected before publication. I think they are point out consent issues in multiple cases, but also talk about "problems that should be corrected before publication". I can't really see how there would problems to be corrected before publication. If it's only a consent issue but the rest of the methodology is sound, they should just say "Drori should have waited for the consent to go ahead with publication, but the paper is fine"?
        bluepod4 3y ago
        To clarify, when I said this was the _conclusion_ of the memo, I meant that it was the literal closing statement (besides the one-liner/quip about GPT-4 not being able to earn an MIT degree). I didn’t mean that it was the be-all and end-all of those CSAIL profs’ perspective(s). The wording is still very suspect to me and seems like the Writing Center would have some constructive feedback for them (unless the meaning of words doesn’t matter).
        dkqmduems 3y ago
        I take the stance that getting it right is implicitly understood to be primary, as it should be in science
        bluepod4 3y ago
        I agree but not everyone thinks like this even at MIT, as you should know since you said that you’re an alum.
- 23B1 3y ago
  You... think it's lame that an academic institution would focus on protecting their students and professors?
  - az226 3y ago
    Paper didn’t include the exams themselves, please correct me if my understanding is incorrect.
    - anonymouskimmer 3y ago
      Figure 4 on the last page of the PDF might include a single question.
    - dr_kretyn 3y ago
      Based on what i watched by Yannick on YouTube and haven't actually read... Questions weren't part of the paper but authors made a goof and included all of them in GitHub. Shortly after they deleted these questions but as part of a new commit so one could see the change, thus all the questions.
      - az226 3y ago
        But then this dumb statement should have been taking about the upload to GitHub and not the paper and how collaborators weren’t aware, didn’t get permission to run the analysis, etc. Sure, they’re all connected to the paper but they’re separate issues.
- caddemon 3y ago
  I mean if the dude submitted the work without the permission of the actual lead authors that's a pretty big violation, which also could explain in part why the methodology was so bad.
  - az226 3y ago
    But that’s something they should deal with internally. It is quite strange actually. The statement should also come from MIT’s public affairs office and not the professors themselves. So it looks like they are big mad and acting on their own to save face.
    - deleted 3y ago
      [deleted]
  - rsfern 3y ago
    The person who submitted to arxiv (and promoted the paper on Twitter) is one of the senior authors (listed last in the author block), so I don’t think that’s an excuse for publishing to arxiv before doing due diligence on methodology Apparently some of the other profs were not in the loop about the arxiv submission though
    - stefan_ 3y ago
      Senior author is a euphemism for "happens to run the lab". Explains why the other "senior authors" are lashing out, they never expected having to know what a paper that bears their name is all about.
      - rsfern 3y ago
        Sure - the point is that the preprint wasn’t posted by a student without permission of their advisor, it was posted by a professor without (apparently) permission of their collaborators, who aren’t part of a monolithic lab for what it’s worth. However, I generally agree with your take, and the response seems to me to kind of dodge the responsibility for methodology issues. that responsibility is not really compatible with the “hey we’re submitting to arxiv in a few days, last chance for comments” approach to collaboration that seems to be the minimum expected bar for signing off on the submission form that all authors are aware and agree to publication
- YeGoblynQueenne 3y ago
  The signatories are some of the paper's authors. It's not common for authors to criticise their own methodology as shitty. Anyway it's clear they didn't think the paper was ready to be made public so what is there to criticise about its methodology? It's a work in progress.
CamperBob2 3y ago
Is there a link to the original story? Drori's homepage at MIT seems to have a "Certificate Error" (probably 405: Revoked Diploma.) I assume the paper itself is long gone.
- jlaneve 3y ago
  Original paper is on arxiv: https://arxiv.org/abs/2306.08997 https://arxiv.org/abs/2306.08997
- idontpost 3y ago
  Seems like it's still on arXiv.
- primordialsoup 3y ago
  https://theaireport.substack.com/p/the-ai-report-5-gpt-4-aces-mit-or https://theaireport.substack.com/p/the-ai-report-5-gpt-4-ace...
dfdz 3y ago
https://arxiv.org/abs/2306.08997 https://arxiv.org/abs/2306.08997 “arXiv will not consider removal for reasons such as journal similarity detection, nor failure to obtain consent from co-authors, as these do not invalidate the license applied by the submitter” The submitter can mark the paper as “withdrawn” but it will remain available https://news.ycombinator.com/item?id=32780403 https://news.ycombinator.com/item?id=32780403
- anonymouskimmer 3y ago
  DMCA takedown, then? Or not applicable because the data is not part of the publication? Tables 17 and 18 in the Appendix could probably be removed as they seem to verbatim copy course descriptions, as well as, maybe, Figure 4. > Iddo did not have permission from all the instructors to collect the assignment and exam questions that made up the dataset that was the subject of the paper.
  - skissane 3y ago
    > DMCA takedown, then? Or not applicable because the data is not part of the publication? Legally, you are allowed to refuse to comply with DMCA take down notices. If you do, you are increasing your risk of being sued for copyright infringement, and increasing the potential damages if you lose – but, if you decide (in any individual case) that is a risk worth taking, you are free to take that risk. If MIT tried to issue a DMCA takedown to arXiv over this, arXiv might decide that defending their own policies is worth the risk of being sued by MIT. > Tables 17 and 18 in the Appendix could probably be removed as they seem to verbatim copy course descriptions, as well as, maybe, Figure 4. Probably a sufficiently small extract from the source material, that it would fall under fair use? (Lack of acknowledgement of the specific source may be an issue; but that can be remedied by adding an acknowledgement, rather than removal.)
    - anonymouskimmer 3y ago
      IANAL. For the tables there's very little transformation, and a huge chunk of verbatim text. I don't see how there is any gain versus just publishing the course numbers and titles. For figure 4 this might fall under "unpublished material" protections, which are: https://www2.archivists.org/publications/brochures/copyright-and-unpublished-material https://www2.archivists.org/publications/brochures/copyright... > Generally, material is considered unpublished if it was not intended for public distribution or if only a few copies were created and distribution was limited. > The law distinguishes between published and unpublished material and the courts often afford more copyright protection to unpublished material when an asserted fair use is challenged. > Rather, courts evaluate fair use cases based on four factors, no one of which is determinative in and of itself: 2) > Courts give more protection to works that are “closer to the core of copyright protection,” such as unpublished 4) > The effect of the use upon the potential market for, or value of, the copyrighted work: This factor assesses how, and to what extent, the use damages the existing and potential market for the original. Publication of the (possibly) previously unpublished copyrighted work in figure 4 fully and completely destroys its value. I don't know if a fair use claim can overcome such an impact, though that is up to a court to determine.
dang 3y ago
Recent and related: No, GPT4 Can’t Ace MIT - https://news.ycombinator.com/item?id=36370685 https://news.ycombinator.com/item?id=36370685 - June 2023 (120 comments)
asylteltine 3y ago
[dead]
emmender 3y ago
Such despicable behavior from everyone involved. 1. Professors and MIT students attempting to gain fleeting fame riding on the bandwagon without trying to do deep work 2. Professors at MIT getting upset and lash out that GPT-4 can now get a MIT degree devaluing said degree. What has become of academia these days ?
deleted 3y ago
[deleted]
knaik94 3y ago
>We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. I appreciate the clear stance that MIT has taken regarding where the responsibility lies in this situation. I think some people are missing the context that many of the authors were undergraduate/early career. Research is an iterative process and every paper has to start somewhere. I don't agree that the paper should be withdrawn because Arxiv is not technically a publication, but I also wouldn't consider the paper properly peer reviewed. Teachers own the copyright the exam material. I was taught copyrighted material can't and shouldn't be used as part of an eval dataset. The followup by three other MIT ('24) seniors is a great peer review. https://flower-nutria-41d.notion.site/No-GPT4-can-t-ace-MIT-b27e6796ab5a48368127a98216c76864 https://flower-nutria-41d.notion.site/No-GPT4-can-t-ace-MIT-... No, GPT4 Can’t Ace MIT - https://news.ycombinator.com/item?id=36370685 https://news.ycombinator.com/item?id=36370685 - June 2023 (120 comments)
- deleted 3y ago
  [deleted]
bjourne 3y ago
> We want to emphasize that all the student authors in this paper worked really hard on what could have > been a very interesting and valuable paper had the data been collected with consent. The many > problems with the published work were not the fault of the students. So what did the student authors work "really hard on" if not the same data that was collected without consent? Either all student authors are at fault for working on a paper based on data collected without consent or none are. It's not publishing the paper that is the problem here.
- knaik94 3y ago
  The issue isn't that the results are necessarily bad from using a dataset with copyrighted exams. Using it for training data is "fine". But a copyrighted eval dataset makes it basically impossible to reproduce the results. One part of the original paper was validating the use of the MITQ dataset as a way to evaluate various models. This is seperate from the issues related to the methodology not agreeing with conclusion that GPT4 could "get an MIT degree". The second conclusion and methodology could have been fixed potentially.
  - bjourne 3y ago
    The vast majority of all ml research is using copyrighted data both for training and evaluation.
    - knaik94 3y ago
      I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms. Copyrighted data would make it harder to share and use, to reproduce results. I don't remember coming across a benchmark dataset that was copyrighted.
      - bjourne 3y ago
        That's very odd because I can't for the life of me think of an ml benchmark dataset that is not copyrighted. Copyrighted datasets are everywhere in ml research.
simplesamp 3y ago
What I fail to understand is that these authors were presumably okay sending the paper to Neurips (a top tier conference). I am assuming this based on the paper being in Neurips submission template. And now when the cat is out of the bag in open they are denouncing it.
- artfulmink 3y ago
  Honestly not quite a fair assumption. Some schools have students use neurips templates for class submissions (at least my graduate school did), and so these students may have just been using this format out of habit/familiarity.
  - simplesamp 3y ago
    Maybe. But the paper being exact nine pages (page limit of neurips this year) and being put on arxiv just a month after neurips deadline is too much of a coincidence.
muds 3y ago
Putting papers and code on arXiv shouldn't be punished. The incentive to do this is to protect your idea from getting scooped, and also to inform your close community on interesting problems that you're working on and get feedback. ArXiv is meant for work in progress ideas that won't necessarily stand the peer review process, but this isn't really acknowledged properly on social media. I highly doubt the Twitter storm would have been this intense if the twitter posts explicitly acknowledged this as a "Draft publication which hints as X." But I admit that pointing fingers at nobody in general and social media specifically is a pretty lazy solution. The takeaway IMO seems to be to prepend the abstract with a clear disclaimer sentence conveying the uncertainty of the research in question. For instance, adding a clear "WORKING DRAFT: ..." in the abstract section.
- deleted 3y ago
  [deleted]
- cscurmudgeon 3y ago
  > The incentive to do this is to protect your idea from getting scooped, and The other side is flag planting with half-baked ideas and results.
  - deleted 3y ago
    [deleted]
- ttpphd 3y ago
  I think you missed the point that data needs to be collected and presented ethically. It's not about it being a work in progress and not peer reviewed.
  - muds 3y ago
    I agree that the data collection process wasn't ethical, and the professor should definitely be reprimanded for that. It's extremely sad that the coauthors weren't aware of this as well. And I feel terrible for the undergrads: their first research experience was publically rebuked for no fault of their own. However, there is no shortage of projects with sketchy data collection methodologies on arXiv that haven't received this amount of attention. The point of putting stuff on arXiv _is_ that the paper will not pass / has not passed peer review in its current form! I might even call arXiv a safe space to publish ideas. We all benefit from this: a lot of interesting papers are only available on arxiv v.s. being shared between specific labs. I'm concerned that this fiasco was enabled by this new paradigm in AI social media reporting, where a project's findings are amplified and all the degrees of uncertainty are repressed. And I'm honestly not sure how to best deal with this other than either amplifying the uncertainty and jankyness in the paper itself to an annoyingly noticeable level, or just going back to the old way of privately sharing ideas. Maybe this is the best case scenario for these sorts of papers? They pushed a paper on a public journal, and got a public "peer review" of the paper. Turns out the community voted "strong reject;" and it also turns out that the stakes for public rejection are (uncomfortably, IMO) higher than for a normal rejection. Maybe this causes the researchers to only publically release better research, or (more likely) this causes the researchers to privately release all future papers.
visarga 3y ago
There is an insightful video review of this paper by Yannic Kilcher: https://youtu.be/Tkijsu129M0?t=30 https://youtu.be/Tkijsu129M0?t=30 TL;DW - Their approach is to sequentially test methods, moving on to the next if one fails. However, this strategy is flawed as it requires ground truth, particularly for multi-choice answers. The analogy could be made to continually rolling a dice until landing on six. Similarly, if a question has four potential answers, the model merely has to attempt four times to stumble upon the correct response. And then they report 100% success rate.
- az226 3y ago
  Yes and no. Some LLMs can’t get the right answer no matter how many rolls of the dice they get. The bigger BS is that several questions weren’t questions and some didn’t have enough detail to be answered and it somehow got 100% on those. This IMO out the entire paper into the trash.
  - caddemon 3y ago
    It graded itself, so that explains that lol
tinyhouse 3y ago
People overreact to this paper because it's from MIT. Lots of crappy papers out there, this is just another one. MIT is not special, they have a lot of average people too.
- caddemon 3y ago
  This was written by young undergrads with no actual research experience and a random visiting lecturer who is obviously trying to career climb (and had it backfire). Certainly there is bad research produced by MIT but I'd hardly call this "produced by MIT" in the traditional academic sense. It is extremely easy actually to get yourself some sort of affiliation. Perhaps they should crack down on that, but I think the relative openness of MIT is overall a good university culture to have, and only a small minority of those with tenuous affiliation are actually grifters.
data_maan 3y ago
Here's a more rigorous take on evaluating the output of GPT models on math: https://arxiv.org/abs/2301.13867 https://arxiv.org/abs/2301.13867
- cscurmudgeon 3y ago
  Unfortunately, negative but robust results like this won't get a fraction of the attention this MIT paper initially got.
- ianbutler 3y ago
  They don't specify GPT-4. The fact they generically refer to ChatGPT leads me to believe they're assessing 3.5. GPT-4 is significantly more competent of a model and while I won't speculate on it's math ability I'd want to see a follow up, specifically for GPT-4.
  - Sunhold 3y ago
    Yup. Page 19: "we focused on the 9th-January-2023 version of ChatGPT"
kleiba 3y ago
Phew, I'm just happy that OpenAI first gathered consent from all data sources on which they trained GPT4.
ec109685 3y ago
Not condoning what the author did, but it’s amazing how little the questions answered in the paper have to do with real life software engineering. Yes, it shows fortitude to pass MIT’s exam, but beyond that, a lot of stress for a test that will soon be forgotten after their undergraduate degree.
PeterStuer 3y ago
One can only dream this level of analysis and falsification would be the norm in academic publishing. Sadly, this is an extreme outlier for now.
sgt101 3y ago
Well, someone's in trouble. If a faculty member allowed their name to be put on this thing then that's on them, if they didn't then whoever did the submission has to go. Also all the papers of all the co-authors who allowed their name on it are now suspect for me and should be investigated seriously. Sure, if the fact is that they said "don't put my name on stuff that you publish unless I have specifically agreed" or "don't publish this thing, it needs work" and then that message got lost - they are off the hook for me. But there needs to be a look at the culture of the labs involved even in that case. That's brutal because as I say there is a possibility that they are victims of someone going over the top, but unfortunately I think MIT has to do it. In addition, for me there is a much wider ethics issue here. How many papers can really be checked properly by someone claiming authorship even if that author isn't really contributing? I can read a paper like this in about a fortnight because I have a job and a life. A faculty member also has a job and a life - they are doing admin and teaching as well as research. So to me it's impossible for someone to check 25 papers or more a year. I am seeing far higher counts than this by many academics. But, this is an extremely conservative threshold in my opinion. When I have contributed to scientific papers it has taken me at least three months of solid work each time. Often these papers get rejected (rightly) and then have to be substantially amended (or occasionally just abandoned) I am not that talented for sure, but I really find it hard to credit that anyone with an actual job (so not a post-doc or a student) can contribute to more than one academic paper a year. Potentially two or three if there is a confluence of papers getting ready for print... but not on a sustained basis. There are two solutions. Every university and research institute needs to investigate all the publications of academics with high paper counts per year. This is a red flag. I definitely think that if folks are in the top quartile in a department it needs to be looked at carefully. The other solution is that no academic publishing venue (conference or journal) should accept more than one paper per year from any author.

160 points by jlaneve 3y ago

So lame they focus on gatekeeping the exams and crying about permission as opposed to challenging the paper’s shit methodology.

centmot 3y ago
Precisely.
11101010001100 3y ago
You realize this is only one part of the response?
- bluepod4 3y ago
  > We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. This is the conclusion of the memo. The problems with methodology are clearly a secondary concern. They also seem to imply that issues with consent are solely or mainly the cause of the methodology problems. This could be true, but idk. This is my opinion as an MIT grad who still views OCW content and similar content from other universities from time-to-time. (Complete speculation: I almost feel as if certain professors or lecturers are “embarrassed” about their content.)
  - dkqmduems 3y ago
    Working hard does not necessarily mean getting it right. (Also an mit grad).
    - bluepod4 3y ago
      No one said it did. Did you mean to reply to someone else? This thread was about whether the response focused on the consent or methodology problems and it’s my opinion that they are focusing on the consent issues more. I only copied a quote from the memo.
      - quadrifoliate 3y ago
        > He did so without the consent of many of his co-authors and despite having been told of problems that should be corrected before publication. I think they are point out consent issues in multiple cases, but also talk about "problems that should be corrected before publication". I can't really see how there would problems to be corrected before publication. If it's only a consent issue but the rest of the methodology is sound, they should just say "Drori should have waited for the consent to go ahead with publication, but the paper is fine"?
        bluepod4 3y ago
        To clarify, when I said this was the _conclusion_ of the memo, I meant that it was the literal closing statement (besides the one-liner/quip about GPT-4 not being able to earn an MIT degree). I didn’t mean that it was the be-all and end-all of those CSAIL profs’ perspective(s). The wording is still very suspect to me and seems like the Writing Center would have some constructive feedback for them (unless the meaning of words doesn’t matter).
      - dkqmduems 3y ago
        I take the stance that getting it right is implicitly understood to be primary, as it should be in science
        bluepod4 3y ago
        I agree but not everyone thinks like this even at MIT, as you should know since you said that you’re an alum.
23B1 3y ago
You... think it's lame that an academic institution would focus on protecting their students and professors?
- az226 3y ago
  Paper didn’t include the exams themselves, please correct me if my understanding is incorrect.
  - anonymouskimmer 3y ago
    Figure 4 on the last page of the PDF might include a single question.
  - dr_kretyn 3y ago
    Based on what i watched by Yannick on YouTube and haven't actually read... Questions weren't part of the paper but authors made a goof and included all of them in GitHub. Shortly after they deleted these questions but as part of a new commit so one could see the change, thus all the questions.
    - az226 3y ago
      But then this dumb statement should have been taking about the upload to GitHub and not the paper and how collaborators weren’t aware, didn’t get permission to run the analysis, etc. Sure, they’re all connected to the paper but they’re separate issues.
caddemon 3y ago
I mean if the dude submitted the work without the permission of the actual lead authors that's a pretty big violation, which also could explain in part why the methodology was so bad.
- az226 3y ago
  But that’s something they should deal with internally. It is quite strange actually. The statement should also come from MIT’s public affairs office and not the professors themselves. So it looks like they are big mad and acting on their own to save face.
  - deleted 3y ago
    [deleted]
- rsfern 3y ago
  The person who submitted to arxiv (and promoted the paper on Twitter) is one of the senior authors (listed last in the author block), so I don’t think that’s an excuse for publishing to arxiv before doing due diligence on methodology Apparently some of the other profs were not in the loop about the arxiv submission though
  - stefan_ 3y ago
    Senior author is a euphemism for "happens to run the lab". Explains why the other "senior authors" are lashing out, they never expected having to know what a paper that bears their name is all about.
    - rsfern 3y ago
      Sure - the point is that the preprint wasn’t posted by a student without permission of their advisor, it was posted by a professor without (apparently) permission of their collaborators, who aren’t part of a monolithic lab for what it’s worth. However, I generally agree with your take, and the response seems to me to kind of dodge the responsibility for methodology issues. that responsibility is not really compatible with the “hey we’re submitting to arxiv in a few days, last chance for comments” approach to collaboration that seems to be the minimum expected bar for signing off on the submission form that all authors are aware and agree to publication
YeGoblynQueenne 3y ago
The signatories are some of the paper's authors. It's not common for authors to criticise their own methodology as shitty. Anyway it's clear they didn't think the paper was ready to be made public so what is there to criticise about its methodology? It's a work in progress.

CamperBob2 3y ago

Is there a link to the original story? Drori's homepage at MIT seems to have a "Certificate Error" (probably 405: Revoked Diploma.) I assume the paper itself is long gone.

jlaneve 3y ago
Original paper is on arxiv: https://arxiv.org/abs/2306.08997 https://arxiv.org/abs/2306.08997
idontpost 3y ago
Seems like it's still on arXiv.
primordialsoup 3y ago
https://theaireport.substack.com/p/the-ai-report-5-gpt-4-aces-mit-or https://theaireport.substack.com/p/the-ai-report-5-gpt-4-ace...

dfdz 3y ago

https://arxiv.org/abs/2306.08997 https://arxiv.org/abs/2306.08997 “arXiv will not consider removal for reasons such as journal similarity detection, nor failure to obtain consent from co-authors, as these do not invalidate the license applied by the submitter” The submitter can mark the paper as “withdrawn” but it will remain available https://news.ycombinator.com/item?id=32780403 https://news.ycombinator.com/item?id=32780403

anonymouskimmer 3y ago
DMCA takedown, then? Or not applicable because the data is not part of the publication? Tables 17 and 18 in the Appendix could probably be removed as they seem to verbatim copy course descriptions, as well as, maybe, Figure 4. > Iddo did not have permission from all the instructors to collect the assignment and exam questions that made up the dataset that was the subject of the paper.
- skissane 3y ago
  > DMCA takedown, then? Or not applicable because the data is not part of the publication? Legally, you are allowed to refuse to comply with DMCA take down notices. If you do, you are increasing your risk of being sued for copyright infringement, and increasing the potential damages if you lose – but, if you decide (in any individual case) that is a risk worth taking, you are free to take that risk. If MIT tried to issue a DMCA takedown to arXiv over this, arXiv might decide that defending their own policies is worth the risk of being sued by MIT. > Tables 17 and 18 in the Appendix could probably be removed as they seem to verbatim copy course descriptions, as well as, maybe, Figure 4. Probably a sufficiently small extract from the source material, that it would fall under fair use? (Lack of acknowledgement of the specific source may be an issue; but that can be remedied by adding an acknowledgement, rather than removal.)
  - anonymouskimmer 3y ago
    IANAL. For the tables there's very little transformation, and a huge chunk of verbatim text. I don't see how there is any gain versus just publishing the course numbers and titles. For figure 4 this might fall under "unpublished material" protections, which are: https://www2.archivists.org/publications/brochures/copyright-and-unpublished-material https://www2.archivists.org/publications/brochures/copyright... > Generally, material is considered unpublished if it was not intended for public distribution or if only a few copies were created and distribution was limited. > The law distinguishes between published and unpublished material and the courts often afford more copyright protection to unpublished material when an asserted fair use is challenged. > Rather, courts evaluate fair use cases based on four factors, no one of which is determinative in and of itself: 2) > Courts give more protection to works that are “closer to the core of copyright protection,” such as unpublished 4) > The effect of the use upon the potential market for, or value of, the copyrighted work: This factor assesses how, and to what extent, the use damages the existing and potential market for the original. Publication of the (possibly) previously unpublished copyrighted work in figure 4 fully and completely destroys its value. I don't know if a fair use claim can overcome such an impact, though that is up to a court to determine.

dang 3y ago

Recent and related: No, GPT4 Can’t Ace MIT - https://news.ycombinator.com/item?id=36370685 https://news.ycombinator.com/item?id=36370685 - June 2023 (120 comments)

asylteltine 3y ago

[dead]

emmender 3y ago

Such despicable behavior from everyone involved. 1. Professors and MIT students attempting to gain fleeting fame riding on the bandwagon without trying to do deep work 2. Professors at MIT getting upset and lash out that GPT-4 can now get a MIT degree devaluing said degree. What has become of academia these days ?

deleted 3y ago

[deleted]

knaik94 3y ago

>We want to emphasize that all the student authors in this paper worked really hard on what could have been a very interesting and valuable paper had the data been collected with consent. The many problems with the published work were not the fault of the students. I appreciate the clear stance that MIT has taken regarding where the responsibility lies in this situation. I think some people are missing the context that many of the authors were undergraduate/early career. Research is an iterative process and every paper has to start somewhere. I don't agree that the paper should be withdrawn because Arxiv is not technically a publication, but I also wouldn't consider the paper properly peer reviewed. Teachers own the copyright the exam material. I was taught copyrighted material can't and shouldn't be used as part of an eval dataset. The followup by three other MIT ('24) seniors is a great peer review. https://flower-nutria-41d.notion.site/No-GPT4-can-t-ace-MIT-b27e6796ab5a48368127a98216c76864 https://flower-nutria-41d.notion.site/No-GPT4-can-t-ace-MIT-... No, GPT4 Can’t Ace MIT - https://news.ycombinator.com/item?id=36370685 https://news.ycombinator.com/item?id=36370685 - June 2023 (120 comments)

deleted 3y ago
[deleted]

bjourne 3y ago

> We want to emphasize that all the student authors in this paper worked really hard on what could have > been a very interesting and valuable paper had the data been collected with consent. The many > problems with the published work were not the fault of the students. So what did the student authors work "really hard on" if not the same data that was collected without consent? Either all student authors are at fault for working on a paper based on data collected without consent or none are. It's not publishing the paper that is the problem here.

knaik94 3y ago
The issue isn't that the results are necessarily bad from using a dataset with copyrighted exams. Using it for training data is "fine". But a copyrighted eval dataset makes it basically impossible to reproduce the results. One part of the original paper was validating the use of the MITQ dataset as a way to evaluate various models. This is seperate from the issues related to the methodology not agreeing with conclusion that GPT4 could "get an MIT degree". The second conclusion and methodology could have been fixed potentially.
- bjourne 3y ago
  The vast majority of all ml research is using copyrighted data both for training and evaluation.
  - knaik94 3y ago
    I don't mean evaluating a model at the end of training, I mean datasets used to quantify the relative performance across models and algorithms. Copyrighted data would make it harder to share and use, to reproduce results. I don't remember coming across a benchmark dataset that was copyrighted.
    - bjourne 3y ago
      That's very odd because I can't for the life of me think of an ml benchmark dataset that is not copyrighted. Copyrighted datasets are everywhere in ml research.

simplesamp 3y ago

What I fail to understand is that these authors were presumably okay sending the paper to Neurips (a top tier conference). I am assuming this based on the paper being in Neurips submission template. And now when the cat is out of the bag in open they are denouncing it.

artfulmink 3y ago
Honestly not quite a fair assumption. Some schools have students use neurips templates for class submissions (at least my graduate school did), and so these students may have just been using this format out of habit/familiarity.
- simplesamp 3y ago
  Maybe. But the paper being exact nine pages (page limit of neurips this year) and being put on arxiv just a month after neurips deadline is too much of a coincidence.

muds 3y ago

Putting papers and code on arXiv shouldn't be punished. The incentive to do this is to protect your idea from getting scooped, and also to inform your close community on interesting problems that you're working on and get feedback. ArXiv is meant for work in progress ideas that won't necessarily stand the peer review process, but this isn't really acknowledged properly on social media. I highly doubt the Twitter storm would have been this intense if the twitter posts explicitly acknowledged this as a "Draft publication which hints as X." But I admit that pointing fingers at nobody in general and social media specifically is a pretty lazy solution. The takeaway IMO seems to be to prepend the abstract with a clear disclaimer sentence conveying the uncertainty of the research in question. For instance, adding a clear "WORKING DRAFT: ..." in the abstract section.

deleted 3y ago
[deleted]
cscurmudgeon 3y ago
> The incentive to do this is to protect your idea from getting scooped, and The other side is flag planting with half-baked ideas and results.
- deleted 3y ago
  [deleted]
ttpphd 3y ago
I think you missed the point that data needs to be collected and presented ethically. It's not about it being a work in progress and not peer reviewed.
- muds 3y ago
  I agree that the data collection process wasn't ethical, and the professor should definitely be reprimanded for that. It's extremely sad that the coauthors weren't aware of this as well. And I feel terrible for the undergrads: their first research experience was publically rebuked for no fault of their own. However, there is no shortage of projects with sketchy data collection methodologies on arXiv that haven't received this amount of attention. The point of putting stuff on arXiv _is_ that the paper will not pass / has not passed peer review in its current form! I might even call arXiv a safe space to publish ideas. We all benefit from this: a lot of interesting papers are only available on arxiv v.s. being shared between specific labs. I'm concerned that this fiasco was enabled by this new paradigm in AI social media reporting, where a project's findings are amplified and all the degrees of uncertainty are repressed. And I'm honestly not sure how to best deal with this other than either amplifying the uncertainty and jankyness in the paper itself to an annoyingly noticeable level, or just going back to the old way of privately sharing ideas. Maybe this is the best case scenario for these sorts of papers? They pushed a paper on a public journal, and got a public "peer review" of the paper. Turns out the community voted "strong reject;" and it also turns out that the stakes for public rejection are (uncomfortably, IMO) higher than for a normal rejection. Maybe this causes the researchers to only publically release better research, or (more likely) this causes the researchers to privately release all future papers.

visarga 3y ago

There is an insightful video review of this paper by Yannic Kilcher: https://youtu.be/Tkijsu129M0?t=30 https://youtu.be/Tkijsu129M0?t=30 TL;DW - Their approach is to sequentially test methods, moving on to the next if one fails. However, this strategy is flawed as it requires ground truth, particularly for multi-choice answers. The analogy could be made to continually rolling a dice until landing on six. Similarly, if a question has four potential answers, the model merely has to attempt four times to stumble upon the correct response. And then they report 100% success rate.

az226 3y ago
Yes and no. Some LLMs can’t get the right answer no matter how many rolls of the dice they get. The bigger BS is that several questions weren’t questions and some didn’t have enough detail to be answered and it somehow got 100% on those. This IMO out the entire paper into the trash.
- caddemon 3y ago
  It graded itself, so that explains that lol

tinyhouse 3y ago

People overreact to this paper because it's from MIT. Lots of crappy papers out there, this is just another one. MIT is not special, they have a lot of average people too.

caddemon 3y ago
This was written by young undergrads with no actual research experience and a random visiting lecturer who is obviously trying to career climb (and had it backfire). Certainly there is bad research produced by MIT but I'd hardly call this "produced by MIT" in the traditional academic sense. It is extremely easy actually to get yourself some sort of affiliation. Perhaps they should crack down on that, but I think the relative openness of MIT is overall a good university culture to have, and only a small minority of those with tenuous affiliation are actually grifters.

data_maan 3y ago

Here's a more rigorous take on evaluating the output of GPT models on math: https://arxiv.org/abs/2301.13867 https://arxiv.org/abs/2301.13867

cscurmudgeon 3y ago
Unfortunately, negative but robust results like this won't get a fraction of the attention this MIT paper initially got.
ianbutler 3y ago
They don't specify GPT-4. The fact they generically refer to ChatGPT leads me to believe they're assessing 3.5. GPT-4 is significantly more competent of a model and while I won't speculate on it's math ability I'd want to see a follow up, specifically for GPT-4.
- Sunhold 3y ago
  Yup. Page 19: "we focused on the 9th-January-2023 version of ChatGPT"

kleiba 3y ago

Phew, I'm just happy that OpenAI first gathered consent from all data sources on which they trained GPT4.

ec109685 3y ago

Not condoning what the author did, but it’s amazing how little the questions answered in the paper have to do with real life software engineering. Yes, it shows fortitude to pass MIT’s exam, but beyond that, a lot of stress for a test that will soon be forgotten after their undergraduate degree.

PeterStuer 3y ago

One can only dream this level of analysis and falsification would be the norm in academic publishing. Sadly, this is an extreme outlier for now.

sgt101 3y ago

Well, someone's in trouble. If a faculty member allowed their name to be put on this thing then that's on them, if they didn't then whoever did the submission has to go. Also all the papers of all the co-authors who allowed their name on it are now suspect for me and should be investigated seriously. Sure, if the fact is that they said "don't put my name on stuff that you publish unless I have specifically agreed" or "don't publish this thing, it needs work" and then that message got lost - they are off the hook for me. But there needs to be a look at the culture of the labs involved even in that case. That's brutal because as I say there is a possibility that they are victims of someone going over the top, but unfortunately I think MIT has to do it. In addition, for me there is a much wider ethics issue here. How many papers can really be checked properly by someone claiming authorship even if that author isn't really contributing? I can read a paper like this in about a fortnight because I have a job and a life. A faculty member also has a job and a life - they are doing admin and teaching as well as research. So to me it's impossible for someone to check 25 papers or more a year. I am seeing far higher counts than this by many academics. But, this is an extremely conservative threshold in my opinion. When I have contributed to scientific papers it has taken me at least three months of solid work each time. Often these papers get rejected (rightly) and then have to be substantially amended (or occasionally just abandoned) I am not that talented for sure, but I really find it hard to credit that anyone with an actual job (so not a post-doc or a student) can contribute to more than one academic paper a year. Potentially two or three if there is a confluence of papers getting ready for print... but not on a sustained basis. There are two solutions. Every university and research institute needs to investigate all the publications of academics with high paper counts per year. This is a red flag. I definitely think that if folks are in the top quartile in a department it needs to be looked at carefully. The other solution is that no academic publishing venue (conference or journal) should accept more than one paper per year from any author.