13 ms·
MIT asks arXiv to withdraw preprint of paper on AI and scientific discovery
- pvg 1y agoThe paper had an HN thread a few months ago https://news.ycombinator.com/item?id=42115310 https://news.ycombinator.com/item?id=42115310
- tsurba 1y agoNice that someone realized then already it sounds sus https://news.ycombinator.com/item?id=42128532 https://news.ycombinator.com/item?id=42128532
- mmooss 1y agoThat's not a signal: There always are comments saying the research is suspect.
- jwilber 1y agoTaking a contrarian stance on everything (exactly what you’re doing here) is not a signal. Explaining why research seems sus (what the comment does) is some form (strength proportional to the explanation) of signal.
- mmooss 1y ago> Taking a contrarian stance on everything I pointed out one issue with the GGP. That's not contrarian and not about everything.
- throwaway290 1y agoWhat was the one issue that you pointed out? That it is somehow NOT nice when someone notices when a paper sounds sus? Or that existence of negative comments is "not a signal" when it clearly is? None of it makes any sense
- mmooss 1y agoIt's not hard to parse my short comment. > it is somehow NOT nice Who said that? How could you possibly get that out of my short comment? Are you just making things up? Then what you say also becomes just noise, not a signal. > existence of negative comments is "not a signal" when it clearly is It's clearly not and I demonstrated why. What is your argument against my reasoning? A fundamental of information theory (or decision theory) is that if something gives you the same response every time, it's noise and not signal. If the weather forecaster always predicts it will rain, they aren't 'correct' when it rains; their predictions are just noise. If your coworker criticizes everything you do, their criticisms aren't signals about your work, they are just (annoying) noise. It's no more meaningful than water dripping on a drum.
- throwaway290 1y ago... It can't be the same every time when the entire comment consists of specifics only concerning one this paper.
- dang 1y agoThanks! Macroexpanded: Artificial Intelligence, Scientific Discovery, and Product Innovation [pdf] - https://news.ycombinator.com/item?id=42115310 https://news.ycombinator.com/item?id=42115310 - Nov 2024 (47 comments)
- intoamplitudes 1y agoFirst impressions: 1. The data in most of the plots (see the appendix) look fake. Real life data does not look that clean. 2. In May of 2022, 6 months before chatGPT put genAI in the spotlight, how does a second-year PhD student manage to convince a large materials lab firm to conduct an experiment with over 1,000 of its employees? What was the model used? It only says GANs+diffusion. Most of the technical details are just high-level general explanations of what these concepts are, nothing specific. "Following a short pilot program, the lab began a large-scale rollout of the model in May of 2022." Anyone who has worked at a large company knows -- this just does not happen.
- pixl97 1y ago>The data in most of the plots (see the appendix) look fak Could a Benford's Law analysis apply here to detect that?
- constantcrying 1y agoHow would you apply it, why would it be applicable?
- tough 1y agoFake data is usually too clean
- constantcrying 1y agoAnd? How is that at all a relevant observation?
- tough 1y agoAnyone looking at inly the data objectively should br able to comento terms that is distribution is unnatural, as it turns out to fake organic isnt as easy
- gowld 1y agoBad OP Title Better title: MIT disavows heavily-discussed economics preprint paper about Artificial Intelligence and Scientific Discovery.
- jdhwosnhw 1y agoIt’s the title of the original article…
- dang 1y agoI've attempted to put a neutral title at the top of this page. If someone can come up with a better (i.e. more accurate and neutral) one, we can change it again. (Since press release titles about negative news tend to studiously avoid saying anything, we tend to classify them in the "misleading" bucket of https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html, which justifies rewriting them.)
- forgotpwd16 1y agoPerhaps replace "take down" with "withdraw" (arXiv's mechanism to deal with bad papers post-publication; what MIT calls for) or "retract" (the mechanism that traditional journals employ and similar to previous; a common term in academia). In arXiv's way of handling papers, "removal" and "withdraw" are distinct[0] and, based on some comments in this thread, current title seems to create confusion that is about the former. [0]: https://info.arxiv.org/help/withdraw.html https://info.arxiv.org/help/withdraw.html: "Articles that have been announced and made public cannot be completely removed. A withdrawal creates a new version of the paper marked as withdrawn."
- ipsum2 1y agoMIT's article is quite scant on details. WSJ has more information, but still no specifics: https://www.wsj.com/tech/ai/mit-says-it-no-longer-stands-behind-students-ai-research-paper-11434092 https://www.wsj.com/tech/ai/mit-says-it-no-longer-stands-beh... > The paper was championed by MIT economists Daron Acemoglu, who won the 2024 economics Nobel, and David Autor. The two said they were approached in January by a computer scientist with experience in materials science who questioned how the technology worked, and how a lab that he wasn’t aware of had experienced gains in innovation. Unable to resolve those concerns, they brought it to the attention of MIT, which began conducting a review.
- neonate 1y agohttps://archive.ph/r63jR https://archive.ph/r63jR
- timewizard 1y ago[flagged]
- in9 1y agohttps://en.wikipedia.org/wiki/List_of_Nobel_Memorial_Prize_laureates_in_Economic_Sciences https://en.wikipedia.org/wiki/List_of_Nobel_Memorial_Prize_l...
- throwup238 1y agoNobel _Memorial_ Prize in Economic Sciences That’s not a Nobel Prize.
- bee_rider 1y agoIf that’s the distinction, it would have been helpful for the original comment to note that they were just sharing some silly trivia instead of making a point.
- 1y ago
- ayhanfuat 1y ago> by a former second-year PhD student Seems pretty serious if they kicked him out.
- dhosek 1y agoI always wonder what happens with these high-profile transgressors. I once created a Google News alert for a high-level Apple employee who went to jail for some criminal act at Apple and never saw any indication of him again. I’m guessing his career in economics is likely over (he’d previously worked at the NY Fed before starting at MIT) and I wonder what he’ll end up doing—will he be able to find some sort of white-color work in the future or will he be condemned to retail or food-service employment.
- NitpickLawyer 1y ago> will he be able to find some sort of white-color work in the future or will he be condemned to retail or food-service employment. Lay low for a year, work on some start-up-ish looking project, then use his middle name to get hired at one of the many AI startups? (only half joking)...
- Loughla 1y agoWhite collar encompasses a lot, outside of economics or finance. Also, there are companies who will see that win at any cost mentality as a positive trait. I'm betting whoever it is, is okay now.
- jasonfarnon 1y agoOr, as in the case of the LaCour/UCLA kid, a lot of outfits will agree with the ends if not the means. Still, getting caught doing this has to close 100 doors for every 1 door it opens.
- Aurornis 1y agoThe MIT announcement says they asked him to retract the paper but he wouldn't, which led to them making the public statement about the paper. They may have thought they could jump into an industry job, including the paper and all of its good press coverage on their resume. Only the author can retract an arXiv paper, not their academic institution. It wouldn't be hard to come up with a story that they decided to leave the academic world and go into industry early. MIT coming out and calling for the paper's retraction certainly hampers that plan. They could leave it up and hope that some future employer is so enamored with their resume that nobody does a Google search about it, but eventually one of their coworkers is going to notice.
- 12_throw_away 1y agoReading the paper (which is still up) ... the "AI" (sigh) tool described there would not have been particularly novel or unusual, even if the research was conducted several years ago. ML + inverse design for materials has been used for decades.
- Aurornis 1y ago> Our understanding is that only authors of papers appearing on arXiv can submit withdrawal requests. We have directed the author to submit such a request, but to date, the author has not done so. Between this and the subtle reference to “former second-year PhD student” it makes sense that they’d have to make a public statement. They do a good job of toeing the required line of privacy while also giving enough information to see what’s going on. I wonder if the author thought they could leave the paper up and ride it into a new position while telling a story about voluntarily choosing to leave MIT. They probably didn’t expect MIT to make a public statement about the paper and turn it into a far bigger news story than it would have been if the author quietly retracted it.
- JohnKemeny 1y agoSeeing as how the author has signed in with an account whose email address is username@mit.edu, MIT could just take over the account. Edit: this comment was only partially serious, not meant as legal advice to MIT.
- kragen 1y agoThat kind of thing might lead to arXiv not accepting any more papers from MIT, or at least any more takedowns.
- Aurornis 1y agoThat's not how it works in the real world. That would be a fraudulent request and I suspect they'd invite legal trouble by impersonating someone else to access a computer system. Furthermore, if the author could demonstrate to arXiv that the request was fraudulent, the paper would be reinstated. The narrative would also switch to people being angry at MIT for impersonating a student to do something.
- NoMoreNicksLeft 1y ago>That's not how it works in the real world. That would be a fraudulent request and I suspect they'd invite legal trouble by impersonating someone else to access a computer system. Emails are not people. You can impersonate a person, but you can't impersonate an email. If I own a company and I issue the email dick.less@privateequity.com but then have to fire him... using this email address to transfer company assets back to someone who can be responsible for them isn't fraud (for that purpose, at least). How is this not the same issue?
- tokai 1y agoImpressively the paper seems to have been cited 50 times already. I don't mind much if its taken down or not, but with the old guard publishers you can at least get a redaction notice or comment about the issues with a paper embedded in the publication. If you find this paper cited somewhere and follow it to the source at arxiv, you will never be made aware of the disputes surrounding the research. Preprint servers has somewhat of a weakness here.
- forgotpwd16 1y agoA weakness that goes hand-in-hand with the lack of peer review. There's moderation but shouldn't be considered equivalent to it. Trusting the study means trusting the author or reviewing the paper yourself. If a withdraw happens, either the author comments on why they did it[0] or, similarly to previous, you've to search it yourself. [0] E.g. arxiv/0812.0848: "This paper has been withdrawn by the author due to a crucial definition error of Triebel space".
- Majromax 1y ago> A weakness that goes hand-in-hand with the lack of peer review Peer review is not well equipped to catch fraud and deliberate deception. A half-competent fraud will result in data that looks reasonable at first glance, and peer reviewers aren't in the business of trying to replicate studies and results. Instead, peer review is better at catching papers that either have internal quality problems (e.g. proofs or arguments that don't prove what they claim to prove) or are missing links to a crucial part of the literature (e.g. claiming an already-known result as novel). Here, the value of peer review is more ambiguous. It certainly improves the quality of the paper, but it also delays its publication by a few months. The machine learning literature gets around this by having almost everything available in preprint with peer-reviewed conferences acting as post-facto gatekeepers, but that just reintroduces the problem of non-peer-reviewed research being seen and cited.
- gus_massa 1y agoMost of the 50 citations are preprint servers (like arXiv) or aggregators (like researchgate). It would be nice to count the number of citations in research papers in peer review journals. > you will never be made aware of the disputes surrounding the research ArXiv is not peer review, so it's as confiable as Wordpress or Medium or Blogspot or X/Tweeter or ... The main difference is that the post is in PDF instead of HTML. There is an invitation system to avoid very stupid cases, but it's very week. I remember a weird cryptography "breakthrough", and they published it in arXiv, and the first 5 pages were an explanation of the rule of 9 for divisibility and then a dubious fast factorization algorithm. [The preprint was linked in https://news.ycombinator.com/item?id=20666501 https://news.ycombinator.com/item?id=20666501 and I replied to it (s/module/modulo/g)]
- coderintherye 1y agoNice Twitter thread from Nov '24 analyzing the paper: https://x.com/Robert_Palgrave/status/1856273405965693430 https://x.com/Robert_Palgrave/status/1856273405965693430
- andy99 1y agoIs there a way for not Twitter users to read these?
- madars 1y agoYeah, replace x.com with xcancel.com or nitter.poast.org, e.g., https://xcancel.com/Robert_Palgrave/status/1856273405965693430 https://xcancel.com/Robert_Palgrave/status/18562734059656934...
- genewitch 1y agoWhile poast works, I strongly feel most people on this site are not aligned with what poast represents. Graf has to spend a lot of time (and time = money) getting accounts set up to run nitter, and it would suck if the views of poast users wound up costing the internet a twitter mirror. I strongly recommend people not investigate this unless they think 4chan is quaint. As in, if the reason you are not using X is because of the outrage at Elon and the "typical user" of X, then maybe use xcancel instead.
- kurisufag 1y agofwiw i believe strongly that there would not still be a nitter without graf's work on it. in the space of working on cool shit, he's good people. see: moldbug and urbit, for a similar case.
- genewitch 1y agooh, totally. I was hoping to avoid people going "i wonder what poast is", but Graf said they've been backlinked before and it's no big deal; but i can't remove my earlier comment. So i guess i misread!
- hooloovoo_zoo 1y agoI don't think arXiv should take it down even if it is fraud. ArXiv is more about being a permanent store than a quality judge.
- modzu 1y agostore of what? fake scientific articles or genuine preprints? if the latter clean this crap up
- pbhjpbhj 1y ago>Earlier this year, the COD conducted a confidential internal review based upon allegations it received regarding certain aspects of this paper. While student privacy laws and MIT policy prohibit the disclosure of the outcome of this review, we are writing to inform you that MIT has no confidence in the provenance, reliability or validity of the data and has no confidence in the veracity of the research contained in the paper. Based upon this finding, we also believe that the inclusion of this paper in arXiv may violate arXiv’s Code of Conduct. It sounds like "we don't like it and won't tell you why, we're hiding behind MIT policy and vague notions of privacy". MIT should just demonstrate in a paper what the shortcomings are and print it, adding it to the citation tree of the original. Looking very briefly at the paper and speculating wildly, I could imagine that the company who were subject of it - or their staff - might not appreciate it and have put pressure on MIT?? Solid amount of Streisand Effect going on here -- lots of attention has been bought to the paper (and that is everything after all!).
- shkkmo 1y ago> Looking very briefly at the paper and speculating wildly, I could imagine that the company who were subject of it - or their staff - might not appreciate it and have put pressure on MIT?? The apparent issue is that the data appears to have been entirely fabricated and is a lie. The author appears to simply be a fraud
- duskwuff 1y ago> It sounds like "we don't like it and won't tell you why, we're hiding behind MIT policy and vague notions of privacy". FERPA is federal law. It is quite likely that MIT is legally bound to not release some pieces of evidence which are crucial in this case (hypothetically, for example: that the student's educational record is inconsistent with claims made in the paper).
- dougb5 1y agoThis makes me think about the credibility of single-author vs. multi-author papers in different disciplines. In computer science, a paper is seen as suspicious if there's just one author (at least nowadays). But in economics it seems much more common. Can an economist explain this for me (or perhaps a paper written by multiple economists?)
- type0 1y ago> But in economics it seems much more common non-scientific studies can't be replicated
- als0 1y agoWhat’s that got to do with the number of authors?
- forgotpwd16 1y agoA study that cannot be replicated is a study that cannot be falsified. Authors don't mind putting their names on them because there's no accountability to be held and is purely net positive (one more publication and additional citations).
- mizzao 1y agoWhile in CS multi-author papers are the norm, in econ it is quite common to have solo-authored papers, with senior PhD students solo-authoring papers without their advisor as a coauthor. And it is rare to see an econ paper with more than 3 authors. As a CS PhD who has worked with many economists, my understanding is that the culture sees it as diluting credit, and so might put people in the acknowledgements where computer science folks would add them as authors.
- shanemhansen 1y ago"I don't endorse this paper. Therefore you should take it down. I won't tell you why. Trust me bro." Whether MIT is right or wrong, the arrogance displayed is staggering. The only thing more shocking is that obviously this behavior works for them and they are used to people obeying them without question because they are MIT.
- levocardia 1y agoMore like "Because it involves a student, FERPA won't allow us to legally disclose what's going on, but we kicked the student out so you should take the hint and realize what was going on"
- ricksunny 1y agoIt is indeed disappointing posture the institute is putting on full display here.
- azinman2 1y agoThey become aware of academic dishonesty where a student tried to publish a paper with faked data, and taking it down is disappointing posturing? What?
- ricksunny 1y agowhich becomes part of the record. Can put a banner up top. Expunging takes it out of the record, folks can't trace the history of it. Also, I didn't say 'posturing' which has a particular meanring, I said it's 'a posture'.
- roywiggins 1y agoRetracting a paper is basically putting a big RETRACTED banner on it, yeah, which is what MIT is asking the author to do.
- varelse 1y ago[dead]
- RS-232 1y ago[flagged]
- elcritch 1y agoIn my opinion the paper shouldn’t be take down. Instead a note should be added noting the concerns with the pre-print and that’s it’s likely fraudulent. Edit: Since the paper has been cited, others may still need to reference the paper to determine if it materially affects a paper citing it. If the paper is removed it’s just a void.
- aaron695 1y ago[dead]
- azinman2 1y agounless arXiv has a "there used to be a paper here, but it was retracted" page
- jinseokim 1y agoThat's what happens when a paper is withdrawn [1], and MIT requested to withdraw the paper [2]. This news title saying that they requested to take down the paper is subtly incorrect. [1]: https://info.arxiv.org/help/withdraw.html#:~:text=Previous%20versions%20remain%20accessible. https://info.arxiv.org/help/withdraw.html#:~:text=Previous%2... [2]: https://economics.mit.edu/news/assuring-accurate-research-record#:~:text=MIT%20respectfully%20request%20that%20the%20paper%20be%20marked%20as%20withdrawn https://economics.mit.edu/news/assuring-accurate-research-re...
- ugh123 1y agoI agree, the offense should have a public trail. But there should be safeguards to prevent any citing of fraudulent paper, not allowing for bits and pieces to outlive the offense. Citing papers should be marked with a warning until resolved by their authors.
- rdtsc 1y ago> The paper was championed by MIT economists Daron Acemoglu, who won the 2024 economics Nobel, and David Autor. The two said they were approached in January by a computer scientist with experience in materials science who questioned how the technology worked, and how a lab that he wasn’t aware of had experienced gains in innovation. Unable to resolve those concerns, they brought it to the attention of MIT, which began conducting a review. So the PhD student might have been kicked out. But what about the people who "championed it". If they worked with the student, surely they might have figured out the mythical lab full of 1000s material scientists might not exist, it might exist but they never actually used any AI tool.
- raphman 1y agoApparently, none of the 21 people mentioned in the acknowledgments questioned the source of the dataset. One of them also wrote a quite popular Twitter thread about the research. When notified of the recent events, he curtly replied that "It indeed seems like the data used in the paper is unreliable." [no need to mention them by name, I think]
- a_bonobo 1y agoThis happens again and again in research - I'm just reminded of the stem cell scandal around Obakata. Before the fraud was uncovered: dozens of senior researchers supporting the research, using the glory for their own gains. After the fraud is exposed: nobody wants to have been involved, it was only that one junior person. It doesn't excuse the fraud of the junior person but it makes you think how many senior-level people out there are riding similar fraudulent waves, doing zero checks on their own junior people's work.
- tanewishly 1y agoMost of the processes surrounding and supporting science are not robust against a dedicated adversary seeking to exploit the system. This is nothing new - Newton ordered and then wrote the anonymous report commissioned by (iirc) the Royal Society to decide who invented calculus, him or Leibniz. Basically, science is quite vulnerable to malicious exploiters. Part of this is because society isn't funding science anywhere near sufficiently to do a priori in-depth checks. You claim you got data on hundreds of measurable thingies in a certain way (from surveying people to scanning the web to whatever)? If it's not blatantly obviously a lie, it'll probably be accepted. Which is inevitable: at one point, you're going to have to accept the data as genuine. If there's no obvious red flags, you'd only waste time on further checking data - you'd need to do a real deep dive (expensive time-wise) to come up with circumstantial evidence that may still be explainable in a benign manner. For scientists, it is almost always more profitable to spend such time investments on furthering their own scientific efforts. So yes, there are various ways in which someone willing, dedicated and sufficiently skilled can "Nigerian-Prince" the scientific process. Thankfully, the skill to do so typically requires intimate knowledge of the scientific process and how to conduct research -- this cheating is not easily accessible to outside bullshitters (yet).
- deleted 1y ago[deleted]
- jrochkind1 1y agoyou'd think that such a widely cited fraudulent paper might have caused problems in other research, but probably nobody who cited it actually read it anyway, so. it's turtles all the way down.
- generationP 1y agoThat looks like the kind of paper that causes companies to lose lots of money by hyping up what is likely a less-than-impressive method. But it does not make any theoretical claims, so it cannot contaminate research.
- jrochkind1 1y agoI wonder why it is so cited in the literature.
- griffzhowl 1y agoAI hype, a write-up in WSJ with superlative comments from famous MIT profs. Should do it. Most of the citations are to other papers on preprint servers and I guess a lot of people were working on similar themes and added it as a reference
- jrochkind1 1y agoSo, indeed, I'd guess cited it without reading it.
- bigbacaloa 1y ago[dead]
- nikolayasdf123 1y ago> paper should be withdrawn from public discourse. just post a correction notice on arXiv. let others decide if there is merit to it or not. silencing is so anti-science. shame on MIT.
- 1024core 1y agoLink to paper: https://arxiv.org/abs/2412.17866 https://arxiv.org/abs/2412.17866
- FilosofumRex 1y agoMIT is hiding its own culpability by throwing the Student under the proverbial bus. Acemoglu and Autor who are notorious attention seekers and very media savvy and wealthy profs had vouched for him. There is no way a 2nd year PhD students could have pulled this off on his own without a trace of his whereabouts and contacts in the industry. A cursory review of the first paragraph of the abstract of his single author paper should've set off alarms: "AI-assisted researchers discover 44% more materials, resulting in a 39% increase in patent filings and a 17% rise in downstream product innovation". Anyone with rudimentary familiarity with industrial materials science research would have suspected those double digit numbers - even single digit improvements are extremely rare.
- mizzao 1y agoThis is a recording of a seminar on this paper the author gave via zoom: https://cassyni.com/events/MiPYGu3qzKP5MQFWNUn9Tb https://cassyni.com/events/MiPYGu3qzKP5MQFWNUn9Tb In retrospect, there seems to be a tell that when he's lying he won't look at the screen/camera: his eyes go up, left, right, anywhere but forward. What I find scary is that this practice of extemporaneous fabrication may be a well-ingrained habit at this point that isn't limited to the scientific realm of the author's life.
- ok_dad 1y ago> his eyes go up, left, right, anywhere but forward In this case perhaps that's a tell, but there are a lot of people who don't like looking people in the eyes for other reasons, too, just so you're aware.
- CSMastermind 1y agoAnyone who is a practiced liar will be able to look you dead in the eyes while they do it.
- Loughla 1y agoThat's not really science though. Body language in general, and eye placement specifically is my most hated pseudoscience. I struggle to make and maintain eye contact, because bad childhood trauma. Eye contact was perceived as aggression and disrespect by the man who lived in my house, and would lead to beatings. There are a lot of us that struggle with that, for many different reasons.
- Syzygies 1y agoIf I've learned one thing using AI, it's that other people's experiences are not particularly relevant. So this paper is a meh! for me whatever its provenance. Compare the failure rates for condoms.
- deleted 1y ago[deleted]
- tomrod 1y agoI wonder what the issue was.
- eterm 1y agoInteresting, the paper got immediately flagged as suspicious by HN commenter lysecret: https://news.ycombinator.com/item?id=42128532 https://news.ycombinator.com/item?id=42128532
- yolkedgeek 1y agoSo MIT is trying to tell us that there was no incentives, no VC money, no corruption involved here and just a naughty naughty PhD student? This is how I see this: They published a false paper, knowing they would eventually be caught. But the boost that the paper had in AI marketcap and market bullishness and hype, far outweighs the consequences of this apology(not really) statement. So they got a lot of money for hyping the AI, and now they pay off a small amount of it as backlash. still a huge net income. Pharma companies do this all the time. this is how they make money. Reminder that if you link articles for your argument, doesn't necessarily mean that you are right. There are substantial amounts of false and slightly distorted papers and articles out there, even from the most valid publishers.
- beowulfey 1y agoI work in academia. There is no way anything is so organized and directed as that. That would require laughable levels of efficiency and planning across multiple organizational layers. Do you know how long it takes to make literally any decision? Who would even be directing such a scheme? Are the president and this PhD student in cahoots on this insider trading scheme you are envisioning?
- bigbacaloa 1y ago[dead]
- varelse 1y ago[dead]