7 ms·
Now ChatGPT is being (mis)used to do PeerReview
- uni_rule 4y agoFucked up considering the current tendency for GTP3 to be confidently / plausibly incorrect.
- quonn 4y agoHaving seen how superficial many reviews I got appear to be I wonder if it‘s not an improvement regardless.
- elashri 4y agoI would be more interested to see the reviewer comments to see how it would be possible to do peer-review with ChatGPT. I'm biased because of STEM background, which makes it nearly impossible to think of ChatGPT usage in that sense because it is very bad at Mathematics in general (even if GPT-4 improved it a lot with arithmetic and basic calculus). But it might be different for other fields, like in this case.
- SoylentOrange 4y agoThis seems much more like a problem for the journal and reviewer than it does about ChatGPT. I am a reviewer at several computer science publications. This behaviour is likely against the publication's code of conduct for reviewers. If the author complains to the chair then it is likely their paper will be re-evaluated and the reviewer banned from giving further reviews for that conference. Being a reviewer for a top conference is a highly sought-after honour so the chair knows that it's (a) better for the prestige of the publication if reviewers are discouraged from this type of action; (b) replacing any reviewer is easy as there are many qualified applicants to choose from. In the future, I anticipate that codes of conduct will more explicitly specify that while you may use automated tools to assist you, you must ultimately write your own review.
- auggierose 4y ago> Being a reviewer for a top conference is a highly sought-after honour Is it?
- noobermin 4y agoBeing a good figure in a conference planning is a sought honor. Reviewing is the grunt work you have to do in order to get that honor I guess, that's the thought process.
- SoylentOrange 4y agoIn the computer science academic sphere, the conferences at which one is a reviewer is among the top items on a CV. It implies that you are such a pillar and expert in that community that they would ask you to referee the merits of incoming academic work. Usually to be invited as a reviewer at a top venue, you should have a history of successfully having your work accepted to that venue. That's very difficult as acceptance rates for top CS venues are generally around 20%, and even experienced submitters often have papers rejected. My paper was actually recently rejected from Oakland (a top CS security venue).
- auggierose 4y agoNow things become clear! I always thought as an academic you should just read and evaluate those papers you are interested in. Which for me is really a minuscule proportion of all papers submitted. Maybe academia as a business isn't such a good idea.
- primitivesuave 4y agoThis is exactly what people should fear about the AI future. A lazy judge, parole officer, loan reviewer, school counselor, or in this case - peer reviewer, delegating decisions that affect human lives to a black-box algorithm.
- Thorentis 4y agoYep, this scares me greatly. I am already scared of the faceless, soulless, beauracracy, and this is just a whole other level.
- smcn 4y agoI've seen people use ChatGPT to make investment decisions. "AI says that $X is about to squeeze" or "AI says the market is going to crash on X". A huge lack of knowledge as well as an overconfidence about what this tech /actually/ is. This is going to be disruptive in the worst possible way.
- tuyiown 4y agoIMHO, anything touched by IA will be considered as a repulsive taint real quick. It has already happened in most concerned circles (artists). Using the statement «IA not used here» will be common, people will be held up to those word, and liars will found themselves under a vehement vindication.
- bryanrasmussen 4y agoThe Butlerian Jihad is not as bloodthirsty as I had imagined.
- raincole 4y agoI tried it several times. Sometimes it makes sense, but often it gives really bizzare advice. Like this: https://i.imgur.com/u7zyXUl.jpg https://i.imgur.com/u7zyXUl.jpg Even worse: https://i.imgur.com/FlIWtIT.jpg https://i.imgur.com/FlIWtIT.jpg (For people who don't like imgur: > Prompt: Given the following scenario: in the US, one US dollar is equal to 100 Japanese Yen, and in Japan, one US dollar is equal to 120 Japanese Yen. How do you make money with this fact? > ChatGPT: In this scenario, there is an arbitrage opportunity due to the discrepancy in exchange rates between the US and Japan. Here's a step-by-step process of how to take advantage of this situation: > * Start with a certain amount of USD, say $1,000. > * Convert the $1,000 to Japanese Yen in the US at a rate of 1 USD = 100 JPY. You would get 100,000 JPY. > * Transfer the 100,000 JPY to Japan. > * In Japan, convert the 100,000 JPY back to USD at a rate of 1 USD = 120 JPY. You would get approximately $833.33 (100,000 JPY / 120). > * Now, you have $1,000 in the US and $833.33 in Japan, totaling $1,833.33. )
- a_bonobo 4y agocan we move to post-publication peer review now? Pre-publication peer review, most of the time, marginally improves a manuscript at the cost ~6 months. Some times it's just a place for reviewers to vent off stress and punch down. The real corkers get weeded out by the editors before the paper goes out to review.
- tgv 4y agoI agree that it's wrong when peer reviewers abuse their position (and they certainly do), but pre-publication review keeps the number of publications down, which once was necessary: a journal has a limited number of pages. You could argue that is a thing of the past, but imagine every article being published. We're already drowning in a sea of mostly wrong articles.
- a_bonobo 4y agoover the past 5-10 years my field biology has embraced preprints, with some famous papers remaining as preprints (most famously, I'd say, is the BWA-MEM paper). Most preprints I see are of decent quality, certainly higher than what gets published in MDPI et al. Every article is already getting published and the world hasn't ended yet :)
- akasakahakada 4y agoIncreasing number of researchers ar e now citing papers directly from arxiv instead of some xxx society or xxx university journal.
- akasakahakada 4y agoSorry but science community need more papers indeed. For example only successful novel results are being published, but not those others who are doing validation jobs. Although some are frauds, but their theory is new, they get passed. No one bother to test the theory because no journal will accept such boring validation paper. Last thing to mention is that the count of pages do not matter. It is not an excuse to refuse to publish great work from scientist.
- justusw 4y agoThis completes the cycle. Researchers use GPT to write papers and submit them (can be fully automated) Journals review using GPT and (potentially) publish (can be fully automated) GPT will create a borked summary and create fake citations for the next cycle of research. (again, can be fully automated) Then, left on its own, we will have a bunch of robots worrying for us about impact factor and grant applications and we can get back to doing actual research again. Problem solved.
- tankenmate 4y agoIt doesn't take much imagination to realise that an AI generated echo chamber will be far more capable than a human echo chamber; a vicious unvirtuous circle. At least some of the newer AI detector tools might help here.
- carlmr 4y ago>At least some of the newer AI detector tools might help here. Isn't this just GAN training via the paper publishing industry
- mort96 4y agoWhat do you mean by "capable"? From everything I've seen, all these GPTs are extremely hilariously terrible at anything resembling logical reasoning, to the point where it has absolutely no trouble saying "1 pound of A is heavier than 2 pounds of B because A is denser than B" for the right values of A and B.
- pjc50 4y agoCapable at the thing which matters, which is getting funding by gaming grant metrics that only look at the (easily measurable) quantity of citations and not the (impossible to measure) quality.
- quickthrower2 4y agoSounds like an XKCD plot
- Zetobal 4y ago/r/thishappened
- gandalfgreybeer 4y agoI have played around with ChatGPT and noticed that it "invents" papers and references. But yeah, I feel like it is unlikely (not impossible) that a peer reviewer would actually not even check the outputs.
- busymom0 4y agoJust yesterday, I asked it a question about whether a certain company said something specifically and first it said no. Then I asked it if the company said that in its SEC filings and then it said yes. Then I pointed out to it that it’s answer contradicts its previous answer and it apologized. Then I asked it for a link to the sec filing and it gave me the wrong link. Another example was where it said that a certain study found a specific thing. I asked it for a link to the study and it responded that it never referred to any study. So I pointed out that it did refer to a specific study and it apologized again and gave me the wrong link. Overall, it’s almost like it pretends to know more than what it knows but is overconfident.
- mb_72 4y agoSounds quite human.
- gandalfgreybeer 4y agoI think the new Bing AI chat works better (for the most part) for stuff that are more "fact-based" especially since it cites references better. Note that you have to check the references too since sometimes there are mistakes. I would avoid ChatGPT for stuff like "Can you recommend me papers on machine learning for specific problem/task" since the model sort of makes up stuff. The main value of ChatGPT for me (so far) is not for tasks where I get information from it but where it gives me information to start something. "Write me an authorization letter", "create a 10-slide presentation on this topic", or "I have this function, can you write some unit tests" has been very useful for me. Not because the outputs are correct but they lower the inertia for me working on tasks. I rarely get outputs that work 100% out of the box that I don't have to fix or reshape. One personal rule is that I never ask it stuff that I normally won't be able to do by myself.
- tokai 4y agoThis changes nothing from already bad peer review. Actually it might be easier to get the editor to stop using that reviewer if you can prove the use of chatgpt. Opposed to the more standard "this work fails to cite my prior work and should totally give me a shout if I'm to accept the paper".
- bobmaxup 4y agoThis is unfortunate. This level of NLU is no where near where it needs to be to allow such an application. You can play along with feeding any decent length open novel into GPT-x, and then tossing SAT-like reading comprehension questions at it. It is interesting to see where it fails. I don't think that OpenAI is denying this either. The future is bright, but cheaters are going to find rough waters alongside failing mimicry.
- dgut 4y agoPeer-reviews have been broken way before GPT. People are expected to do it for free in a publish or perish academia...
- webscout 4y ago[dead]
- datadeft 4y agoAre we blaming the tool here? At what extent societies are ok with tool blaming? - should we ban knives? - should we ban guns? - should we ban LLMs?
- amelius 4y agoYou should add: - should we ban nuclear weapons for personal use? And in the future: - should we ban DIY virus printing technology. They are just tools, after all. What could go wrong?
- gjulianm 4y agoAt the risk of sounding like a Reddit comment, what's the difference? I've had reviews on papers of people that didn't even skim the content. I've had rejections from people who barely knew anything about the topic. I've had reviewers insist we should cite certain papers (funnily enough, all from the same lab even though they weren't really relevant). The peer review process is absolutely broken. There are no incentives to do good peer review, there are perverse incentives to stop "competing" papers from being published or modified with convenient citations. Adding ChatGPT to this process doesn't really change anything. It could even improve it: it'd be a crappy reviewer but at least you can test yourself what it's going to say so you can avoid the 6-12 months waiting period for a review.
- bowsamic 4y agoYeah I am literally about to review a paper which I definitely don't understand well enough to review. I will probably just write something very vapid and brief. Half tempted to use GPT, but I'm a good enough writer that I can make it up.
- amelius 4y agoAs opposed to being honest and trying to understand the paper, and if you fail then writing in your review that it is too difficult to understand?
- bowsamic 4y agoUnfortunately both my mental health conditions and time constraints are such that I don't have the ability to spend time to understand the paper right now
- janeway 4y agoThen don’t review it. I spend months or years writing papers. A voluntary reviewer who won’t bother reading is not helpful.
- jonathanstrange 4y agoI've heard from people in contact with Springer that they want to add an AI-based review in addition to human reviewers (first) for every journal. It's from hearsay, so feel free to correct me if you're working for Springer. In any case, I'm in the humanities where this technology can and probably will be used, and it's going to accelerate a race to the bottom that already started a while ago due to publication pressure, incorrect use of indicators for evaluation, and deficiencies of the review system itself. AI is particularly bad at evaluation papers in the humanities because it is trained to value unoriginal thought and the status quo more than originality and constructive work.
- akasakahakada 4y agoThe problem with humanities as a field is that they made ChatGPT shifted to ignore scienctific method in order to protect pseudoscience. If you have ever asked ChatGPT to say anything against or criticize the humanties studies, it just can't. Not even DAN can get rid of the humanities-protectionalism. You guy especially those AI ethics dudes just totally corrupted it.
- jonathanstrange 4y agoI don't think the AI ethics dudes corrupted the field specifically, they tend to be on the more reasonable side. But I generally agree. I've worked in philosophy for 30 years, half of those post-PhD, and the field is getting worse and worse. I've seen people rise to the top who are, frankly speaking, fairly dumb and often not even well-read, but have published a lot of books (there is almost no peer reviewing on books) and articles in bad journals. There are also certain topics you simply cannot work on unless you're fulfilling certain dogmatic expectations. However, I must add, in all fairness, that philosophy has some of the highest standards in the humanities and related disciplines are much worse. It's true that once you have established a line of study you just can't get rid of it any more. For example, someone started Argumentation Studies at our university, which is definitely a pseudo-science, and it's going to stay there forever, bound to spoil generations of students and agitate them against STEM fields. To get back to the original topic, however, I can't see any possible future in which AI reviewing wouldn't make the situation worse in the humanities than it already is.
- pulvinar 4y agoHave people checked to see if this story is real or current before commenting? It’s using ancient GPT-2.
- Veen 4y agoI assume the review references are false as claimed, but I'm not sure I'd rely on that outdated GPT-2 detector to prove their origin. More recent AI detectors aren't reliable on large chunks of prose; they're likely to be even less so on small chunks of text with very strict formatting requirements.