5 ms·

What's the ratio of real citations summarized correctly vs syntactically plausible citations to nonexistent papers? Lawyers and judges have been getting in re

• Swizec 1y ago
  As of the last few months all citations were correct. I checked and obviously wouldn’t use any that I didn’t verify. I’m using chatgpt so whatever they’re doing is working
  • Kim_Bruning 1y ago
    4o or o3? Especially o3 seems to go all in on accurate citations.
• sixhobbits 1y ago
  This got a lot of press in the beginning and the raw models definitely still do it, but all the major players now use 'web search' or 'research mode', so they basically google 200+ pages and then read them all and keep the real links if they use them.
• jll29 1y ago
  I saw a paper that cite an author like this: "some metric (Smith 1971)". The author last name was correct, but the year cited should read "2004" instead. Turns out the Smith listed in the bibliography wasn't the smith that invented the 2004 metric from computer science for which he was cited, but another Smith with the same last name from the field of medicine, who wrote a very unrelated paper in 1971 that was referred to. I made the author aware (not mentioning any suspicions of potential LLM involvement...) by email and got told I wasn't the first one to point that out. Papers like that are now creeping into journals, conference proceedings and online archives, mostly unchecked/unnoticed, which waters down the quality a lot. PS: cited author's name changed.