2 ms·
> Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. A recent win for me has been to use LLMs for finding refere
by Swizec 1y ago
> Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains.
A recent win for me has been to use LLMs for finding references. I’ll write a paragraph about one of those “everyone knows this” concepts or a “I’ve seen this be true dozens of times IRL”. But it could use a citation.
So you go “Hey LLM, find me papers that support or refute this claim <paragraph>”. And it does. With links. It’s wonderful. Unlike Google there’s no recency bias, I’ve found original source blogs from 2005 (since requoted lots in SEO spam), papers about software engineering truths from the 1960’s (requoted ad nauseam by SEO spam), and sometimes even “Here are 2 papers that say you’re right and 3 papers that say you’re not”. Then I get to dig in and figure out who’s right.
Easily 10x faster than doing the research myself.
- PaulHoule 1y agoWhat are you using? Microsoft Copilot frequently gives correct answers with totally wrong citations.
- Polizeiposaune 1y agoWhat's the ratio of real citations summarized correctly vs syntactically plausible citations to nonexistent papers? Lawyers and judges have been getting in real legal trouble from the latter.
- Swizec 1y agoAs of the last few months all citations were correct. I checked and obviously wouldn’t use any that I didn’t verify. I’m using chatgpt so whatever they’re doing is working
- Kim_Bruning 1y ago4o or o3? Especially o3 seems to go all in on accurate citations.
- sixhobbits 1y agoThis got a lot of press in the beginning and the raw models definitely still do it, but all the major players now use 'web search' or 'research mode', so they basically google 200+ pages and then read them all and keep the real links if they use them.
- jll29 1y agoI saw a paper that cite an author like this: "some metric (Smith 1971)". The author last name was correct, but the year cited should read "2004" instead. Turns out the Smith listed in the bibliography wasn't the smith that invented the 2004 metric from computer science for which he was cited, but another Smith with the same last name from the field of medicine, who wrote a very unrelated paper in 1971 that was referred to. I made the author aware (not mentioning any suspicions of potential LLM involvement...) by email and got told I wasn't the first one to point that out. Papers like that are now creeping into journals, conference proceedings and online archives, mostly unchecked/unnoticed, which waters down the quality a lot. PS: cited author's name changed.