4 ms·
Writing with LLM is like doing a mapping from human prompt -> document. The longer the prompt, the closer the output document to the human intent. If the prompt
by ozgung 1y ago
Writing with LLM is like doing a mapping from human prompt -> document. The longer the prompt, the closer the output document to the human intent. If the prompt is short (= human intent is low), the output must be filled with fluff, generated from the general knowledge/common sense in LLMs knowledge-base. If the prompt is very detailed, it's not much different than writing it yourself. Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains.
-> (GPT 4.5 "rewrite like a good editor")
Writing with an LLM can be viewed as translating human intent—expressed via a prompt—into a written document. The longer and more detailed the prompt, the closer the resulting document aligns with the original intent. Shorter prompts, indicating lower clarity or precision, compel the LLM to rely on general knowledge or common sense to fill in content, often resulting in fluff. On the other hand, excessively detailed prompts become nearly equivalent to writing the document yourself. Optimal productivity with LLMs occurs somewhere between these two extremes.
- yapyap 1y agoJust write it yourself man, if you have to reread everything spit out by an LLM closely for faults and all that why even bother. and if you aren’t proofreading, shame on you
- Swizec 1y ago> Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. A recent win for me has been to use LLMs for finding references. I’ll write a paragraph about one of those “everyone knows this” concepts or a “I’ve seen this be true dozens of times IRL”. But it could use a citation. So you go “Hey LLM, find me papers that support or refute this claim <paragraph>”. And it does. With links. It’s wonderful. Unlike Google there’s no recency bias, I’ve found original source blogs from 2005 (since requoted lots in SEO spam), papers about software engineering truths from the 1960’s (requoted ad nauseam by SEO spam), and sometimes even “Here are 2 papers that say you’re right and 3 papers that say you’re not”. Then I get to dig in and figure out who’s right. Easily 10x faster than doing the research myself.
- PaulHoule 1y agoWhat are you using? Microsoft Copilot frequently gives correct answers with totally wrong citations.
- Polizeiposaune 1y agoWhat's the ratio of real citations summarized correctly vs syntactically plausible citations to nonexistent papers? Lawyers and judges have been getting in real legal trouble from the latter.
- Swizec 1y agoAs of the last few months all citations were correct. I checked and obviously wouldn’t use any that I didn’t verify. I’m using chatgpt so whatever they’re doing is working
- Kim_Bruning 1y ago4o or o3? Especially o3 seems to go all in on accurate citations.
- sixhobbits 1y agoThis got a lot of press in the beginning and the raw models definitely still do it, but all the major players now use 'web search' or 'research mode', so they basically google 200+ pages and then read them all and keep the real links if they use them.
- jll29 1y agoI saw a paper that cite an author like this: "some metric (Smith 1971)". The author last name was correct, but the year cited should read "2004" instead. Turns out the Smith listed in the bibliography wasn't the smith that invented the 2004 metric from computer science for which he was cited, but another Smith with the same last name from the field of medicine, who wrote a very unrelated paper in 1971 that was referred to. I made the author aware (not mentioning any suspicions of potential LLM involvement...) by email and got told I wasn't the first one to point that out. Papers like that are now creeping into journals, conference proceedings and online archives, mostly unchecked/unnoticed, which waters down the quality a lot. PS: cited author's name changed.
- mlsu 1y agoNo, the prompt, no matter the amount of detail, is always strictly better than the model output. It has to be, because the whole point of reading and writing is _communication_. Communication between TWO people. This AI interlocutor is like a permanent cataract. It always makes it harder to see, never easier.
- HPsquared 1y agoIt's a translator. Very useful when the message needs to be in a specific format or if you're talking to a computer, or if you need help with "grammar" or "protocol" (in all its forms).
- jayd16 1y agoI don't think this is a great analogy. These things can pull in preexisting explanations and such. It doesn't just use the prompt so it's not a strict ceiling.
- goopypoop 1y agoMy butler is an excellent researcher but I think there's something wrong with him
- polynomial 1y agoHe might be a murderer, many such cases.
- PaulHoule 1y agoI get people writing comments on photos I post to social in languages that I have partial comprehension of such as Japanese and Portuguese. I use Copilot to translate their messages, ask specific questions about particular words, get explanations about idioms and supply context and ask it to translate my replies which I always translate back through another LLM to try to catch problems, sometimes I ask it to make an edit, sometimes I make an edit myself. Often I use LLMs to "have a conversation with a language" such as researching the cognate between the phrase "woo" to describe the supernatural in English and the similarly pronounced character 巫 (wu) in Chinese which is used in words like 女巫 (witch -- that first character means "woman") but it is not the "wu" in 武侠 (wǔxiá -- martial arts) I am sure one of these days I am going to embarrass myself but with partial comprehension I could do that without the help of an LLM.
- abujazar 1y agoGiving the LLM access to sources and enabling it to look up things that you would otherwise have to check yourself dramatically improves the usefulness and productivity, though.
- vouaobrasil 1y agoOn the other hand, looking things up yourself allows you to become quite familiar with the way things work in a more detailed way than just getting the answer through an LLM.
- GuinansEyebrows 1y agoyou give up understanding for... what, exactly? what do you gain from transitioning from active to passive participant in your communicative life?
- armchairhacker 1y agoI’ve seen good LLM summaries. LLMs are best at using words and phrases that “flow well”. Sometimes they write nonsense that flows well (even the summaries have to be checked because sometimes the meaning is different), but sometimes they find better words and phrases that I couldn’t find myself. They’re especially good if you have almost stream-of-consciousness writing and can’t bother to rephrase it yourself, e.g. you’re writing a boring technical report. Even for something where quality is important, like an epic story: you can write a very rough draft, revise with an LLM, then revise manually.