3 ms·

I’ve tried to use flagship models to produce long-ish technical writing content and the result has not been satisfactory. The resulting content lacks consisten

by mccoyb 1y ago

I’ve tried to use flagship models to produce long-ish technical writing content and the result has not been satisfactory. The resulting content lacks consistency and coherency. The generated prose is rather breathless: in carefully articulated content, every sentence should have a place. Flagship models (Opus 4~) don’t seem to understand the value of a sentence. I’ve tried to prompt engineer this behavior (one should carefully attend to each sentence: how is it contributing to the technical narrative in this section, and overall?), but didn’t have much success. I suspect this might be solved by research on grounding generation against world models: much of verifying “is this sentence correct here?” has to do with my sharing of a world model of some domain with my audience. I use that world model to debug my own writing.

melvinroest 1y ago
Maybe do a new prompt at paragraph level?
fredrikholm 1y ago
I was born in 1990, quite late into the end tail of ordinary people not writing letters by hand, reading (a single book/article) for hours on end etc. and it's already evident that people outside of occupations that demand it just don't consume or produce as much (quality) writing as they used to. My fear isn't that LLMs will fail to meet our standards, my fear is that LLMs will drag us down to a new low of homogenized, dull and lifeless form of writing. Perhaps I'm biased, but it feels as if the few times I am impressed by someones writing, the material is 20 years old or older, even if said author is still producing work.
- mccoyb 1y ago
  I think you're accurately describing a "regression to the mean" phenomenon, which LLMs have transmitted to areas involving programmatic control through text. I recently used an LLM to help scaffold a piece of technical writing. I showed it to several of my peers (sharp and accomplished PhD students), and they immediately identified the paragraphs (even sentences in captions!) written by an LLM. I had spent a good amount of time prompt engineering Opus 4 to try and produce a coherent piece of content aligned with a narrative which I had already developed, but it was still _immediately obvious_ what was not consistent ... it stood out like a sore thumb to experts. By the the time we finished the content, there was not a sentence of LLM-generated writing in the final product. Possibly an incorrect extrapolation, but I think my experience here is the current status quo. Code is a bit easier for these systems, because the LLM can align against test suites, type systems, etc. But for writing ... you just can't trust the current capabilities to pass the sniff test by experts. After all, how does one commmunicate "coherence" as a test which the LLM might align its generations against? That's why you need a shared world model (with your audience) -- and if we could produce a computational representation of such a thing, we might have a chance at better coherent / consistent technical writing generations.

by mccoyb 1y ago

melvinroest 1y ago

Maybe do a new prompt at paragraph level?

fredrikholm 1y ago

I was born in 1990, quite late into the end tail of ordinary people not writing letters by hand, reading (a single book/article) for hours on end etc. and it's already evident that people outside of occupations that demand it just don't consume or produce as much (quality) writing as they used to. My fear isn't that LLMs will fail to meet our standards, my fear is that LLMs will drag us down to a new low of homogenized, dull and lifeless form of writing. Perhaps I'm biased, but it feels as if the few times I am impressed by someones writing, the material is 20 years old or older, even if said author is still producing work.

mccoyb 1y ago
I think you're accurately describing a "regression to the mean" phenomenon, which LLMs have transmitted to areas involving programmatic control through text. I recently used an LLM to help scaffold a piece of technical writing. I showed it to several of my peers (sharp and accomplished PhD students), and they immediately identified the paragraphs (even sentences in captions!) written by an LLM. I had spent a good amount of time prompt engineering Opus 4 to try and produce a coherent piece of content aligned with a narrative which I had already developed, but it was still _immediately obvious_ what was not consistent ... it stood out like a sore thumb to experts. By the the time we finished the content, there was not a sentence of LLM-generated writing in the final product. Possibly an incorrect extrapolation, but I think my experience here is the current status quo. Code is a bit easier for these systems, because the LLM can align against test suites, type systems, etc. But for writing ... you just can't trust the current capabilities to pass the sniff test by experts. After all, how does one commmunicate "coherence" as a test which the LLM might align its generations against? That's why you need a shared world model (with your audience) -- and if we could produce a computational representation of such a thing, we might have a chance at better coherent / consistent technical writing generations.