3 ms·

In fact, post-LLM quality monitoring is a huge time trap. I can't write anything for humans using LLMs, because there is so much to change that it ends up like

In fact, post-LLM quality monitoring is a huge time trap. I can't write anything for humans using LLMs, because there is so much to change that it ends up like the Ship of Theseus, I would have been faster writing it myself than delegating the task.

keeganpoppen 10d ago
completely agree. i will admit to taking llm phrases when i am really trying to refine every last detail and it just makes a suggestion that is too good to unsee. but in general it doesn’t feel like it helps directly with the “word choosing” part of the writing task at all (if you are someone who cares about word choice), which is… definitely a pretty big part of the job, lol. it is great at analyzing the argument, finding inconsistencies, helping you think through what parts should be cut, helping you refine examples or fix the occasional “how do i get this phrase to work correctly in this transition?” kinds of stuff. but anything where LLMs are the prima materia… that stuff literally only makes sense _to me_. which makes sense, because it is written _for_ me, no matter what instructions i actually give it, bc of memories and a million other things. and i say this as a complete maximalist wrt. trying to use llms for absolutely every last thing they possibly can be used for, just to see what it’s like. i guess i would say that it does very, very little to make the writing process meaningfully faster; it _can_ do _plenty_ to help make your output better though, which is definitely something— just isn’t the thing most people are looking for.
jerf 10d ago
I've had some success writing certain explicitly technical documents, with a style guide provided to the LLM that it can match, and then going over it by basically iterating on every sentence in the document and asking "can this sentence be removed?". Style guides are a big tools people are missing out on. It isn't enough to say "write concisely and technically". Give them a sample. Yea verily these many 5 or 6 years ago, "style transfer" was a big thing that early LLM tech was doing. It's still very good at it. That said I still tend to cut out at least 25% of the resulting verbiage and adding back another 10% or so more of my own original content even under those circumstances. Where it has been really helpful is that I tend to want to write in a conversational style that doesn't seem to match most people's expectations of a technical document. The LLMs let me write my way and style-shift it into something closer to what people expect. And LLMs, since let's be honest they're the primary audience nowadays. Which I am not even upset about; I'd rather 5 LLMs read the architecture document than the amortized .2 or so humans I could expect in the same circumstance 5 years ago. Which also implies, in many cases, I am feeding the AI as much text as I expect to come out, or in some cases, even more, as I am describing context, reasoning, and other things that may impact the writing but are not necessarily repeated in the final text. I factored out a lot of the context into my user-level CLAUDE.md which has helped cut that down a bit.
- aprilthird2021 10d ago
  Having the style guide and proofreading it and linking it is more effort than the lazy slopulists want to put in, so your writing will reach far better than theirs will
  - skydhash 10d ago
    I was reading the manual of a car head unit, and it was better quality than 99% of AI flavored content.
saulpw 10d ago
You have to assume that in many situations a human is asking an LLM about the content you've provided, even if they or another human reads some or all of it themselves too. So it's valuable to proactively ask your own LLM in the same way, so you can see what the other person will get out of it. And tune the verbiage until the LLM tells them what you want it to :)
- streetfighter64 10d ago
  Sorry if this comes off as critical, but are you seriously doing that? I can't imagine optimizing my writing for people choosing to mangle it through a LLM rather than for the people reading it as-is. Also the word "verbiage" ticks me off as a LLM-ism on the level of "delve" or "smoking gun". Are its users aware that in addition to its usage as somewhat of a synonym of "idiolect", it more commonly means "a profusion of words usually of little or obscure content", i.e. slop?
  - saulpw 10d ago
    I wrote a contract proposal a few months ago and I absolutely did a few rounds with an LLM to get the result I wanted. Anyone applying for a job should be doing this with their cover letters. You know that HR has a pipeline set up with a prompt like "here's the job description, here's the candidate's info, rate the candidate on a scale from 1-10" so you can do the same and learn how to boost your own signal. I have zero qualms about doing this. As for "verbiage", I guess that's too bad, I'm going to continue using it anyway. Interesting factoid about the word though. I will say that "utilize" is one of my pet peeves (99% of the time the word you actually want is "use") and I've excised it from my spoken vocabulary, but I like(d) how verbiage rhymes with foliage so I'm more reticent to give that one up.
    - streetfighter64 10d ago
      Fair enough, cover letters and contract proposals are more marketing than actual information, so that's fair game I suppose. Good trick. I wouldn't use it in something meant to transmit actual information though, like a manual or something.
TomGarden 10d ago
Agreed. Time trap, and even worse imo: creative flow trap. You enter critique mode too early in the process
imnotr0b0t 10d ago
True, by the way, editing often takes much longer than writing the same text from scratch yourself, I've noticed that too.
chaostheory 10d ago
I agree and disagree. Yes, it turns into the Ship of Theseus, but I feel it helps jump start things. This was actually the way some writers work I.e. just write fast expecting garbage. Keep molding it like clay until you get what you want.
Razengan 10d ago
[dead]
GarnetFloride 10d ago
So much of my time right now is completely rebuilding tutorials that the devs had generated because except for steps like open and save its just wrong and it doesn't work and customers are not going to let that slide.
lubujackson 10d ago
This has started to be an issue at my work with code. Everyone started using the thermo-nuclear-code-quality-review skill and, while it does a great job finding consolidation opportunities and architecturally-weak code, it also continues to expand PRs well beyond their scope until you end up revamping far more than you intended...
- strix_varius 9d ago
  hey thanks for mentioning that skill, I hadn't heard of it before but I just ran it on a non trivial codebase I've been planning to refactor and it did a great job. I don't use a lot of "skills" so if there are others you've started using frequently I'd be curious to learn about them.

keeganpoppen 10d ago

completely agree. i will admit to taking llm phrases when i am really trying to refine every last detail and it just makes a suggestion that is too good to unsee. but in general it doesn’t feel like it helps directly with the “word choosing” part of the writing task at all (if you are someone who cares about word choice), which is… definitely a pretty big part of the job, lol. it is great at analyzing the argument, finding inconsistencies, helping you think through what parts should be cut, helping you refine examples or fix the occasional “how do i get this phrase to work correctly in this transition?” kinds of stuff. but anything where LLMs are the prima materia… that stuff literally only makes sense _to me_. which makes sense, because it is written _for_ me, no matter what instructions i actually give it, bc of memories and a million other things. and i say this as a complete maximalist wrt. trying to use llms for absolutely every last thing they possibly can be used for, just to see what it’s like. i guess i would say that it does very, very little to make the writing process meaningfully faster; it _can_ do _plenty_ to help make your output better though, which is definitely something— just isn’t the thing most people are looking for.

jerf 10d ago

I've had some success writing certain explicitly technical documents, with a style guide provided to the LLM that it can match, and then going over it by basically iterating on every sentence in the document and asking "can this sentence be removed?". Style guides are a big tools people are missing out on. It isn't enough to say "write concisely and technically". Give them a sample. Yea verily these many 5 or 6 years ago, "style transfer" was a big thing that early LLM tech was doing. It's still very good at it. That said I still tend to cut out at least 25% of the resulting verbiage and adding back another 10% or so more of my own original content even under those circumstances. Where it has been really helpful is that I tend to want to write in a conversational style that doesn't seem to match most people's expectations of a technical document. The LLMs let me write my way and style-shift it into something closer to what people expect. And LLMs, since let's be honest they're the primary audience nowadays. Which I am not even upset about; I'd rather 5 LLMs read the architecture document than the amortized .2 or so humans I could expect in the same circumstance 5 years ago. Which also implies, in many cases, I am feeding the AI as much text as I expect to come out, or in some cases, even more, as I am describing context, reasoning, and other things that may impact the writing but are not necessarily repeated in the final text. I factored out a lot of the context into my user-level CLAUDE.md which has helped cut that down a bit.

aprilthird2021 10d ago
Having the style guide and proofreading it and linking it is more effort than the lazy slopulists want to put in, so your writing will reach far better than theirs will
- skydhash 10d ago
  I was reading the manual of a car head unit, and it was better quality than 99% of AI flavored content.

saulpw 10d ago

You have to assume that in many situations a human is asking an LLM about the content you've provided, even if they or another human reads some or all of it themselves too. So it's valuable to proactively ask your own LLM in the same way, so you can see what the other person will get out of it. And tune the verbiage until the LLM tells them what you want it to :)

streetfighter64 10d ago
Sorry if this comes off as critical, but are you seriously doing that? I can't imagine optimizing my writing for people choosing to mangle it through a LLM rather than for the people reading it as-is. Also the word "verbiage" ticks me off as a LLM-ism on the level of "delve" or "smoking gun". Are its users aware that in addition to its usage as somewhat of a synonym of "idiolect", it more commonly means "a profusion of words usually of little or obscure content", i.e. slop?
- saulpw 10d ago
  I wrote a contract proposal a few months ago and I absolutely did a few rounds with an LLM to get the result I wanted. Anyone applying for a job should be doing this with their cover letters. You know that HR has a pipeline set up with a prompt like "here's the job description, here's the candidate's info, rate the candidate on a scale from 1-10" so you can do the same and learn how to boost your own signal. I have zero qualms about doing this. As for "verbiage", I guess that's too bad, I'm going to continue using it anyway. Interesting factoid about the word though. I will say that "utilize" is one of my pet peeves (99% of the time the word you actually want is "use") and I've excised it from my spoken vocabulary, but I like(d) how verbiage rhymes with foliage so I'm more reticent to give that one up.
  - streetfighter64 10d ago
    Fair enough, cover letters and contract proposals are more marketing than actual information, so that's fair game I suppose. Good trick. I wouldn't use it in something meant to transmit actual information though, like a manual or something.

TomGarden 10d ago

Agreed. Time trap, and even worse imo: creative flow trap. You enter critique mode too early in the process

imnotr0b0t 10d ago

True, by the way, editing often takes much longer than writing the same text from scratch yourself, I've noticed that too.

chaostheory 10d ago

I agree and disagree. Yes, it turns into the Ship of Theseus, but I feel it helps jump start things. This was actually the way some writers work I.e. just write fast expecting garbage. Keep molding it like clay until you get what you want.

Razengan 10d ago

[dead]

GarnetFloride 10d ago

So much of my time right now is completely rebuilding tutorials that the devs had generated because except for steps like open and save its just wrong and it doesn't work and customers are not going to let that slide.

lubujackson 10d ago

This has started to be an issue at my work with code. Everyone started using the thermo-nuclear-code-quality-review skill and, while it does a great job finding consolidation opportunities and architecturally-weak code, it also continues to expand PRs well beyond their scope until you end up revamping far more than you intended...

strix_varius 9d ago
hey thanks for mentioning that skill, I hadn't heard of it before but I just ran it on a non trivial codebase I've been planning to refactor and it did a great job. I don't use a lot of "skills" so if there are others you've started using frequently I'd be curious to learn about them.