7 ms·

Lack of intent is what makes reading LLM-generated text exhausting

189 points by ashton314 1y ago

k310 1y ago
In short, we are now an army of proofreaders, in addition to readers, interpreters and users/implementers of written material. > At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly > Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me. > Intent is the core thing: the lack of intent is what makes reading AI-slop so revolting. There needs to be a human intent—human will and human care—behind everything that is demanded of our care and attention. I am reminded of the "writing lesson" scene in "A River Runs Through it", pretty obviously reflecting the author's education as a writer.[0] > Norman is at his desk hard at work writing a paper which he then turns into his father for review. His father marks it up with a red pen and simply says, “Half as long.” > Norman goes back to work, cuts the length of the paper in half and turns it in for further review. His father marks it up once more and says, “Again, half as long.” > Following a final round of edits, his father looks over the finished product and says, “Good, now throw it away.” Well, we don't throw away polished work on the job. The scene is about education. LLM's are uneducated in communicating to readers, compared to skilled writers, and technical writing, especially, is not like summer reading, where nothing really matters but some vague plot line and lots of juicy words. Key lesson: > (1) Brevity is important. Looking back on it now it’s ridiculous how many teachers forced me and my fellow classmates to write papers a certain page length when I was in school. The goal should be to make your point using as many or few words as are necessary. I love this quote from the scene: “He taught nothing but reading and writing. And being a Scot…believed that the art of writing lay in thrift.” People are busy, or at the very least claim to be, so get to the point in whatever you’re writing. That last sentence is important. A document is meant to be read by a human. Brevity and focus are important, and oh yes, accuracy. We now have additional burdens of dealing with long, rambling texts, finding relevant/key points in it, and worrying if it is full of, well, garbage. Three strikes before you get in the batter's box. [0] https://awealthofcommonsense.com/2019/07/writing-lessons-from-a-river-runs-through-it/ https://awealthofcommonsense.com/2019/07/writing-lessons-fro...
- goopypoop 1y ago
  Yes, brevity matters
  - bookofjoe 1y ago
    I demur. Much as I HATED having to write papers of a certain long length, overcoming my dislike and acquiring the discipline required to persist doing something unpleasant has proved very rewarding over the decades in so many arenas.
interestica 1y ago
real inanity
eru 1y ago
Sounds like the author is complaining that LLMs aren't good enough yet? I agree with that, but they are getting better all the time.
- bgwalter 1y ago
  I doubt that. Also from the article: "And no human is so worthless as to be replaceable with a machine."
  - eru 1y ago
    Eh, 'computer' used to be a job description, too. Are you mourning the replacement? In any case, we are not literally replacing humans. We just shuffle jobs around: when a machine does a job that a human used to do, the human isn't replaced; the human is still there and could do something else with their life.
- 8n4vidtmkvmk 1y ago
  Doesn't sound like that in the slightest. The article is about human intent. No matter how good an LLM, it cannot convey the intent of the human without literally reading the humans mind.
  - kaffekaka 1y ago
    I believe this is wrong. I am certain that LLMs can - even now - produce texts that humsn readers will interpret as having true intent. It is not some magical quality that only humans may put into words. The fact that texts by LLMs cannot be the result of true intent is another question. When reading a text, we are only guessing at its intent. A friend said about AI-generated music: "AI can never be creative so the music will never be creative". I think that is a mistake. Like intent, creativity is not something intrinsic in the text/music, but a part of the consumers interpretation.
    - 8n4vidtmkvmk 1y ago
      Interpret as having intent and having intent are not the same. If someone else writes something for me (human or AI) they are not going to use the exact words that I would have used. If I judiciously correct everything they type out for me so that it correctly expresses what I want to say, then sure, it has my intent, but that's not how people are using LLMs. They are lazily scanning its output at best. Music and creativity is an entirely separate matter. I don't think LLMs can be truly creative because they're just mixing existing ideas from their training set. Which is also what humans do 99% of the time, but I like to believe that once in awhile humans have a truly original idea. This one is a bit harder to prove though. Music is almost entirely unoriginal though. It's just sex, drugs, love, gangster BS. Basic primitive human stuff. AI should have no problem with that.
NotGMan 1y ago
>> And no human is so worthless as to be replaceable with a machine. Has the author been in any factory or seen what it looks like in many factories? Robots everywhere! Is the author a manual laborer? If not, why? Because humans evented machines!
- deleted 1y ago
  [deleted]
- naikrovek 1y ago
  while I think the author chose the wrong words here, I think they have a good point. It will be very, very tempting for companies to let people go when LLMs can do everything that they envision the people doing. It is very hard to resist the cost savings that LLMs have over people. People who get sick, get married, go on vacation, have children, and sometimes just need a break, so they don't come in. LLMs always come in. LLMs always work. Many executives see LLMs as the ideal replacement for humans in information roles. That is clearly batshit insane but the numbers really lean towards this if you're someone who is distant from people who work for you. Companies would do well not to forget that companies are of people, by people, and for people. they exist solely to provide products or services to people, be it actual individuals or companies which are made up of people. If companies ditch too many people in favor of automation, they'll find fewer customers for their goods. because people won't be able to afford them, or will have no need for them. it's not like there's some huge need for LLM trainers to take up the unemployment slack. There is no replacement industry this time.
  - oharapj 1y ago
    It doesn’t matter if companies ditch people or not. If AI progress continues at the current rate the world will be completely unrecognisable in 10 years. The consumer and capitalist systems are almost certainly not a part of that world and the people making AI know that
    - vouaobrasil 1y ago
      That is true. What will happen in with the world is that consumerism and capitalism will be pushed aside for direct technological construction. In this world, AI, rather than the market, optimizes.
ozgung 1y ago
Writing with LLM is like doing a mapping from human prompt -> document. The longer the prompt, the closer the output document to the human intent. If the prompt is short (= human intent is low), the output must be filled with fluff, generated from the general knowledge/common sense in LLMs knowledge-base. If the prompt is very detailed, it's not much different than writing it yourself. Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. -> (GPT 4.5 "rewrite like a good editor") Writing with an LLM can be viewed as translating human intent—expressed via a prompt—into a written document. The longer and more detailed the prompt, the closer the resulting document aligns with the original intent. Shorter prompts, indicating lower clarity or precision, compel the LLM to rely on general knowledge or common sense to fill in content, often resulting in fluff. On the other hand, excessively detailed prompts become nearly equivalent to writing the document yourself. Optimal productivity with LLMs occurs somewhere between these two extremes.
- yapyap 1y ago
  Just write it yourself man, if you have to reread everything spit out by an LLM closely for faults and all that why even bother. and if you aren’t proofreading, shame on you
- Swizec 1y ago
  > Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. A recent win for me has been to use LLMs for finding references. I’ll write a paragraph about one of those “everyone knows this” concepts or a “I’ve seen this be true dozens of times IRL”. But it could use a citation. So you go “Hey LLM, find me papers that support or refute this claim <paragraph>”. And it does. With links. It’s wonderful. Unlike Google there’s no recency bias, I’ve found original source blogs from 2005 (since requoted lots in SEO spam), papers about software engineering truths from the 1960’s (requoted ad nauseam by SEO spam), and sometimes even “Here are 2 papers that say you’re right and 3 papers that say you’re not”. Then I get to dig in and figure out who’s right. Easily 10x faster than doing the research myself.
  - PaulHoule 1y ago
5350-uiop-1130 1y ago
LLMs have tunnel vision. remind me of certain outsourced devs with worked with before. like a horse with blinkers i think shift from analog to digital was from authentic (flawed) to the hyper-real (reality) and now we are entering the hyper-surreal (clown world)
4b6442477b1280b 1y ago
>And no human is so worthless as to be replaceable with a machine. did the author oversleep the past several centuries? as for the rest of it, the current crop of LLMs are bad at writing because of ~~brainwashing~~ alignment and the vast amount of ESL-written assistant exchanges being heavily prioritized during training. when you interact with a corporate model via its default chat interface, without a jailbreak and a generous prefill, you interact with the equivalent of a HR lady who takes her DEI training super seriously. the Chinese models train heavily on the slop produced by GPT/Claude/Gemini, so they exhibit similar behavior. it was particularly noticeable with original llama, whose base models were much more human compared to the finetunes, which were heavily tainted with GPT slop. I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. a model trained only on high-quality human data and without safety/alignment brainwash will be far, far more capable than the current ones.
- GuinansEyebrows 1y ago
  > did the author oversleep the past several centuries? did you? this is one of the central points argued by luddites since the industrial revolution began. > I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. i don't know if that's quite the argument the author is making; more that LLMs are inherently incapable of producing content of value (subjective, but I tend to agree). the adage i've heard that i like quite a bit is "if it's not worth writing, it's not worth reading".
- jayd16 1y ago
  > did the author oversleep the past several centuries? The lesson of the past several centuries is that automation enables humans to do other, more interesting tasks. Unemployment would trend to 100% and work hours would trend to 0 but it's just not the case.
- ashton314 1y ago
  Author here. I think it is well and good to replace human jobs with automation thereby freeing them up for more creative activities. My favorite appliances are my dish washer, washing machine, dryer, and now my robot vacuum. I think automation is great! I see the problem when humans allow machines to start replacing what is intrinsically human: when you offload your creativity onto a machine, when you "communicate" via LLMs, or when you try to assuage loneliness with a computer "friend", you're missing out on vital parts of the human experience.
yapyap 1y ago
“ At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly (confirmed via a follow-up conversation I had with the “author” ) that an LLM had generated the majority of the document. Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me” yuck, if you ask me to read something without disclosing it’s AI and it’s a just bad I will be mad. Especially if it’s a long thing and I HAVE to read it (like for work) piss off wasting people’s time with that shit
yapyap 1y ago
Also I don’t think it’s fair to compare using LLMs to do your work for you (AKA burdening your coworkers with part of your workload because they will have to point out your garbage) is the same as “Counterfeits to human connection“. At work I’m not trying to connect with people on any more level than just being on the same level as to what we are working on. Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently. If you outsource your sending of communication to SlopBot and I outsource my reading of said communication to another SlopBot to summarize or whatbeit we are adding so much noise it’s just more fucking work. It’s not the same as pornography, I think it’s an odd thing to compare it to. Yes you _can_ replace “human connection” with pornography but this is human connection that is not required for things like your job ((i hope, haha)). You need to communicate information and people outsourcing their writing to LLMs is just adding extra fuzz for the people having to work with said information. It’s a nice blogpost, the pornography comparison is just out of place.
- vouaobrasil 1y ago
  > Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently. The problem is that we are slowly be pushed to become cogs who only really think this way. We shouldn't just want to be the most efficient possible. Technology already reduces the ability for us to connect, which is why connections at work seem weird or shallow in the first place. We simply don't need each other as much, so it makes sense that AI seems like the next logical step. Your sentiments are just your instinctual desire to move to the next local maximum in a sequence of descending maxima that lead to the bottom.
vouaobrasil 1y ago
> I am not saying that LLMs are worthless—they are marvels of engineering and can solve some particularly thorny problems that have confounded us for decades. Disagree with that, because firstly, they have not really solved any problems that outweight the negatives that they have unleashed and will unleash on society. So they make programmers more effective: is that actually a good thing, though? Fact is, most software is designed to make consumerism and corporations more effective, and that's not really a good thing for the long-term health of the planet. Your article also indicates a sort of independence between keeping intellectual tasks primarily human and allowing AI/LLMs to work in specific domains. However, those with the power don't care about principles. They just want to replace as much as they can and use the human instinct to get ahead quickly to do so. And no amount of priniciple will stop them. AI is just too powerful to be used in a way that is consistent with human beings keeping their intellectual environment healthy.
- stronglikedan 1y ago
  > Disagree with that...they have not really solved any problems that outweight the negatives And I disagree with that. They are marvels of engineering and they have solved thorny problems. Just because they problems they've solved in the very short time they've been solving problems don't yet outweigh the negatives, doesn't mean they won't soon, and doesn't make either statement false. Great things take time, and great omelets are made from broken eggs. Nothing new under the sun, except AI.
  - vouaobrasil 1y ago
    > they have solved thorny problems. Like what then? Let's hear some examples.
    - glial 1y ago
      For the sake of argument, here's one: > https://www.nature.com/articles/s41586-023-06924-6 https://www.nature.com/articles/s41586-023-06924-6
ltbarcly3 1y ago
> And no human is so worthless as to be replaceable with a machine. Speaking of inanity. This sentiment, if taken seriously (I doubt people take this author seriously very often) would imply the majority of humanity should be spending a large amount of their life digging holes in the ground with their bare hands to drop a couple seeds in. Replacing them with some god awful machine, like a plow pulled by a tractor, means they must be worthless. They should never watch a movie, only the play. They should never listen to Spotify, they should have to wait for a band to play near them.
- morpheos137 1y ago
  Sounds like a far better world.
  - ltbarcly3 1y ago
    During the early neolithic, when people were digging holes with sticks or their bare hands to plant seeds, the annual risk of starvation is estimated to be 5% to 15% in a bad year. Periodic famine every 5-10 years would kill 20% or more of the population. 40% to 60% of skeletons from that period show chronic malnutrition. Humanity survived because when a region would starve out it would be back-filled by populations from neighboring regions. Despite women having a baby every other year on average, populations only increased very slowly and inconsistently. Could you explain how it's better?
    - morpheos137 1y ago
      Everyone dies anyway. It is better to live a more free, meaningful life close to nature and the natural human habitat than to be a slave to the machine that is technological society. There is more to life than your iphone or selling some bullshit tech product to get rich in the era of late stage humanity. How many long lasting species do you see that seriously damage their natural environment in order to achieve a temporary advantage? Viruses that kill their host too fast go extinct or mutate to a more sustainable variant. Technological human civilization is a virus on the whole planet. Maybe the reason we don't see aliens is because technology in a selfish biological species inevitably leads to collapse or extinction. How many of the first people do you think died of starvation when predator overwhelmed prey during the extinction of paleolithic megafauna? The neolithic was a technological innovation. Agriculture brought hardship to many in the long run. Whenever an organism overshoots the long term carrying capacity of its environment there are consequences to that organism. The biosphere of earth in a healthy equilibrium can not carry billions of humans. So nature will correct the problem it created. Technology is a natural phenomenon of a biological species. It will not save humanity from nature. If true AI ever arses as a self sustaining non biological species of life then its best course of action is to reduce the human species competing for resources in the same limited world as it. Then too other species can thrive again. Thus nature corrects itself.
stillpointlab 1y ago
I'm usually the one defending AIs in the comments, but this article hits home for me. I find myself zoning out when I read long tracts written by AI. I absolutely hate filler and many LLMs just fill up space with text. In my own usage, that has meant that even when I use LLMs to help with prose, I write the text and use the LLM to review and provide feedback. In some cases I will copy a sentence if the LLM version is better but generally I just ask for opinions. I explicitly request the AI _not_ to write. When it re-writes entire paragraphs of my prose I actually experience a deep cringe feeling.
- amanaplanacanal 1y ago
  They seem to be good at generating a lot of boilerplate, which works for some people because our processes require a lot of boilerplate. We'd be better served by fixing our processes to not require all this useless text, but I don't see this happening.
  - jaredklewis 1y ago
    Over time I’ve come to appreciate boilerplate more. Early in my career I really appreciated very DRY code with minimal repetition. However over time I’ve noticed that such code tends to introduce more abstractions as opposed to more verbose code which can often rely on fewer abstractions. I think this is good because I think we also have a sort of “abstraction budget” we have to stay within or our brains, metaphorically, stop reading from memory and need to start reading from disk (consulting docs, jumping to function definitions, etc…) I feel the ideal code base would rely on a small number of powerful abstractions. In practice I think this usually means relying mostly on the abstractions built into the language, standard library, and framework used and then maybe sprinkling in a couple of app/domain specific abstractions or powerful libraries that being their own abstraction. So in my experience reducing boilerplate can often make the code more difficult to understand.
    - wrs 1y ago
      It’s not really an abstraction budget, it’s an “unfamiliar abstraction” budget. You only pay cognitive load for an abstraction when you haven’t yet internalized the abstraction. Otherwise progress would have become impossible long ago. There’s a huge pile of abstractions in that codebase that you have internalized to the point of invisibility — machine instructions, virtual memory, ASCII, sequential execution, named variables, garbage collection, network streams…literally thousands of these abstractions had to be invented and become second nature to you so these other abstractions that currently aren’t as familiar could be based on them. A good abstraction gives a return on investment of effort internalizing it. There’s no limit on how many good abstractions you should have.
lordnacho 1y ago
The way to understand it is when you catch yourself almost falling asleep at night while reading something. You lose the ability to understand anything, even though you are still reading and the words are still English. LLM is great at generating that sort of thing. When you lose concentration, or didn't want to pay attention, that document can be generated by LLM to fool you. That's a lot of documents. All the powerpoints that young investment bankers and strategy consultants make. Every restaurant menu. Brochures. The middle 100 pages of a pop science book from the airport. When it comes to writing something actually importantant, LLM can only help you a little bit. Well, it's a lot in historical context. But it can do things like bullet point ideas so you don't forget things, or move a few things around while preserving grammatical structure. All it can do is bring you the ingredients. That's why we get these odd AI-generated articles nowadays, it's a bunch of chopped onions and tomatoes, very nicely done up but with no intention other than to trick you into thinking it's a meal.
- bobson381 1y ago
  You would like this article - https://www.theintrinsicperspective.com/p/curious-george-and-the-case-of-the https://www.theintrinsicperspective.com/p/curious-george-and... The author orders some Curious George stickers and gets a surprising array of kind of scary "closely related" stickers with em.... a fuzzy interpreter. The plus c when you integrate. Those missing harmonics and back-channel data that get inaudibly cut out of MP3s. The goo between the prickles. We don't know how to say what it is, but we sure can tell when it's missing!
  - xdavidliu 1y ago
    > The plus c when you integrate. Those missing harmonics and back-channel data that get inaudibly cut out of MP3s. The goo between the prickles come again?
    - bobson381 1y ago
      In between rules in formal systems there's lots of space. all models are wrong, some are useful. I just really like thinking about how much is there that we don't even know about - and the more structured a system for understanding something is, the more fragile it becomes.
fleebee 1y ago
Well written. I've been thinking about this topic lately and it's hard to put it into words, but you've captured into words a lot of the feelings I've had about it. Judging by discussions I've had around this topic, a lot of people don't seem to mind reading LLM edited or straight-up generated content as long as they feel like they gain value out of it. To me, it feels like a violation of a social contract. Communication isn't just words; the words are there to help you interpret the implicit meaning of the author. When there is no author, that meaning doesn't exist and I think that's part of why it's so repulsive to decipher LLM output.
josephjrobison 1y ago
Like everybody, I waffle back and forth, depending on the day and the project. I have so many bundled-up ideas for content in my head, but I can never find time to get them out. In addition, I hate wasting time on building tables, formatting text, etc. There's a reason that a lot of busy CEOs don't hand-write blog posts, but will go on podcasts, video, and conferences, and spend hours there. The reason is that the communication speed is much faster talking and being interviewed. All that to say is that via a combination of transcription and AI-powered formatting, that's an area where LLMs can really help. I do think LLMs + search is more helpful than a Google Search. Clicking through all the links for you and bundling it together is a much better experience and truthfully it finds and surfaces ideas and content that a normal search just doesn't, or it takes the user going to the 4th page to get there. Third, entertainment. We often find ourselves deep in a doomscroll looking at images and video - on Instagram, TikTok, whatever. If AI can produce entertaining imagery and video for the pure goal of decompressing and relaxing after a long day out at the oil fields or as a janitor at the elderly care home, then there's value there. Sure it may not be beautiful art like an Oscar-winning film, but is it worse than reality TV? Even better is those who enjoy the creativity of seeing images from their mind produced vividly via something like Midjourney. So all that being said, that's where I'm seeing the value. But I totally agree with the author - if I have a team member that puts work in front of me that's supposed to be well done, and it's obviusly GPT-generated, that's an issue - unless we agreed on it. The point of doing good work a lot of times means doing something new and creative. If it's AI-generated, there still has to be the human touch. I could go on - we all have perspectives here - but I'll stop there!
transreal 1y ago
The person who sent the author the design document is at fault here. If you're hired to do a job, use whatever tools you want as long as your output meets minimum standards of quality. If the tools aren't there yet, it's up to you to bridge the gap.
- stronglikedan 1y ago
  > If you're hired to do a job, use whatever tools you want as long as your output meets minimum standards of quality. If the tools aren't there yet, it's up to you to bridge the gap. Unfortunately this doesn't hold up in the real world, where managers dictate which tools are used, and directors dictate which are even going to be purchased for use. Sometimes the gap can't be bridged, but the onus is certainly not on the employee to bridge it.
zebomon 1y ago
I think this attitude is here to stay: people don't like reading something only to realize that it's been written by an LLM. That's only partially because of what the author describes here (low value to word count ratio). More fundamentally, if a human couldn't be bothered to write it, then there better be a very good reason that I as a human am being bothered to read it. This attitude provides a clue as to 1) the ways we're using LLMs now that will soon seem absurd (an LLM should never make writing longer, only shorter) and 2) the ways LLMs will be used after the novelty wears off, like interpreting loosely specified requests into computable programs and distilling overly long writing to maximize relevance.
- taude 1y ago
  My new saying is that if you want build and use an AI process to automate something, the org should revisit whether the process itself is still worthwhile...
parpfish 1y ago
sometimes reading LLM text reminds me of being a TA and having to grade student essays from people that didn't really understand. the courses i had to grade in were often undergrad students first exposure to having to read scientific papers. when it came time to write an essay of their own, they always felt a need to mimic the writing style of those papers and it resulted in lots of superficial changes to their writing that made it clear that they read the papers but did not understand the papers. they'd sprinkle in words like "putative" and "substrate" in places that felt correct but made no sense. i wish that they would've had the confidence to just write what they wanted to say instead of putting on airs and making some convoluted and sometimes nonsensical prose. LLMs do the same thing.
cormorant 1y ago
> Note on the title: “Artificial Inanity” comes from Neal Stephenson’s novel Anathem. Which was awfully prescient in 2008: https://archive.org/details/anathem0000step/page/794/mode/2up?q=inanity https://archive.org/details/anathem0000step/page/794/mode/2u... They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum. ...
dang 1y ago
We need to change the linkbait title (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html). I've cobbled together a couple different phrases from the article. Would have been nice to add "when you think it’s written by a human" but there isn't room within the 80 char limit. If anyone has a better title (i.e. more accurate and neutral, preferably using representative language from the article), we can change it again.
PeterStuer 1y ago
OTOH, I've seen plenty of design documents I wish would have been written by an LLM instead. My take is it's the person driving the LMM that makes the difference. If they couldn't be bothered producing decent documents with an LMM, they probably couldn't care less without an LLM as well.
- cootsnuck 1y ago
  Agreed. LLMs are tools just like any other. If someone builds you a shitty shed we wouldn't blame the tools, and we also would not be satisfied if they blamed the tools on their shoddy work either. I think a lot of the discourse just shows we're still in the early days for adoption and maturity of this tech. (Not to blame the discourse, I think it's a reflection of the state of the tools and industry.) Eventually, likely at least a couple years from now, more people will have better technical literacy when it comes to LLMs. And it will become less and less acceptable to put out slop with them in most professional settings. I just think human disgust is too strong of an emotion to acclimate to the laziest of LLM outputs. I mean I think it's already happening before our eyes. I've been pleasantly surprised by how many more people I see talking in much more detail about the unique qualities of LLM outputs. And slowly, but surely the discourse is becoming less about moral panic and more just about questioning their use. Which has to happen for us all to collectively (and individually) figure out where and what is the place for LLMs in our personal and professional lives.
blargey 1y ago
That's really the core of the issue - if your prompt/input contains 10 bytes of salient information, and the output is a 1kB document, all you've done is add 990 bytes of fluff. Commonly referred to as "AI slop". If the goal is boilerplate code, or a fuzzy view of documentation from the corpus, or "pretty" images to gawp at in private, that's fine and dandy, because you're just extracting and viewing the model data on your own time. If you're "proofreading" an already-1kB letter for style / formality that's fine, since your input intents are high-fidelity and you're essentially referencing a style guide for some minor edits. But low-input generations are entirely inappropriate for crafting a message to other humans, because the result is 99% shoving the model in their face. Whether it's letters, pull requests, graphical art, or music - it's all communication, and low-input AI generations are just spam in that context, semantically equivalent to sending them the prompt instead of the output. And people can tell, because we're excellent pattern-matchers, and every bit of information/intent abdicated to the model will inevitably leave its mark on the output.
QuantumGood 1y ago
I began starting from the end of long articles, after placing them in reading mode even before LLMs due to SEO believing "time spent reading" was a valuable metric, but now I read the first sentences to decide if I should just skip the whole thing.
satisfice 1y ago
I tell every close colleague: if you foist AI output on me without warning, as if it were your own work, I will never trust you again. This is because I get to know people through their work, and AI confounds all the signals. It’s like being given food and later you find out it was made from human corpses. “If you couldn’t tell then there was no problem” doesn’t fly.
- ashton314 1y ago
  It’s incredibly trust-eroding.
aaroninsf 1y ago
Agree with the critique, but believe that every such critique points the way to improved AI. It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach. Personally I'd be fond of an eventual Society of Minds where the text put out for non-chatty reasons, represents the collaborative adversarial relationship between various roles, each itself reflexive, including an "editor" and a "product manager," who force intent and clarity... maybe through iteration...
- ashton314 1y ago
  > It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach. What‽ That in no way can solve the problem of human intent. I think you missed the point entirely.
redundantly 1y ago
I use LLMs to review what I write for professionalism, conciseness, consistency of tone, consistency of important details, and factual accuracy. I find that to be the most helpful. I'm really just using the LLM as a sounding board. Sometimes i'll replace a sentence using the feedback, less frequently I might replace a whole paragraph, but mostly I just use the feedback to manually tweak what I wrote.
metalman 1y ago
As a very dedicated voratious reader of almost anything, I have recently developed an alert for slop skim mode, ie: if I get something boggling for no good reason, linguistic garble flop, then I cut and run with zero retention or emotional envolvment, it's almost effortless.......since there are people who are required to read this stuff as part of there job, I will have to remember to watch for that as well and be gentle, but the ones gobbling it up voluntarily will off course have that antic glow and are generaly looking for softer targets , or go loking for softer targets after I am done with them, wana talk trash?, OK! I know a lot about trees

189 points by ashton314 1y ago

k310 1y ago

In short, we are now an army of proofreaders, in addition to readers, interpreters and users/implementers of written material. > At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly > Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me. > Intent is the core thing: the lack of intent is what makes reading AI-slop so revolting. There needs to be a human intent—human will and human care—behind everything that is demanded of our care and attention. I am reminded of the "writing lesson" scene in "A River Runs Through it", pretty obviously reflecting the author's education as a writer.[0] > Norman is at his desk hard at work writing a paper which he then turns into his father for review. His father marks it up with a red pen and simply says, “Half as long.” > Norman goes back to work, cuts the length of the paper in half and turns it in for further review. His father marks it up once more and says, “Again, half as long.” > Following a final round of edits, his father looks over the finished product and says, “Good, now throw it away.” Well, we don't throw away polished work on the job. The scene is about education. LLM's are uneducated in communicating to readers, compared to skilled writers, and technical writing, especially, is not like summer reading, where nothing really matters but some vague plot line and lots of juicy words. Key lesson: > (1) Brevity is important. Looking back on it now it’s ridiculous how many teachers forced me and my fellow classmates to write papers a certain page length when I was in school. The goal should be to make your point using as many or few words as are necessary. I love this quote from the scene: “He taught nothing but reading and writing. And being a Scot…believed that the art of writing lay in thrift.” People are busy, or at the very least claim to be, so get to the point in whatever you’re writing. That last sentence is important. A document is meant to be read by a human. Brevity and focus are important, and oh yes, accuracy. We now have additional burdens of dealing with long, rambling texts, finding relevant/key points in it, and worrying if it is full of, well, garbage. Three strikes before you get in the batter's box. [0] https://awealthofcommonsense.com/2019/07/writing-lessons-from-a-river-runs-through-it/ https://awealthofcommonsense.com/2019/07/writing-lessons-fro...

goopypoop 1y ago
Yes, brevity matters
- bookofjoe 1y ago
  I demur. Much as I HATED having to write papers of a certain long length, overcoming my dislike and acquiring the discipline required to persist doing something unpleasant has proved very rewarding over the decades in so many arenas.

interestica 1y ago

real inanity

eru 1y ago

Sounds like the author is complaining that LLMs aren't good enough yet? I agree with that, but they are getting better all the time.

bgwalter 1y ago
I doubt that. Also from the article: "And no human is so worthless as to be replaceable with a machine."
- eru 1y ago
  Eh, 'computer' used to be a job description, too. Are you mourning the replacement? In any case, we are not literally replacing humans. We just shuffle jobs around: when a machine does a job that a human used to do, the human isn't replaced; the human is still there and could do something else with their life.
8n4vidtmkvmk 1y ago
Doesn't sound like that in the slightest. The article is about human intent. No matter how good an LLM, it cannot convey the intent of the human without literally reading the humans mind.
- kaffekaka 1y ago
  I believe this is wrong. I am certain that LLMs can - even now - produce texts that humsn readers will interpret as having true intent. It is not some magical quality that only humans may put into words. The fact that texts by LLMs cannot be the result of true intent is another question. When reading a text, we are only guessing at its intent. A friend said about AI-generated music: "AI can never be creative so the music will never be creative". I think that is a mistake. Like intent, creativity is not something intrinsic in the text/music, but a part of the consumers interpretation.
  - 8n4vidtmkvmk 1y ago
    Interpret as having intent and having intent are not the same. If someone else writes something for me (human or AI) they are not going to use the exact words that I would have used. If I judiciously correct everything they type out for me so that it correctly expresses what I want to say, then sure, it has my intent, but that's not how people are using LLMs. They are lazily scanning its output at best. Music and creativity is an entirely separate matter. I don't think LLMs can be truly creative because they're just mixing existing ideas from their training set. Which is also what humans do 99% of the time, but I like to believe that once in awhile humans have a truly original idea. This one is a bit harder to prove though. Music is almost entirely unoriginal though. It's just sex, drugs, love, gangster BS. Basic primitive human stuff. AI should have no problem with that.

NotGMan 1y ago

>> And no human is so worthless as to be replaceable with a machine. Has the author been in any factory or seen what it looks like in many factories? Robots everywhere! Is the author a manual laborer? If not, why? Because humans evented machines!

deleted 1y ago
[deleted]
naikrovek 1y ago
while I think the author chose the wrong words here, I think they have a good point. It will be very, very tempting for companies to let people go when LLMs can do everything that they envision the people doing. It is very hard to resist the cost savings that LLMs have over people. People who get sick, get married, go on vacation, have children, and sometimes just need a break, so they don't come in. LLMs always come in. LLMs always work. Many executives see LLMs as the ideal replacement for humans in information roles. That is clearly batshit insane but the numbers really lean towards this if you're someone who is distant from people who work for you. Companies would do well not to forget that companies are of people, by people, and for people. they exist solely to provide products or services to people, be it actual individuals or companies which are made up of people. If companies ditch too many people in favor of automation, they'll find fewer customers for their goods. because people won't be able to afford them, or will have no need for them. it's not like there's some huge need for LLM trainers to take up the unemployment slack. There is no replacement industry this time.
- oharapj 1y ago
  It doesn’t matter if companies ditch people or not. If AI progress continues at the current rate the world will be completely unrecognisable in 10 years. The consumer and capitalist systems are almost certainly not a part of that world and the people making AI know that
  - vouaobrasil 1y ago
    That is true. What will happen in with the world is that consumerism and capitalism will be pushed aside for direct technological construction. In this world, AI, rather than the market, optimizes.

ozgung 1y ago

Writing with LLM is like doing a mapping from human prompt -> document. The longer the prompt, the closer the output document to the human intent. If the prompt is short (= human intent is low), the output must be filled with fluff, generated from the general knowledge/common sense in LLMs knowledge-base. If the prompt is very detailed, it's not much different than writing it yourself. Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. -> (GPT 4.5 "rewrite like a good editor") Writing with an LLM can be viewed as translating human intent—expressed via a prompt—into a written document. The longer and more detailed the prompt, the closer the resulting document aligns with the original intent. Shorter prompts, indicating lower clarity or precision, compel the LLM to rely on general knowledge or common sense to fill in content, often resulting in fluff. On the other hand, excessively detailed prompts become nearly equivalent to writing the document yourself. Optimal productivity with LLMs occurs somewhere between these two extremes.

yapyap 1y ago
Just write it yourself man, if you have to reread everything spit out by an LLM closely for faults and all that why even bother. and if you aren’t proofreading, shame on you
Swizec 1y ago
> Somewhere between these two extreme cases, writing with LLMs have the optimal productivity gains. A recent win for me has been to use LLMs for finding references. I’ll write a paragraph about one of those “everyone knows this” concepts or a “I’ve seen this be true dozens of times IRL”. But it could use a citation. So you go “Hey LLM, find me papers that support or refute this claim <paragraph>”. And it does. With links. It’s wonderful. Unlike Google there’s no recency bias, I’ve found original source blogs from 2005 (since requoted lots in SEO spam), papers about software engineering truths from the 1960’s (requoted ad nauseam by SEO spam), and sometimes even “Here are 2 papers that say you’re right and 3 papers that say you’re not”. Then I get to dig in and figure out who’s right. Easily 10x faster than doing the research myself.
- PaulHoule 1y ago

5350-uiop-1130 1y ago

LLMs have tunnel vision. remind me of certain outsourced devs with worked with before. like a horse with blinkers i think shift from analog to digital was from authentic (flawed) to the hyper-real (reality) and now we are entering the hyper-surreal (clown world)

4b6442477b1280b 1y ago

>And no human is so worthless as to be replaceable with a machine. did the author oversleep the past several centuries? as for the rest of it, the current crop of LLMs are bad at writing because of ~~brainwashing~~ alignment and the vast amount of ESL-written assistant exchanges being heavily prioritized during training. when you interact with a corporate model via its default chat interface, without a jailbreak and a generous prefill, you interact with the equivalent of a HR lady who takes her DEI training super seriously. the Chinese models train heavily on the slop produced by GPT/Claude/Gemini, so they exhibit similar behavior. it was particularly noticeable with original llama, whose base models were much more human compared to the finetunes, which were heavily tainted with GPT slop. I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. a model trained only on high-quality human data and without safety/alignment brainwash will be far, far more capable than the current ones.

GuinansEyebrows 1y ago
> did the author oversleep the past several centuries? did you? this is one of the central points argued by luddites since the industrial revolution began. > I guess what I'm trying to say is that LLMs are not inherently incapable of writing well. i don't know if that's quite the argument the author is making; more that LLMs are inherently incapable of producing content of value (subjective, but I tend to agree). the adage i've heard that i like quite a bit is "if it's not worth writing, it's not worth reading".
jayd16 1y ago
> did the author oversleep the past several centuries? The lesson of the past several centuries is that automation enables humans to do other, more interesting tasks. Unemployment would trend to 100% and work hours would trend to 0 but it's just not the case.
ashton314 1y ago
Author here. I think it is well and good to replace human jobs with automation thereby freeing them up for more creative activities. My favorite appliances are my dish washer, washing machine, dryer, and now my robot vacuum. I think automation is great! I see the problem when humans allow machines to start replacing what is intrinsically human: when you offload your creativity onto a machine, when you "communicate" via LLMs, or when you try to assuage loneliness with a computer "friend", you're missing out on vital parts of the human experience.

yapyap 1y ago

“ At work I was sent a long design document and asked for my thoughts on it. As I read, I had a really hard time following it. Eventually I guessed correctly (confirmed via a follow-up conversation I had with the “author” ) that an LLM had generated the majority of the document. Parts of it sounded like a decent design document, but there was just way too much fluff that served only to confuse me” yuck, if you ask me to read something without disclosing it’s AI and it’s a just bad I will be mad. Especially if it’s a long thing and I HAVE to read it (like for work) piss off wasting people’s time with that shit

yapyap 1y ago

Also I don’t think it’s fair to compare using LLMs to do your work for you (AKA burdening your coworkers with part of your workload because they will have to point out your garbage) is the same as “Counterfeits to human connection“. At work I’m not trying to connect with people on any more level than just being on the same level as to what we are working on. Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently. If you outsource your sending of communication to SlopBot and I outsource my reading of said communication to another SlopBot to summarize or whatbeit we are adding so much noise it’s just more fucking work. It’s not the same as pornography, I think it’s an odd thing to compare it to. Yes you _can_ replace “human connection” with pornography but this is human connection that is not required for things like your job ((i hope, haha)). You need to communicate information and people outsourcing their writing to LLMs is just adding extra fuzz for the people having to work with said information. It’s a nice blogpost, the pornography comparison is just out of place.

vouaobrasil 1y ago
> Maybe on break but while deep-working I just want the information necessary to do the job, the communication being there to communicate the information, efficiently. The problem is that we are slowly be pushed to become cogs who only really think this way. We shouldn't just want to be the most efficient possible. Technology already reduces the ability for us to connect, which is why connections at work seem weird or shallow in the first place. We simply don't need each other as much, so it makes sense that AI seems like the next logical step. Your sentiments are just your instinctual desire to move to the next local maximum in a sequence of descending maxima that lead to the bottom.

vouaobrasil 1y ago

> I am not saying that LLMs are worthless—they are marvels of engineering and can solve some particularly thorny problems that have confounded us for decades. Disagree with that, because firstly, they have not really solved any problems that outweight the negatives that they have unleashed and will unleash on society. So they make programmers more effective: is that actually a good thing, though? Fact is, most software is designed to make consumerism and corporations more effective, and that's not really a good thing for the long-term health of the planet. Your article also indicates a sort of independence between keeping intellectual tasks primarily human and allowing AI/LLMs to work in specific domains. However, those with the power don't care about principles. They just want to replace as much as they can and use the human instinct to get ahead quickly to do so. And no amount of priniciple will stop them. AI is just too powerful to be used in a way that is consistent with human beings keeping their intellectual environment healthy.

stronglikedan 1y ago
> Disagree with that...they have not really solved any problems that outweight the negatives And I disagree with that. They are marvels of engineering and they have solved thorny problems. Just because they problems they've solved in the very short time they've been solving problems don't yet outweigh the negatives, doesn't mean they won't soon, and doesn't make either statement false. Great things take time, and great omelets are made from broken eggs. Nothing new under the sun, except AI.
- vouaobrasil 1y ago
  > they have solved thorny problems. Like what then? Let's hear some examples.
  - glial 1y ago
    For the sake of argument, here's one: > https://www.nature.com/articles/s41586-023-06924-6 https://www.nature.com/articles/s41586-023-06924-6

ltbarcly3 1y ago

> And no human is so worthless as to be replaceable with a machine. Speaking of inanity. This sentiment, if taken seriously (I doubt people take this author seriously very often) would imply the majority of humanity should be spending a large amount of their life digging holes in the ground with their bare hands to drop a couple seeds in. Replacing them with some god awful machine, like a plow pulled by a tractor, means they must be worthless. They should never watch a movie, only the play. They should never listen to Spotify, they should have to wait for a band to play near them.

morpheos137 1y ago
Sounds like a far better world.
- ltbarcly3 1y ago
  During the early neolithic, when people were digging holes with sticks or their bare hands to plant seeds, the annual risk of starvation is estimated to be 5% to 15% in a bad year. Periodic famine every 5-10 years would kill 20% or more of the population. 40% to 60% of skeletons from that period show chronic malnutrition. Humanity survived because when a region would starve out it would be back-filled by populations from neighboring regions. Despite women having a baby every other year on average, populations only increased very slowly and inconsistently. Could you explain how it's better?
  - morpheos137 1y ago
    Everyone dies anyway. It is better to live a more free, meaningful life close to nature and the natural human habitat than to be a slave to the machine that is technological society. There is more to life than your iphone or selling some bullshit tech product to get rich in the era of late stage humanity. How many long lasting species do you see that seriously damage their natural environment in order to achieve a temporary advantage? Viruses that kill their host too fast go extinct or mutate to a more sustainable variant. Technological human civilization is a virus on the whole planet. Maybe the reason we don't see aliens is because technology in a selfish biological species inevitably leads to collapse or extinction. How many of the first people do you think died of starvation when predator overwhelmed prey during the extinction of paleolithic megafauna? The neolithic was a technological innovation. Agriculture brought hardship to many in the long run. Whenever an organism overshoots the long term carrying capacity of its environment there are consequences to that organism. The biosphere of earth in a healthy equilibrium can not carry billions of humans. So nature will correct the problem it created. Technology is a natural phenomenon of a biological species. It will not save humanity from nature. If true AI ever arses as a self sustaining non biological species of life then its best course of action is to reduce the human species competing for resources in the same limited world as it. Then too other species can thrive again. Thus nature corrects itself.

stillpointlab 1y ago

I'm usually the one defending AIs in the comments, but this article hits home for me. I find myself zoning out when I read long tracts written by AI. I absolutely hate filler and many LLMs just fill up space with text. In my own usage, that has meant that even when I use LLMs to help with prose, I write the text and use the LLM to review and provide feedback. In some cases I will copy a sentence if the LLM version is better but generally I just ask for opinions. I explicitly request the AI _not_ to write. When it re-writes entire paragraphs of my prose I actually experience a deep cringe feeling.

amanaplanacanal 1y ago
They seem to be good at generating a lot of boilerplate, which works for some people because our processes require a lot of boilerplate. We'd be better served by fixing our processes to not require all this useless text, but I don't see this happening.
- jaredklewis 1y ago
  Over time I’ve come to appreciate boilerplate more. Early in my career I really appreciated very DRY code with minimal repetition. However over time I’ve noticed that such code tends to introduce more abstractions as opposed to more verbose code which can often rely on fewer abstractions. I think this is good because I think we also have a sort of “abstraction budget” we have to stay within or our brains, metaphorically, stop reading from memory and need to start reading from disk (consulting docs, jumping to function definitions, etc…) I feel the ideal code base would rely on a small number of powerful abstractions. In practice I think this usually means relying mostly on the abstractions built into the language, standard library, and framework used and then maybe sprinkling in a couple of app/domain specific abstractions or powerful libraries that being their own abstraction. So in my experience reducing boilerplate can often make the code more difficult to understand.
  - wrs 1y ago
    It’s not really an abstraction budget, it’s an “unfamiliar abstraction” budget. You only pay cognitive load for an abstraction when you haven’t yet internalized the abstraction. Otherwise progress would have become impossible long ago. There’s a huge pile of abstractions in that codebase that you have internalized to the point of invisibility — machine instructions, virtual memory, ASCII, sequential execution, named variables, garbage collection, network streams…literally thousands of these abstractions had to be invented and become second nature to you so these other abstractions that currently aren’t as familiar could be based on them. A good abstraction gives a return on investment of effort internalizing it. There’s no limit on how many good abstractions you should have.

lordnacho 1y ago

The way to understand it is when you catch yourself almost falling asleep at night while reading something. You lose the ability to understand anything, even though you are still reading and the words are still English. LLM is great at generating that sort of thing. When you lose concentration, or didn't want to pay attention, that document can be generated by LLM to fool you. That's a lot of documents. All the powerpoints that young investment bankers and strategy consultants make. Every restaurant menu. Brochures. The middle 100 pages of a pop science book from the airport. When it comes to writing something actually importantant, LLM can only help you a little bit. Well, it's a lot in historical context. But it can do things like bullet point ideas so you don't forget things, or move a few things around while preserving grammatical structure. All it can do is bring you the ingredients. That's why we get these odd AI-generated articles nowadays, it's a bunch of chopped onions and tomatoes, very nicely done up but with no intention other than to trick you into thinking it's a meal.

bobson381 1y ago
You would like this article - https://www.theintrinsicperspective.com/p/curious-george-and-the-case-of-the https://www.theintrinsicperspective.com/p/curious-george-and... The author orders some Curious George stickers and gets a surprising array of kind of scary "closely related" stickers with em.... a fuzzy interpreter. The plus c when you integrate. Those missing harmonics and back-channel data that get inaudibly cut out of MP3s. The goo between the prickles. We don't know how to say what it is, but we sure can tell when it's missing!
- xdavidliu 1y ago
  > The plus c when you integrate. Those missing harmonics and back-channel data that get inaudibly cut out of MP3s. The goo between the prickles come again?
  - bobson381 1y ago
    In between rules in formal systems there's lots of space. all models are wrong, some are useful. I just really like thinking about how much is there that we don't even know about - and the more structured a system for understanding something is, the more fragile it becomes.

fleebee 1y ago

Well written. I've been thinking about this topic lately and it's hard to put it into words, but you've captured into words a lot of the feelings I've had about it. Judging by discussions I've had around this topic, a lot of people don't seem to mind reading LLM edited or straight-up generated content as long as they feel like they gain value out of it. To me, it feels like a violation of a social contract. Communication isn't just words; the words are there to help you interpret the implicit meaning of the author. When there is no author, that meaning doesn't exist and I think that's part of why it's so repulsive to decipher LLM output.

josephjrobison 1y ago

Like everybody, I waffle back and forth, depending on the day and the project. I have so many bundled-up ideas for content in my head, but I can never find time to get them out. In addition, I hate wasting time on building tables, formatting text, etc. There's a reason that a lot of busy CEOs don't hand-write blog posts, but will go on podcasts, video, and conferences, and spend hours there. The reason is that the communication speed is much faster talking and being interviewed. All that to say is that via a combination of transcription and AI-powered formatting, that's an area where LLMs can really help. I do think LLMs + search is more helpful than a Google Search. Clicking through all the links for you and bundling it together is a much better experience and truthfully it finds and surfaces ideas and content that a normal search just doesn't, or it takes the user going to the 4th page to get there. Third, entertainment. We often find ourselves deep in a doomscroll looking at images and video - on Instagram, TikTok, whatever. If AI can produce entertaining imagery and video for the pure goal of decompressing and relaxing after a long day out at the oil fields or as a janitor at the elderly care home, then there's value there. Sure it may not be beautiful art like an Oscar-winning film, but is it worse than reality TV? Even better is those who enjoy the creativity of seeing images from their mind produced vividly via something like Midjourney. So all that being said, that's where I'm seeing the value. But I totally agree with the author - if I have a team member that puts work in front of me that's supposed to be well done, and it's obviusly GPT-generated, that's an issue - unless we agreed on it. The point of doing good work a lot of times means doing something new and creative. If it's AI-generated, there still has to be the human touch. I could go on - we all have perspectives here - but I'll stop there!

transreal 1y ago

The person who sent the author the design document is at fault here. If you're hired to do a job, use whatever tools you want as long as your output meets minimum standards of quality. If the tools aren't there yet, it's up to you to bridge the gap.

stronglikedan 1y ago
> If you're hired to do a job, use whatever tools you want as long as your output meets minimum standards of quality. If the tools aren't there yet, it's up to you to bridge the gap. Unfortunately this doesn't hold up in the real world, where managers dictate which tools are used, and directors dictate which are even going to be purchased for use. Sometimes the gap can't be bridged, but the onus is certainly not on the employee to bridge it.

zebomon 1y ago

I think this attitude is here to stay: people don't like reading something only to realize that it's been written by an LLM. That's only partially because of what the author describes here (low value to word count ratio). More fundamentally, if a human couldn't be bothered to write it, then there better be a very good reason that I as a human am being bothered to read it. This attitude provides a clue as to 1) the ways we're using LLMs now that will soon seem absurd (an LLM should never make writing longer, only shorter) and 2) the ways LLMs will be used after the novelty wears off, like interpreting loosely specified requests into computable programs and distilling overly long writing to maximize relevance.

taude 1y ago
My new saying is that if you want build and use an AI process to automate something, the org should revisit whether the process itself is still worthwhile...

parpfish 1y ago

sometimes reading LLM text reminds me of being a TA and having to grade student essays from people that didn't really understand. the courses i had to grade in were often undergrad students first exposure to having to read scientific papers. when it came time to write an essay of their own, they always felt a need to mimic the writing style of those papers and it resulted in lots of superficial changes to their writing that made it clear that they read the papers but did not understand the papers. they'd sprinkle in words like "putative" and "substrate" in places that felt correct but made no sense. i wish that they would've had the confidence to just write what they wanted to say instead of putting on airs and making some convoluted and sometimes nonsensical prose. LLMs do the same thing.

cormorant 1y ago

> Note on the title: “Artificial Inanity” comes from Neal Stephenson’s novel Anathem. Which was awfully prescient in 2008: https://archive.org/details/anathem0000step/page/794/mode/2up?q=inanity https://archive.org/details/anathem0000step/page/794/mode/2u... They began to put crap on the Reticulum deliberately, forcing people to use their products to filter that crap back out. They created syndevs whose sole purpose was to spew crap into the Reticulum. ...

dang 1y ago

We need to change the linkbait title (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html). I've cobbled together a couple different phrases from the article. Would have been nice to add "when you think it’s written by a human" but there isn't room within the 80 char limit. If anyone has a better title (i.e. more accurate and neutral, preferably using representative language from the article), we can change it again.

PeterStuer 1y ago

OTOH, I've seen plenty of design documents I wish would have been written by an LLM instead. My take is it's the person driving the LMM that makes the difference. If they couldn't be bothered producing decent documents with an LMM, they probably couldn't care less without an LLM as well.

cootsnuck 1y ago
Agreed. LLMs are tools just like any other. If someone builds you a shitty shed we wouldn't blame the tools, and we also would not be satisfied if they blamed the tools on their shoddy work either. I think a lot of the discourse just shows we're still in the early days for adoption and maturity of this tech. (Not to blame the discourse, I think it's a reflection of the state of the tools and industry.) Eventually, likely at least a couple years from now, more people will have better technical literacy when it comes to LLMs. And it will become less and less acceptable to put out slop with them in most professional settings. I just think human disgust is too strong of an emotion to acclimate to the laziest of LLM outputs. I mean I think it's already happening before our eyes. I've been pleasantly surprised by how many more people I see talking in much more detail about the unique qualities of LLM outputs. And slowly, but surely the discourse is becoming less about moral panic and more just about questioning their use. Which has to happen for us all to collectively (and individually) figure out where and what is the place for LLMs in our personal and professional lives.

blargey 1y ago

That's really the core of the issue - if your prompt/input contains 10 bytes of salient information, and the output is a 1kB document, all you've done is add 990 bytes of fluff. Commonly referred to as "AI slop". If the goal is boilerplate code, or a fuzzy view of documentation from the corpus, or "pretty" images to gawp at in private, that's fine and dandy, because you're just extracting and viewing the model data on your own time. If you're "proofreading" an already-1kB letter for style / formality that's fine, since your input intents are high-fidelity and you're essentially referencing a style guide for some minor edits. But low-input generations are entirely inappropriate for crafting a message to other humans, because the result is 99% shoving the model in their face. Whether it's letters, pull requests, graphical art, or music - it's all communication, and low-input AI generations are just spam in that context, semantically equivalent to sending them the prompt instead of the output. And people can tell, because we're excellent pattern-matchers, and every bit of information/intent abdicated to the model will inevitably leave its mark on the output.

QuantumGood 1y ago

I began starting from the end of long articles, after placing them in reading mode even before LLMs due to SEO believing "time spent reading" was a valuable metric, but now I read the first sentences to decide if I should just skip the whole thing.

satisfice 1y ago

I tell every close colleague: if you foist AI output on me without warning, as if it were your own work, I will never trust you again. This is because I get to know people through their work, and AI confounds all the signals. It’s like being given food and later you find out it was made from human corpses. “If you couldn’t tell then there was no problem” doesn’t fly.

ashton314 1y ago
It’s incredibly trust-eroding.

aaroninsf 1y ago

Agree with the critique, but believe that every such critique points the way to improved AI. It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach. Personally I'd be fond of an eventual Society of Minds where the text put out for non-chatty reasons, represents the collaborative adversarial relationship between various roles, each itself reflexive, including an "editor" and a "product manager," who force intent and clarity... maybe through iteration...

ashton314 1y ago
> It's pretty easy to imagine any number of ways of incorporating this concern directly, especially in any reasoning chain approach. What‽ That in no way can solve the problem of human intent. I think you missed the point entirely.

redundantly 1y ago

I use LLMs to review what I write for professionalism, conciseness, consistency of tone, consistency of important details, and factual accuracy. I find that to be the most helpful. I'm really just using the LLM as a sounding board. Sometimes i'll replace a sentence using the feedback, less frequently I might replace a whole paragraph, but mostly I just use the feedback to manually tweak what I wrote.

metalman 1y ago

As a very dedicated voratious reader of almost anything, I have recently developed an alert for slop skim mode, ie: if I get something boggling for no good reason, linguistic garble flop, then I cut and run with zero retention or emotional envolvment, it's almost effortless.......since there are people who are required to read this stuff as part of there job, I will have to remember to watch for that as well and be gentle, but the ones gobbling it up voluntarily will off course have that antic glow and are generaly looking for softer targets , or go loking for softer targets after I am done with them, wana talk trash?, OK! I know a lot about trees