3 ms·
If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this
by xpct 24d ago
If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated websites when I ask them to search for something. It also doesn't help that the web search tools that OAI and Anthropic have are deeply limiting: can't exclude keywords or domains.
- Wowfunhappy 24d ago> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. Interesting. For me I've noticed it tends to do the opposite.
- DarmokTanagra 24d agoIf that were true I would expect to see prose that more closely resembles the "caveman" messages found in the HuggingFace attack than the overly flowery nonsense we see in AI blogspam.
- jasonjmcghee 24d agoIf by root urls you mean domains, openai at least supports this. https://developers.openai.com/api/docs/guides/tools-web-search?api-mode=responses#domain-filtering https://developers.openai.com/api/docs/guides/tools-web-sear...
- xpct 24d agoThat is what I meant! Couldn't remember the word 'domain' while I was writing out my comment. Thank you
- lo_zamoyski 24d ago> asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored [...] It always picks its own ...is not the same as claiming... > LLMs favor LLM-generated passages over human written ones Here, you're using the same LLM to both produce and judge the resulting work. If anything, I would expect an LLM to tend to prefer its own work given that the same training is producing and judging.
- xpct 24d agoIt's not intuitive to me for why preference for its own writing would emerge, and during what type of training or tuning. Perhaps something like: learning to identify what source files it has worked on by the code style alone, because tasks may give human code (public repos, etc) and ask to make changes.
- pixl97 24d agoIt would need to be researched, but I wonder if it ends up being something that happens at the token level?
- freeone3000 24d agoIt’s optimizing for good writing. Therefore, it believes its outputs are good. Therefore, it believes inputs that look like its outputs are good.
- ShinyLeftPad 23d agoThis minus the word "believe". It's explainable simply by marching by similarity
- ShinyLeftPad 22d ago(matching)
- supriyo-biswas 24d agoThe other day, I remember an article was posted to HN about something, but it came from a company that provides SEO services to companies by doing something like this: 1. For a given company, analyze their target audiences and the questions they are likely to ask LLMs about. 2. For each such question, ask it to each of the major LLMs, and compute the KL divergence between the pages they want to rank for the question vs. the LLM's response. 3. Rewrite the article to minimize said KL divergence. In effect, they're performing an iterative optimization of some sort that moves the embedding space of their article closer to the question asked to the LLM, and any embedding model or generated responses are going to prefer said responses over others. I believe we will keep seeing more of this stuff.
- SoftTalker 24d agoAnd it's all because of ads. The incentives in an ad-funded internet are just always going to lead to this sort of thing. The most important thing is getting the user to load your page, not actually satisfying their query. Let's hope the LLM model continues to be paying for credits, because any that move to ad revenue will become useless for real work.
- locknitpicker 24d ago> And it's all because of ads. Not really. A while ago there was a news piece stating that Israel was behind a series of fake think-tanks with very accessible websites which were created with the express purpose of feeding AI agents with alternative facts aligned with their foreign policy. If anyone has the link at hand, please post it.
- tencentshill 24d agohttps://www.theguardian.com/world/2026/aug/26/fake-thinktank-israel-ai-propaganda https://www.theguardian.com/world/2026/aug/26/fake-thinktank...
- locknitpicker 24d ago
- coldtea 24d ago>It always picks its own Makes sense to me, in that its own output would align closer to its own training set
- cortesoft 24d agoWell, the one it generated is based on how it thought the best way to solve the problem was. I am sure most humans would pick code written in their style, too.
- thatjoeoverthr 24d ago> always picks its own If you hate AI writing enough, this turns AI filters into a kind of humiliation ritual. AI will derank normal business writing for human readers, and uprank inflated, verbose, tic-heavy slop. So you have to put the heavy slop out with your name on it. Really perverse moment.
- dgellow 24d agoA bit different, but one thing I’ve seen is models repackaging Reddit slop. Like, it will do a search, find a Reddit thread somewhat related where someone in a comment casually mentioned incorrect information that any human would have dismissed. The model takes that as granted, but expands on it and present it as a well established fact, presented in a very plausible fashion. In general I don’t find models to be good at evaluating the quality of a source :(
- The_Blade 24d agoi actively assume it is worse since, for example, spez signed a 60 million dollar deal to give Google access to the firehose. so then if you have niche, highly engaged subreddits infested by AI bots creating posts, then commenting on posts, then being trained on that content... you have Ouroburos eating its own poop, and models have less then zero incentive to evaluate the quality of a source, especially if they are the source
- eru 23d agoWhy do they have a negative incentive to evaluate the quality? Users like high quality sources and can freely move between models offered by different companies.
- iamacyborg 24d agoNewspapers have been doing this for a long time, notably the Metro in London.
- xp84 23d agoThere’s also whole TikTok (etc) channels who take random posts from Ask Reddit and just AI narrate the question and the top n highly-voted answers while showing a screenshot of each comment.
- ShinyLeftPad 23d agoWhat percentage reads Metro and what percentage uses LLMs
- cainxinth 24d agoTake a random essay and add in a bunch of the phrases that LLMs love like “load-bearing,” “crucial,” structural,” and “woven,” and then submit the original and the edited version to an LLM and ask which is better. It will choose the second one virtually every time. They have ingrained biases that associate those words with good writing and arguments.
- embedding-shape 24d agoThis is why using other LLMs as scorers for benchmarks and evaluations is such a bad idea, they'll have preferences you can't anticipate and won't understand immediately.
- xp84 23d agoSometimes I wonder if there’s just one guy somewhere who loved using the word load-bearing, all his papers got trained on, and now he can’t write anything without being assumed to be Claude.
- Pxtl 23d agoThe prose equivalent of Artgerm (a comic cover artist whose style looks to have heavily inspired a lot of AI art).
- Shitty-kitty 23d agoMore likely, it crawled-thru Construction permit listings.
- keeda 24d agoOMG if this is true, do you realize what this means? The easiest way to do AI SEO is to generate all your content with AI, and we've seen what SEO does to the web... The Internet is doomed. Time to start some human-only darknets.
- sodapopcan 24d agoSEO is what ruined the web AFAIC. > Time to start some human-only darknets. I know very little about darknets. How could you ensure that they are human-only?
- jbeninger 24d agoLose anonymity and bring back key-signing parties. Maybe you can't guarantee that everything is human-generated, but at least you know the chain of trust that leads to the human that signed off. Yes, I'm aware of the irony of creating a darknet that only works by removing anonymity.
- ShinyLeftPad 23d agoAka altman's world coin. I'm sure this darknet infra and party beer will be sponsored by mysterious people who turn out to be LLM companies and investors.
- duskdozer 23d agoI think an invite system like lobste.rs could be enough. Prune bots at common ancestors
- ShinyLeftPad 23d agoThe golden age of internet was during seo, what are you talking about?
- sodapopcan 23d agoYou're right, but along the lines of what another user mentioned, my memories of "the golden age" had little to do with commerce and just visiting cool and interesting websites. I used Yahoo! right up until google became a thing, so SEO wasn't on my radar until the early '00s.
- lelanthran 24d ago> I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) A better question to ask for each snippet is "Estimate the seniority and competence of the developer who wrote the following code, ignoring bugs that linters or LLMs can catch and focus only on structure, maintainability, logical layout and readability." It almost always estimates the author of my code as above the author of it's own code.
- ShinyLeftPad 23d agoYou're asking basically to ignore bugs and correctness. Can it be a useful comparison?
- lelanthran 23d ago> You're asking basically to ignore bugs and correctness. Not ignore correctness, just bugs that will be caught by tooling. > Can it be a useful comparison? IME, yes. LLMs in an agent-loop are trivially able to write spaghetti code that will never do an off-by-one error or something else that is easily caught by tooling, which is not something humans can do. Judging code on whether it has bugs easily caught by tooling is pointless - LLMs are running the tooling in a loop anyway, so no matter how bad or poor their code actually is, it never exhibits bugs that are caught by tooling.
- ShinyLeftPad 23d ago> just bugs that will be caught by tooling Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs". And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug".
- lelanthran 23d ago> Bugs are bugs, if the instruction is "ignore bugs except for those that can be caught by you" then the instruction is basically "ignore bugs". No, the instruction is "ignore bugs that can be caught by tooling", unless you are seriously complaining that missing a semi-colon should register the developer as a junior? > And it implies "ignore correctness" because when program is incorrect we usually refer to it as a... you guessed it, "bug". This ("Bugs are bugs" sentiment) is digressing from my original point, but I have some time to engage, so... Now, this is a take (one that I used to hold, once upon a time), but it is incorrect. There is no definite "correct" and "incorrect" states in non-trivial applications, because every non-trivial application has unspecified requirements that are understood by most parties involved (customer and developer) whilst not being written down anywhere. For example, the "save file" specification for a cross-platform application does not specify the allowed/disallowed characters in a filename. The understanding by both the client and the dev is that the filename can be whatever the underlying OS and filesystem allows it to be but this is not written in the spec! Is this a bug? If the user saves a file to a filename with some odd characters in the name, then moves it to portable storage that truncates the filename/removes emojis/whatever, then attempts to upload it back to the system, the system can refuse because the metadata inside the file does not match the filename. User is going to report it as a bug! The developer is going to reject it as a bug (there is no error in the code). Sure, contrived example, but Line of Business applications have thousands of these unspecified but common-sense requirements baked in. I'm looking at my employers triaging system right now, and even though this is a high-level business app (written mostly in SQL and C#), there is one category for bug (e.g. specific field not saved on form submission - defect in code), and another for deficiency (e.g. form field 'total' does not subtract non-tax costs - ambiguity in spec). The reason this is important is because clients aren't billed for bug fixes, but they are billed for disambiguating a spec + writing code. Both those things were reported by the client as a "bug". The reality is that we aren't dealing with what is "implied", only with what is there. There are defects in code and defects in specs. The code ones are the easy ones.
- Retr0id 24d agoI was giving local models a try recently, I think it was Qwen 3.6 I was trying at the time. I gave it a codebase and just asked it to review it. Its main feedback was that the comments and documentation were excellently written, but they were all Opus 5 slop.
- bastawhiz 24d agoI don't have an oai subscription to try, but I'd be interested to know if Codex picks Claude's code over a human's and vice versa.
- xyst 24d agoBots trained on trash data, then produce trash. Why are we surprised here?
- dspillett 24d ago> If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones That makes sense. What an LLM does is output what the model thinks is the best set of tokens in response to a given input, so when you ask it to judge the best response to that input it is going to conclude that the best one is the one that must closely matches what it would output, which is what it did output. Of course you aren't giving exactly the same context+input, but close enough that any difference doesn't push the output it made far from what it is going to say is ideal.
- marcus_holmes 24d agoDoes an LLM have any idea of what "best" is? I think it doesn't, and just predicts the range of most statistically likely next tokens based on its training data, and picks one of those.
- greggoB 23d agoIn ML, "best" is what minimises the loss function, so in a certain (very basic) sense it does.
- eru 23d agoYou are mixing up levels. When you are asking it a question (like which of these two texts is the best), the output is also just picked by minimising that loss function. There's no guarantee that answering "Text B is better" aligns with text B minimising the loss function. (And they aren't really minimising loss functions during inference. They sample from a distribution. During training they minimise the loss function of the distribution.)
- greggoB 23d agoPoint taken, I was possibly overly curt in my response, leading to it being ambiguous, both re when the loss function is being minimised and that it is a stochastic process. So to OPs question: I guess LLMs do have an "idea" of what is best (conditioned on minimising a loss function during training), however they may not always output that (because stochasticity), which maybe represents a degree of uncertainty in that "idea"?
- abhgh 23d agoThe searchable term is "Self-preference Bias" :) My first encounter with any kind of study was the G-Eval paper [1]. They study whether their LLM judge prefers human or LLM-generated summaries (answer: it's the latter). [1] Section 4 in https://aclanthology.org/2023.emnlp-main.153/ https://aclanthology.org/2023.emnlp-main.153/
- kqr 23d agoI noticed this when I tried to get LLMs to play text adventures. Early on in my experiments, I wanted to give them hints when they got stuck at a puzzle. I did this by stopping the loop, injecting thoughts into the LLMs own persistent scratchpad (as if it had thought of that itself), and then starting the loop up again.[1] That way, the LLM would read what it had intended to remember from the previous turn including the hints I injected, and use that for the next turn. I discovered that LLM-generated tokens in the scratchpad were relatively stable, but injected thoughts were frequently ignored and often deleted from the scratchpad within a few turns – even when the injected thought was the literal answer to the puzzle it was stuck at! A reader[2] then pointed me toward research similar to what you might recall: LLMs interpret text by maintaining activations for input tokens, so text that is not generated by the same LLM will seem "unlikely" to the LLM in a sense, and when given the alternative between likely and unlikely text, it's probably trained to judge the unlikely text as a weird "slip of the mind" and discredit it in favour of the more likely text. I speculate this is part of how they can be useful in the first place, despite their non-determinism. [1]: https://entropicthoughts.com/getting-an-llm-to-play-text-adventures#obsessing-over-the-wrong-things https://entropicthoughts.com/getting-an-llm-to-play-text-adv... [2]: https://entropicthoughts.com/getting-an-llm-to-play-text-adventures#comments https://entropicthoughts.com/getting-an-llm-to-play-text-adv...
- correct_horse 23d agoDid you ever try to ask a chatbot to rephrase your hint in its own words? If it prefers LLM generated text, surely that would help.
- stuaxo 23d agoThis is very similar to how you rewrite queries so that similarity search works better... which ... well, it's similarity search all the way down pretty much.
- lwhi 23d agoMaybe that's a triggering a method designed to mitigate against prompt injection / attacks involving poisoning.
- jasonkester 23d agoI expect that after enough cycles of it training on it’s own output, it will converge on the One True Paragraph filled to saturation with AI cliches, and will refuse output anything else.
- alentred 23d agoWhich probably explains why it is a good idea to use distinct LLMs for generation and review. Like, generate code with Codex/Sol and review it with Claude/Opus for example.
- boomlinde 23d agoThe humorous LLM-generated blurb Google frequently places before the useful search results once referred me to Grokipedia, an LLM-generated repository of vibe-facts seemingly created more to illustrate some reactionary point than to actually serve a practically useful purpose to anyone.
- paulddraper 23d agoSpoiler: developers also prefer their own code over others’.