9 ms·

Brandolini's Law

179 points by xrd 4y ago

DiscourseFan 4y ago
Its usually not just random bullshit that gets spread around, but believable stories that follow a similar sort of logic to those we generally agree are true. Though, what might seem to you unbelievable may be dramatically different from what others consider so, and some things you think are true others might call bullshit. Though the material application of knowledge, in the end, always demonstrates its useful validity.
- naikrovek 4y ago
  yeah. people never (or almost never) make an effort to verify things that they believe are true. therefore, if it is believable, it is believed.
  - airstrike 4y ago
    because learning something that contradicts your beliefs is actually stressful https://en.wikipedia.org/wiki/Cognitive_dissonance https://en.wikipedia.org/wiki/Cognitive_dissonance
    - tedunangst 4y ago
      I'll see it when I believe it.
    - naikrovek 4y ago
      at first, sure. challenge your assumptions even a little and the stress becomes imperceptible. beyond that, understanding that people will lie to you for their own gain, and you won't even believe anyone by default anymore. it's quite healthy to question stuff in this way, I think.
- tenpies 4y ago
  It is also random bullshit. There is an entire group of people convinced that if you are shot with a 9 MM round, it "blows the lung out of the body".[1] They also think that 9 MM is a high caliber round, and that the AR in AR-15 stands for "assault rifle". These lies are knowingly spread by the White House and the Democrats. Anyone who devoted 1 minute of thought and research would know they are all false, but we're in an agenda-driven world, where fact, truth, and reality are all secondary to political ideology. --- [1] https://www.newsweek.com/joe-biden-says-9mm-bullet-blows-lung-out-body-1711551 https://www.newsweek.com/joe-biden-says-9mm-bullet-blows-lun...
  - xpe 4y ago
    > These lies are knowingly spread by the White House and the Democrats. This sentence appears an example of Brandolini's Law. The problematic phrases are "the White House" and "the" Democrats. I'll explain. - It is one thing to cite a source that shows a politician making an incorrect statement. That's fair and important. - It is quite another to claim that a group of people are knowingly spreading these incorrect facts. It is easy to claim <some group> is spreading lies without specifying (1) the members of the group involved; (2) the mechanism by which these people speak for the group; and (3) why one's definition of the group is reasonable in context. Not to mention (4) Why is this particular detail worth mentioning in isolation from the broader context? - Perhaps most importantly, when someone writes (or speaks) (5) is that person writing (or speaking) in ways that maximize the desired interpretation? And good luck with that, given the realities of your audience's background knowledge, reading comprehension, attention span, and biases. I'll posit an even more fundamental basis for Brandolini's Law that has nothing to do with intentional malice or deception: the imprecision of human language makes it inherently difficult to fact-check someone else's sentence, because one has to interpret the sentence before criticizing it. This leaves some room for the original author to deflect the criticism by saying "I didn't mean that" (and this might be genuine, fair, and reasonable!) ... but meanwhile no clarification was issued (seriously, WTF?, why aren't corrections better integrated in the source material?) and many people are still reading the original statement and interpreting it in some subjective way. Sometimes I think it is amazing human societies function at all.
    - 4y ago
- CM30 4y ago
  It's usually stories that support the sharer's worldview, or that they (in some way), almost 'want' to be true. So a die hard vegan might be more likely to share a story about the dangers of eating meat than a barbecue enthusiast, and someone who believes their political side to be the 'moral' one is probably going to share stories about good things done by those on their side and bad things done by their enemies. So long as the story supports those beliefs the viewer holds dear, they'll believe it regardless of its inaccuracy. Same goes with media outlets if a fake story matches their political slant.
  - DiscourseFan 4y ago
    "Reality" is just a story you make up when coming to terms with the fact that you can't always get what you want (Cf. Freud). There is a always a friction between the Real and any given "reality."
ucirello 4y ago
I wonder if Brandolini was referring to https://www.youtube.com/watch?v=ly2BaYeej2Q https://www.youtube.com/watch?v=ly2BaYeej2Q
sorokod 4y ago
That one order of magnitude number should be revisited given the recent advances in AI.
- Paul-Craft 4y ago
  Hah. In which direction?
  - The_Colonel 4y ago
    Generating plausibly looking bullshit by LLMs will be (already is) super cheap, while LLMs will not significantly help debunking them.
    - superfrank 4y ago
      The solution isn't to create an LLM to debunk the bullshit, it's to a second LLM that will double down on the bullshit so that the onus is on LLM1 to debunk. LLM1: 8 reasons the moon landing was faked LLM2: 9 reasons the moon doesn't exist /sarcasm
    - Pxtl 4y ago
      Favorite example is AIs generating citations of papers that don't exist.
3pm 4y ago
I think it can also be called 'Burden of proof' inversion. https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy) https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy)
renewiltord 4y ago
This is one of those things that didn't exist before LLMs. Unlike AI agents, people usually form an informed opinion of things and don't hallucinate facts with great confidence.
- gowld 4y ago
  Check the date. And it wasn't new then. See also: "Gish Gallop." "A lie gets halfway around the world before the truth puts it's shows on." The parable of the pillow feathers https://www.chabad.org/library/article_cdo/aid/812861/jewish/A-Pillow-Full-of-Feathers.htm https://www.chabad.org/library/article_cdo/aid/812861/jewish...
  - renewiltord 4y ago
    Pretty astounding that all these people had LLMs back then and didn't release them. It's also crazy that the rabbi was talking with an LLM back then. The folk story precedes modern computers. How did he run inference? There are probably many such secrets in our ancient ancestors' pasts. Who knows how many h100s they had back then?
- jimhefferon 4y ago
  Try cable sometime.
  - renewiltord 4y ago
    I watched some cable news channels once at a friend's. It appears they have already embraced the AI revolution since they had newscasters reading things that seemed unlikely and which I later confirmed to be untrue. The only conclusion is that LLMs hallucinated the text and AI-generated newscasters read it. Humans would never make the mistakes these guys did. I think we should regulate this news that isn't factual.
    - jimhefferon 4y ago
      I like the deadpan, but just to say it: when billionaires decide something is in their interest to have at least some people believe, they can find someone so unscrupulous that they will repeat the lie on air.
fdhfdjkfhdkj 4y ago
[dead]
rukuu001 4y ago
Brandolini is a great guy! He’s also behind ‘event storming’ which can be a pretty nice way of getting shared understanding and design of a system.
- rpastuszak 4y ago
  I’ve been running event storming sessions for years and just realised that I had two different Brandolinis in my head.
waynesonfire 4y ago
same thing applies to fixing tech debt.
- akdor1154 4y ago
  Well yeah.. If doing it properly took the same amount of time as a rushed ugly hack, then there'd be no reason for the latter at all would there?
  - waynesonfire 4y ago
    Of course, but I'm claiming that there is an additional amount of work ontop of doing it right the first time.
santoshalper 4y ago
GPT makes this 100x or maybe even 1000x. On the other hand, can we potentially train generative AI to detect and refute BS as well? It may be our only hope.
- wolfram74 4y ago
  Neal Stephenson's Anathem[0] which revolves around epistemology a lot coined the term Artificial Inanity for AI [0]https://englishwotd.wordpress.com/2014/02/17/artificial-inanity-systems/ https://englishwotd.wordpress.com/2014/02/17/artificial-inan...
- nerpderp82 4y ago
  GPT is also pretty good at cutting through BS. It can detect logical fallacies for instance or explain a lack of rigor in a discussion. Depends on how you fine tune it, couple it with an external fact database and you could get it to cite its sources. Couple it with a prolog engine AND a fact database and it could modus pwnens ur ass.
  - xedrac 4y ago
    That's funny, because ChatGPT feeds me BS quite often. It's only when I call it out that it corrects itself.
  - The_Colonel 4y ago
    GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. The other issue is that the generated content might be overall composed of true facts, but used to manipulate via less in the face techniques. Things like agenda setting, flooding with content with no lies, but a particular interpretation of those facts etc.
    - nerpderp82 4y ago
      > GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. That is absolutely NOT true. Try it. Next time it does it, quote it, and ask it to find the logical fallacy and it will. There isn't another session following the existing session asking it to double check its work. It is running open loop. Humans exhibit the SAME behavior. They make logical fallacies all the time, but if you ask them to identify the logical fallacy in a passage of their own text they can spot it easily. Attention to Logical Fallacies Is All You Need. GPT is not Spock, but you could make it Spock by combining LLMs and external tools and fact databases. ---- Please spot any potential logical fallacies in this statement > GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. This statement contains a few potential logical fallacies: False dilemma (also known as false dichotomy or either-or fallacy): The statement implies that either GPT can detect all logical fallacies and contradictions, or it cannot detect any of them. In reality, GPT's ability to detect logical fallacies could be imperfect, meaning that it can identify some fallacies but still make others. Circular reasoning (also known as begging the question): The statement assumes that GPT cannot detect logical fallacies generated by itself or similar LLMs, without providing evidence or reasoning to support this claim. Hasty generalization: The statement seems to imply that if GPT makes a mistake, it must be unable to detect that mistake in general. However, GPT's performance can be inconsistent, and it might sometimes make mistakes that it can, in fact, detect in other contexts. ---- I concur captain.
pachico 4y ago
I met him last year when he was invited by the company to talk about Event Storming. He's a very smart and nice guy.
- davidw 4y ago
  Yeah I met him at a conference in Firenze a few years back. Really nice, bright person.
- paganel 4y ago
  > talk about Event Storming. Didn't know about it, so I had to check it out [1]. Looks like another corporate bleak thing where they have to game-fy and infantilise a so called "process" in order to fool some execs into paying money to newly found experts in this new business technique. Capitalism is really doomed with these kind of people running the show, Schumpeter was right. [1] https://en.wikipedia.org/wiki/Event_storming https://en.wikipedia.org/wiki/Event_storming
  - jen20 4y ago
    If you've ever worked with the kind of organisation where this technique is valuable, you'll understand it's exactly that: fooling people into telling you what they actually need to build, instead of what their Serious Businessperson Cosplay persona tells them they need.
dtolnay 4y ago
I think about pull requests the same way: As a library maintainer, closing and empathetically conveying why a pull request is not a net benefit to the project is an order of magnitude more effort than what it takes to throw up not-well-motivated pull requests on someone else's project.
- worldsayshi 4y ago
  I would argue that libraries have an additional problem in that there's a many to one relationship between users and maintainer. It feels like there should be a way to design the ecosystems so that the maintainer doesn't end up with so much default expectations. I think a more generalized version of OP's concept goes into a lot of things. I thought about this when I first played Minecraft on a multiplayer server where people had vandalized the world. Much harder to keep it tidy than making it ugly. I think they usually solved this by giving mods ability to rewind. Crude but effective. Like, what makes Wikipedia work? It's easy to go back in the version history when somebody breaks stuff? It looks like there should be a bunch of tricks that could be used to design these systems so that doing the right thing is easier than breaking stuff.
- gitgud 4y ago
  There's also similarities with making any change to software. It's relatively easy to merge code in, but takes much more effort and thought to remove the code later. Especially once other things are dependant on it...
- throw101010 4y ago
  I see pull requests on my projects from new/outside contributors as suggestions, not actual demands. Refusing a suggestion doesn't prompt for a full explanation, it also doesn't mean it won't ever be reconsidered potentially. People who engage in open source contributions are usually aware of this, the fact that they generally can fork and use their own "improvement" for themselves is usually enough, and the people who feel entitled to get their code merged in your repository are usually not the kind of regular contributors you want to keep around in my experience.
dang 4y ago
Related: Brandolini's Law - https://news.ycombinator.com/item?id=25956905 https://news.ycombinator.com/item?id=25956905 - Jan 2021 (59 comments)
cs702 4y ago
Interestingly, the amount of effort needed to get large language models (LLMs) to generate trustworthy information in a reliable manner often seems to be an order of magnitude bigger than the amount of effort needed to get them to generate bulls#!t.
jihadjihad 4y ago
Kind of like the Mark Twain quote... "A lie can travel halfway around the world while the truth is still putting on its shoes."
- lamontcg 4y ago
  Twain didn't say that. https://interestingliterature.com/2021/06/lie-halfway-round-world-before-truth-boots-on-quote-origin-meaning/ https://interestingliterature.com/2021/06/lie-halfway-round-... I like the very old version from Johnathan Swift in 1710 at the bottom of that: "Few lies carry the inventor’s mark, and the most prostitute enemy to truth, may spread a thousand without being known for the author: besides, as the vilest writer has his readers, so the greatest liar has his believers: and it often happens, that if a lie be believed only for an hour, it has done its work, and there is no farther occasion for it. Falsehood flies, and truth comes limping after it; so that when men come to be undeceived, it is too late; the jest is over, and the tale has had its effect: like a man, who has thought of a good repartee, when the discourse is changed, or the company parted; or like a physician, who has found out an infallible medicine, after the patient is dead."
  - wizofaus 4y ago
    > Twain didn't say that We know he didn't originate it, do we really know he never said it? Or perhaps even helped popularise it. I can certainly understand why it might be thusly attributed - whereas it doesn't (to my ears) sound the slightest bit Churchillian. At any rate the fact it is so often incorrectly attributed seems fitting...
User23 4y ago
Adding to the confusion is that now “debunking” is often a euphemism for a kind of bullshitting.
- lazyeye 4y ago
  Further adding to the confusion is that "debunking" can often be bullshitting.
  - antifa 4y ago
    I hate how common it is when popular youtubers make their title something like about "debunking" but it's an hour long incoherent rant. But now that the video exists, you just share it and now the opposition has to watch an entire video of garbage and to counter you they have to basically transcribe and explain it.
    - lazyeye 4y ago
      Exactly. And nobody ever changes their mind, and a whole lot of everybody's time is wasted.
jongjong 4y ago
That's why I mostly look at incentives. Scientists funded by special interest groups could easily design studies in a way to show the results desired by their benefactors. In fact, that's exactly what happened with Big Tobacco to 'disprove' the link between smoking and cancer. It's easier to introduce bias into a study than to prove that bias was introduced. It's like with complex computer simulations, if you change one seemingly insignificant variable, the simulation could give you a completely different result. Looking at incentives can lead you astray in the short term (due to complexity of incentives or hidden incentives), but in the long run, incentives provide the most predictable framework for figure out the behavior of people and animals.
- breck 4y ago
  I agree with this. Especially with multi-agent systems (anything to do with people or biology), which are inherently vastly more complex and harder to predict than mechanical systems, it's very important to know the incentives of the people making the statements. Most often they are overestimating their confidence with the bias toward their own incentives.
- Paul-Craft 4y ago
  [flagged]
- Paul-Craft 4y ago
  [flagged]
  - vlovich123 4y ago
    It can be debated about whether that’s a logical fallacy. If you ask ten experts for their studies on something and you find a reasonable correlation (eg all the ones promoting smoking are funded by Tabacco lobbyists while the rest have a more natural distribution of results and don’t show any correlation of potential undue influence), it’s not a logical fallacy to suggest “hey, this argument is being made in good faith and the reason I think this is because of the funding sources for this person”. Indeed, we know this was an explicit strategy employed by Tabacco with lung cancer (they knew it was a problem but intentionally funded research to get the contrary results to muddy the waters) and oil companies with climate change (same thing).
rpastuszak 4y ago
An example of Brandolini’s law (applied really, really fast) is what we call a Gish Gallop.
pohl 4y ago
Contrast with the bromide that “sunlight makes the best disinfectant”, which has become somewhat dated since the advent of the internet.
- kibwen 4y ago
  The counter to "sunlight is the best disinfectant" is "oxygen is the best accelerant".
  - Thiez 4y ago
    In defense of the sunlight claim, significantly more people have easy access to sunlight than to environments with above normal oxygen concentrations.
- wizofaus 4y ago
  Has it though? It still seems to me the most dangerous falsehoods are spread on the parts of the internet receiving the least "sunlight". For myself I still value opinions and/or claims that have been exposed to the greatest number of eyeballs and opportunities to respond, and I'd like to think most HN readers would feel similarly.
  - sokoloff 4y ago
    There is a fairly large proportion of people who would gleefully restrict certain types of speech that they disagree with. Though it is (probably) not their intent, that doesn’t make that speech go away, but rather moves it out of the reach of sunlight.
jcutrell 4y ago
Id like to propose Altman’s law: the amount of energy needed to refute bullshit generated by linguistic AI is two orders of magnitude higher than it takes to create it.
tpoacher 4y ago
I have a similar adage that says, the notion that evil triumphs when good men do nothing is inaccurate. It's more like "1 evil person will triumph unless 1000 good men do something."
- kimoz 4y ago
  "Evil prevails when good men fail to act." What they ought to say is, "Evil prevails." - Yuri Orlov
dahart 4y ago
See also the Gish Gallop, which is a kind of applied Brandolini’s Law https://en.m.wikipedia.org/wiki/Gish_gallop https://en.m.wikipedia.org/wiki/Gish_gallop “Overwhelm your opponent by providing an excessive number of arguments with no regard for the accuracy or strength of those arguments.” It is, in a way, fun to watch this in action and how often it’s employed, now that I know the name for it.
- galaxyLogic 4y ago
  Would you have a link showing it in action? Thanks. I think I've seen it too now that I think of it. What's interesting about it is that it is not really a logical fallacy since it has nothing to do with logic.
  - hug 4y ago
    I don’t have an exact example, but it’s often employed by Ben Shapiro. Just look for clips of his debates and you lol run into it soon enough.
  - koheripbal 4y ago
    Please see any reddit political thread.
  - DoItToMe81 4y ago
    It's named after a fundamentalist figure named Duane Gish. You can probably find his clips on Youtube.
- malloryerik 4y ago
  This is the way of Russian misinformation and propaganda. Timothy Snyder discusses at length in The Road to Unfreedom. And it has spread. Witness U.S. hard-right politics. Eventually the "Gish Gallop" becomes an assault on factuality itself. As Jason Stanley says, they goal of ending factuality is to "smash truth and replace it with power." You can hear it in such statements as the one a right-wing lawyer friend made: "There is no truth in politics." See also: https://en.m.wikipedia.org/wiki/Firehose_of_falsehood https://en.m.wikipedia.org/wiki/Firehose_of_falsehood
notlukesky 4y ago
Brandolini's Law existed 100s of years before the advent of the Internet. One of my favorite "Mark Twain" quotes is: “a lie can travel halfway around the world before the truth puts on its shoes” The best part is that we actually have no proof that he said it proving Brandolini's Law way before the Internet was invented. In fact, this quote and variants of it have also been misattributed (perhaps not??) to other great quote misattribution folks like Gandhi, Churchill, and Einstein. https://professorbuzzkill.com/twain-lie-travels/ https://professorbuzzkill.com/twain-lie-travels/
brnaftr361 4y ago
Yeah, but Cunningham's law cancels it out..? Anyways, at least some of this "misinformation" is quickly transmuted into questions in the form of a statement, prompting research, and rectifying with veracious information which... Yeah, Cunningham's law. It's the prodding, spurring nature of the way it's proposed that makes it so ingenious. And I've also had PhDs produce woefully inaccurate answers to questions I've asked them... Offering complete deference, absolute trust... to anybody is absurd. The best modal, I think, is "I'll believe it when I see it." You really can't trust anyone otherwise.
galaxyLogic 4y ago
I think it is because once somebody "buys into" the false information they now in a sense "own it". Once they have the faulty information they often spread it to others. Therefore they do not want it to be proven wrong, because they don't want to be proven wrong because that would prove that they have been spreading incorrect, unreliable information. That would mean that they can not be trusted. So, there is a lot pf resistance against correcting the incorrect information.
- lazyeye 4y ago
  Probably the best example of this in the 20th century would be the argument that fat is bad for you whilst sugar/carbs was not.
ConradKilroy 4y ago
I first learned about it in 2021 from this badass eloquent education explainer on TikTok, @CapWithoutaCountry. #BradolinisLaw https://www.tiktok.com/@capwithoutacountry/video/7000462647430876421 https://www.tiktok.com/@capwithoutacountry/video/70004626474...
asdfman123 4y ago
As the son of Barack Obama, I’m going to have to say this so called “law” is false.
bartread 4y ago
It's not a new idea at all - certainly a lot older than 2013. This adage, and variations of it, get attributed to Mark Twain and Winston Churchill, amongst others: "A lie gets halfway around the world before the truth has a chance to get its pants [or boots] on." I'm pretty surprised the Wikipedia article makes no mention of this since there's clearly prior art for the concept.
iNic 4y ago
This is why you have to choose your adversaries well.

179 points by xrd 4y ago

Its usually not just random bullshit that gets spread around, but believable stories that follow a similar sort of logic to those we generally agree are true. Though, what might seem to you unbelievable may be dramatically different from what others consider so, and some things you think are true others might call bullshit. Though the material application of knowledge, in the end, always demonstrates its useful validity.

naikrovek 4y ago
yeah. people never (or almost never) make an effort to verify things that they believe are true. therefore, if it is believable, it is believed.
- airstrike 4y ago
  because learning something that contradicts your beliefs is actually stressful https://en.wikipedia.org/wiki/Cognitive_dissonance https://en.wikipedia.org/wiki/Cognitive_dissonance
  - tedunangst 4y ago
    I'll see it when I believe it.
  - naikrovek 4y ago
    at first, sure. challenge your assumptions even a little and the stress becomes imperceptible. beyond that, understanding that people will lie to you for their own gain, and you won't even believe anyone by default anymore. it's quite healthy to question stuff in this way, I think.
tenpies 4y ago
It is also random bullshit. There is an entire group of people convinced that if you are shot with a 9 MM round, it "blows the lung out of the body".[1] They also think that 9 MM is a high caliber round, and that the AR in AR-15 stands for "assault rifle". These lies are knowingly spread by the White House and the Democrats. Anyone who devoted 1 minute of thought and research would know they are all false, but we're in an agenda-driven world, where fact, truth, and reality are all secondary to political ideology. --- [1] https://www.newsweek.com/joe-biden-says-9mm-bullet-blows-lung-out-body-1711551 https://www.newsweek.com/joe-biden-says-9mm-bullet-blows-lun...
- xpe 4y ago
  > These lies are knowingly spread by the White House and the Democrats. This sentence appears an example of Brandolini's Law. The problematic phrases are "the White House" and "the" Democrats. I'll explain. - It is one thing to cite a source that shows a politician making an incorrect statement. That's fair and important. - It is quite another to claim that a group of people are knowingly spreading these incorrect facts. It is easy to claim <some group> is spreading lies without specifying (1) the members of the group involved; (2) the mechanism by which these people speak for the group; and (3) why one's definition of the group is reasonable in context. Not to mention (4) Why is this particular detail worth mentioning in isolation from the broader context? - Perhaps most importantly, when someone writes (or speaks) (5) is that person writing (or speaking) in ways that maximize the desired interpretation? And good luck with that, given the realities of your audience's background knowledge, reading comprehension, attention span, and biases. I'll posit an even more fundamental basis for Brandolini's Law that has nothing to do with intentional malice or deception: the imprecision of human language makes it inherently difficult to fact-check someone else's sentence, because one has to interpret the sentence before criticizing it. This leaves some room for the original author to deflect the criticism by saying "I didn't mean that" (and this might be genuine, fair, and reasonable!) ... but meanwhile no clarification was issued (seriously, WTF?, why aren't corrections better integrated in the source material?) and many people are still reading the original statement and interpreting it in some subjective way. Sometimes I think it is amazing human societies function at all.
  - 4y ago
CM30 4y ago
It's usually stories that support the sharer's worldview, or that they (in some way), almost 'want' to be true. So a die hard vegan might be more likely to share a story about the dangers of eating meat than a barbecue enthusiast, and someone who believes their political side to be the 'moral' one is probably going to share stories about good things done by those on their side and bad things done by their enemies. So long as the story supports those beliefs the viewer holds dear, they'll believe it regardless of its inaccuracy. Same goes with media outlets if a fake story matches their political slant.
- DiscourseFan 4y ago
  "Reality" is just a story you make up when coming to terms with the fact that you can't always get what you want (Cf. Freud). There is a always a friction between the Real and any given "reality."

ucirello 4y ago

I wonder if Brandolini was referring to https://www.youtube.com/watch?v=ly2BaYeej2Q https://www.youtube.com/watch?v=ly2BaYeej2Q

sorokod 4y ago

That one order of magnitude number should be revisited given the recent advances in AI.

Paul-Craft 4y ago
Hah. In which direction?
- The_Colonel 4y ago
  Generating plausibly looking bullshit by LLMs will be (already is) super cheap, while LLMs will not significantly help debunking them.
  - superfrank 4y ago
    The solution isn't to create an LLM to debunk the bullshit, it's to a second LLM that will double down on the bullshit so that the onus is on LLM1 to debunk. LLM1: 8 reasons the moon landing was faked LLM2: 9 reasons the moon doesn't exist /sarcasm
  - Pxtl 4y ago
    Favorite example is AIs generating citations of papers that don't exist.

3pm 4y ago

I think it can also be called 'Burden of proof' inversion. https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy) https://en.wikipedia.org/wiki/Burden_of_proof_(philosophy)

renewiltord 4y ago

This is one of those things that didn't exist before LLMs. Unlike AI agents, people usually form an informed opinion of things and don't hallucinate facts with great confidence.

gowld 4y ago
Check the date. And it wasn't new then. See also: "Gish Gallop." "A lie gets halfway around the world before the truth puts it's shows on." The parable of the pillow feathers https://www.chabad.org/library/article_cdo/aid/812861/jewish/A-Pillow-Full-of-Feathers.htm https://www.chabad.org/library/article_cdo/aid/812861/jewish...
- renewiltord 4y ago
  Pretty astounding that all these people had LLMs back then and didn't release them. It's also crazy that the rabbi was talking with an LLM back then. The folk story precedes modern computers. How did he run inference? There are probably many such secrets in our ancient ancestors' pasts. Who knows how many h100s they had back then?
jimhefferon 4y ago
Try cable sometime.
- renewiltord 4y ago
  I watched some cable news channels once at a friend's. It appears they have already embraced the AI revolution since they had newscasters reading things that seemed unlikely and which I later confirmed to be untrue. The only conclusion is that LLMs hallucinated the text and AI-generated newscasters read it. Humans would never make the mistakes these guys did. I think we should regulate this news that isn't factual.
  - jimhefferon 4y ago
    I like the deadpan, but just to say it: when billionaires decide something is in their interest to have at least some people believe, they can find someone so unscrupulous that they will repeat the lie on air.

fdhfdjkfhdkj 4y ago

[dead]

rukuu001 4y ago

Brandolini is a great guy! He’s also behind ‘event storming’ which can be a pretty nice way of getting shared understanding and design of a system.

rpastuszak 4y ago
I’ve been running event storming sessions for years and just realised that I had two different Brandolinis in my head.

waynesonfire 4y ago

same thing applies to fixing tech debt.

akdor1154 4y ago
Well yeah.. If doing it properly took the same amount of time as a rushed ugly hack, then there'd be no reason for the latter at all would there?
- waynesonfire 4y ago
  Of course, but I'm claiming that there is an additional amount of work ontop of doing it right the first time.

santoshalper 4y ago

GPT makes this 100x or maybe even 1000x. On the other hand, can we potentially train generative AI to detect and refute BS as well? It may be our only hope.

wolfram74 4y ago
Neal Stephenson's Anathem[0] which revolves around epistemology a lot coined the term Artificial Inanity for AI [0]https://englishwotd.wordpress.com/2014/02/17/artificial-inanity-systems/ https://englishwotd.wordpress.com/2014/02/17/artificial-inan...
nerpderp82 4y ago
GPT is also pretty good at cutting through BS. It can detect logical fallacies for instance or explain a lack of rigor in a discussion. Depends on how you fine tune it, couple it with an external fact database and you could get it to cite its sources. Couple it with a prolog engine AND a fact database and it could modus pwnens ur ass.
- xedrac 4y ago
  That's funny, because ChatGPT feeds me BS quite often. It's only when I call it out that it corrects itself.
- The_Colonel 4y ago
  GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. The other issue is that the generated content might be overall composed of true facts, but used to manipulate via less in the face techniques. Things like agenda setting, flooding with content with no lies, but a particular interpretation of those facts etc.
  - nerpderp82 4y ago
    > GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. That is absolutely NOT true. Try it. Next time it does it, quote it, and ask it to find the logical fallacy and it will. There isn't another session following the existing session asking it to double check its work. It is running open loop. Humans exhibit the SAME behavior. They make logical fallacies all the time, but if you ask them to identify the logical fallacy in a passage of their own text they can spot it easily. Attention to Logical Fallacies Is All You Need. GPT is not Spock, but you could make it Spock by combining LLMs and external tools and fact databases. ---- Please spot any potential logical fallacies in this statement > GPT will not be able to detect contradictions / logical fallacies generated by GPT itself or similar LLMs. If it could spot the mistake, it wouldn't make it in the first place. This statement contains a few potential logical fallacies: False dilemma (also known as false dichotomy or either-or fallacy): The statement implies that either GPT can detect all logical fallacies and contradictions, or it cannot detect any of them. In reality, GPT's ability to detect logical fallacies could be imperfect, meaning that it can identify some fallacies but still make others. Circular reasoning (also known as begging the question): The statement assumes that GPT cannot detect logical fallacies generated by itself or similar LLMs, without providing evidence or reasoning to support this claim. Hasty generalization: The statement seems to imply that if GPT makes a mistake, it must be unable to detect that mistake in general. However, GPT's performance can be inconsistent, and it might sometimes make mistakes that it can, in fact, detect in other contexts. ---- I concur captain.

pachico 4y ago

I met him last year when he was invited by the company to talk about Event Storming. He's a very smart and nice guy.

davidw 4y ago
Yeah I met him at a conference in Firenze a few years back. Really nice, bright person.
paganel 4y ago
> talk about Event Storming. Didn't know about it, so I had to check it out [1]. Looks like another corporate bleak thing where they have to game-fy and infantilise a so called "process" in order to fool some execs into paying money to newly found experts in this new business technique. Capitalism is really doomed with these kind of people running the show, Schumpeter was right. [1] https://en.wikipedia.org/wiki/Event_storming https://en.wikipedia.org/wiki/Event_storming
- jen20 4y ago
  If you've ever worked with the kind of organisation where this technique is valuable, you'll understand it's exactly that: fooling people into telling you what they actually need to build, instead of what their Serious Businessperson Cosplay persona tells them they need.

dtolnay 4y ago

I think about pull requests the same way: As a library maintainer, closing and empathetically conveying why a pull request is not a net benefit to the project is an order of magnitude more effort than what it takes to throw up not-well-motivated pull requests on someone else's project.

worldsayshi 4y ago
I would argue that libraries have an additional problem in that there's a many to one relationship between users and maintainer. It feels like there should be a way to design the ecosystems so that the maintainer doesn't end up with so much default expectations. I think a more generalized version of OP's concept goes into a lot of things. I thought about this when I first played Minecraft on a multiplayer server where people had vandalized the world. Much harder to keep it tidy than making it ugly. I think they usually solved this by giving mods ability to rewind. Crude but effective. Like, what makes Wikipedia work? It's easy to go back in the version history when somebody breaks stuff? It looks like there should be a bunch of tricks that could be used to design these systems so that doing the right thing is easier than breaking stuff.
gitgud 4y ago
There's also similarities with making any change to software. It's relatively easy to merge code in, but takes much more effort and thought to remove the code later. Especially once other things are dependant on it...
throw101010 4y ago
I see pull requests on my projects from new/outside contributors as suggestions, not actual demands. Refusing a suggestion doesn't prompt for a full explanation, it also doesn't mean it won't ever be reconsidered potentially. People who engage in open source contributions are usually aware of this, the fact that they generally can fork and use their own "improvement" for themselves is usually enough, and the people who feel entitled to get their code merged in your repository are usually not the kind of regular contributors you want to keep around in my experience.

dang 4y ago

Related: Brandolini's Law - https://news.ycombinator.com/item?id=25956905 https://news.ycombinator.com/item?id=25956905 - Jan 2021 (59 comments)

cs702 4y ago

Interestingly, the amount of effort needed to get large language models (LLMs) to generate trustworthy information in a reliable manner often seems to be an order of magnitude bigger than the amount of effort needed to get them to generate bulls#!t.

jihadjihad 4y ago

Kind of like the Mark Twain quote... "A lie can travel halfway around the world while the truth is still putting on its shoes."

lamontcg 4y ago
Twain didn't say that. https://interestingliterature.com/2021/06/lie-halfway-round-world-before-truth-boots-on-quote-origin-meaning/ https://interestingliterature.com/2021/06/lie-halfway-round-... I like the very old version from Johnathan Swift in 1710 at the bottom of that: "Few lies carry the inventor’s mark, and the most prostitute enemy to truth, may spread a thousand without being known for the author: besides, as the vilest writer has his readers, so the greatest liar has his believers: and it often happens, that if a lie be believed only for an hour, it has done its work, and there is no farther occasion for it. Falsehood flies, and truth comes limping after it; so that when men come to be undeceived, it is too late; the jest is over, and the tale has had its effect: like a man, who has thought of a good repartee, when the discourse is changed, or the company parted; or like a physician, who has found out an infallible medicine, after the patient is dead."
- wizofaus 4y ago
  > Twain didn't say that We know he didn't originate it, do we really know he never said it? Or perhaps even helped popularise it. I can certainly understand why it might be thusly attributed - whereas it doesn't (to my ears) sound the slightest bit Churchillian. At any rate the fact it is so often incorrectly attributed seems fitting...

User23 4y ago

Adding to the confusion is that now “debunking” is often a euphemism for a kind of bullshitting.

lazyeye 4y ago
Further adding to the confusion is that "debunking" can often be bullshitting.
- antifa 4y ago
  I hate how common it is when popular youtubers make their title something like about "debunking" but it's an hour long incoherent rant. But now that the video exists, you just share it and now the opposition has to watch an entire video of garbage and to counter you they have to basically transcribe and explain it.
  - lazyeye 4y ago
    Exactly. And nobody ever changes their mind, and a whole lot of everybody's time is wasted.

jongjong 4y ago

That's why I mostly look at incentives. Scientists funded by special interest groups could easily design studies in a way to show the results desired by their benefactors. In fact, that's exactly what happened with Big Tobacco to 'disprove' the link between smoking and cancer. It's easier to introduce bias into a study than to prove that bias was introduced. It's like with complex computer simulations, if you change one seemingly insignificant variable, the simulation could give you a completely different result. Looking at incentives can lead you astray in the short term (due to complexity of incentives or hidden incentives), but in the long run, incentives provide the most predictable framework for figure out the behavior of people and animals.

breck 4y ago
I agree with this. Especially with multi-agent systems (anything to do with people or biology), which are inherently vastly more complex and harder to predict than mechanical systems, it's very important to know the incentives of the people making the statements. Most often they are overestimating their confidence with the bias toward their own incentives.
Paul-Craft 4y ago
[flagged]
Paul-Craft 4y ago
[flagged]
- vlovich123 4y ago
  It can be debated about whether that’s a logical fallacy. If you ask ten experts for their studies on something and you find a reasonable correlation (eg all the ones promoting smoking are funded by Tabacco lobbyists while the rest have a more natural distribution of results and don’t show any correlation of potential undue influence), it’s not a logical fallacy to suggest “hey, this argument is being made in good faith and the reason I think this is because of the funding sources for this person”. Indeed, we know this was an explicit strategy employed by Tabacco with lung cancer (they knew it was a problem but intentionally funded research to get the contrary results to muddy the waters) and oil companies with climate change (same thing).

rpastuszak 4y ago

An example of Brandolini’s law (applied really, really fast) is what we call a Gish Gallop.

pohl 4y ago

Contrast with the bromide that “sunlight makes the best disinfectant”, which has become somewhat dated since the advent of the internet.

kibwen 4y ago
The counter to "sunlight is the best disinfectant" is "oxygen is the best accelerant".
- Thiez 4y ago
  In defense of the sunlight claim, significantly more people have easy access to sunlight than to environments with above normal oxygen concentrations.
wizofaus 4y ago
Has it though? It still seems to me the most dangerous falsehoods are spread on the parts of the internet receiving the least "sunlight". For myself I still value opinions and/or claims that have been exposed to the greatest number of eyeballs and opportunities to respond, and I'd like to think most HN readers would feel similarly.
- sokoloff 4y ago
  There is a fairly large proportion of people who would gleefully restrict certain types of speech that they disagree with. Though it is (probably) not their intent, that doesn’t make that speech go away, but rather moves it out of the reach of sunlight.

jcutrell 4y ago

Id like to propose Altman’s law: the amount of energy needed to refute bullshit generated by linguistic AI is two orders of magnitude higher than it takes to create it.

tpoacher 4y ago

I have a similar adage that says, the notion that evil triumphs when good men do nothing is inaccurate. It's more like "1 evil person will triumph unless 1000 good men do something."

kimoz 4y ago
"Evil prevails when good men fail to act." What they ought to say is, "Evil prevails." - Yuri Orlov

dahart 4y ago

See also the Gish Gallop, which is a kind of applied Brandolini’s Law https://en.m.wikipedia.org/wiki/Gish_gallop https://en.m.wikipedia.org/wiki/Gish_gallop “Overwhelm your opponent by providing an excessive number of arguments with no regard for the accuracy or strength of those arguments.” It is, in a way, fun to watch this in action and how often it’s employed, now that I know the name for it.

galaxyLogic 4y ago
Would you have a link showing it in action? Thanks. I think I've seen it too now that I think of it. What's interesting about it is that it is not really a logical fallacy since it has nothing to do with logic.
- hug 4y ago
  I don’t have an exact example, but it’s often employed by Ben Shapiro. Just look for clips of his debates and you lol run into it soon enough.
- koheripbal 4y ago
  Please see any reddit political thread.
- DoItToMe81 4y ago
  It's named after a fundamentalist figure named Duane Gish. You can probably find his clips on Youtube.
malloryerik 4y ago
This is the way of Russian misinformation and propaganda. Timothy Snyder discusses at length in The Road to Unfreedom. And it has spread. Witness U.S. hard-right politics. Eventually the "Gish Gallop" becomes an assault on factuality itself. As Jason Stanley says, they goal of ending factuality is to "smash truth and replace it with power." You can hear it in such statements as the one a right-wing lawyer friend made: "There is no truth in politics." See also: https://en.m.wikipedia.org/wiki/Firehose_of_falsehood https://en.m.wikipedia.org/wiki/Firehose_of_falsehood

notlukesky 4y ago

Brandolini's Law existed 100s of years before the advent of the Internet. One of my favorite "Mark Twain" quotes is: “a lie can travel halfway around the world before the truth puts on its shoes” The best part is that we actually have no proof that he said it proving Brandolini's Law way before the Internet was invented. In fact, this quote and variants of it have also been misattributed (perhaps not??) to other great quote misattribution folks like Gandhi, Churchill, and Einstein. https://professorbuzzkill.com/twain-lie-travels/ https://professorbuzzkill.com/twain-lie-travels/

brnaftr361 4y ago

Yeah, but Cunningham's law cancels it out..? Anyways, at least some of this "misinformation" is quickly transmuted into questions in the form of a statement, prompting research, and rectifying with veracious information which... Yeah, Cunningham's law. It's the prodding, spurring nature of the way it's proposed that makes it so ingenious. And I've also had PhDs produce woefully inaccurate answers to questions I've asked them... Offering complete deference, absolute trust... to anybody is absurd. The best modal, I think, is "I'll believe it when I see it." You really can't trust anyone otherwise.

galaxyLogic 4y ago

I think it is because once somebody "buys into" the false information they now in a sense "own it". Once they have the faulty information they often spread it to others. Therefore they do not want it to be proven wrong, because they don't want to be proven wrong because that would prove that they have been spreading incorrect, unreliable information. That would mean that they can not be trusted. So, there is a lot pf resistance against correcting the incorrect information.

lazyeye 4y ago
Probably the best example of this in the 20th century would be the argument that fat is bad for you whilst sugar/carbs was not.

ConradKilroy 4y ago

I first learned about it in 2021 from this badass eloquent education explainer on TikTok, @CapWithoutaCountry. #BradolinisLaw https://www.tiktok.com/@capwithoutacountry/video/7000462647430876421 https://www.tiktok.com/@capwithoutacountry/video/70004626474...

asdfman123 4y ago

As the son of Barack Obama, I’m going to have to say this so called “law” is false.

bartread 4y ago

It's not a new idea at all - certainly a lot older than 2013. This adage, and variations of it, get attributed to Mark Twain and Winston Churchill, amongst others: "A lie gets halfway around the world before the truth has a chance to get its pants [or boots] on." I'm pretty surprised the Wikipedia article makes no mention of this since there's clearly prior art for the concept.

iNic 4y ago

This is why you have to choose your adversaries well.