7 ms·

Strong evidence is common (2021)

62 points by amadeuspagel 3y ago

conformist 3y ago
This is a nice insight. I slightly disagree with the first example in a gambling context, although it makes sense in the context of information theory. I don’t think the choice of prior is right when giving 20:1 posterior odds you’d be willing to bet on. A better prior should be related to the probability of a random person lying about their name when introducing themselves. And the posterior then doesn’t really depend a lot on what name he says.
- layer8 3y ago
  It also depends on how much hinges on the assertion being true or false. If you lose a million dollars if it turns out his name isn’t “Mark Xu”, you’d take much more care to verify the claim.
- nkurz 3y ago
  I'm not sure. Given that there is an individual standing in front of you, does it ever make sense to use the odds of a random individual instead of your best estimate for that particular individual? At the least, it seems likely that the circumstance in which the name is being given would change things greatly: a police officer stopping someone in the dark versus a minister introducing themselves after a sermon. In neither case does a universal estimate of lying about a name seem applicable. And the choice of name does seem important. If someone with no visible appearance of being Asian gives you a very Asian name (or vice versa), you might have a lot more doubt. And if the name is otherwise humorous (Biggus Dickus) or stereotypical (John Doe) this also should affect your estimate. Why would it be a plus that "the posterior then doesn’t really depend a lot on what name he says"?
  - conformist 3y ago
    Yeah, of course, if the name is clearly absurd you’d use the insights from that and update your posterior, and if you have more information about the person you can include it in the prior. The point was that 1/n over all possible names is far from a reasonable prior, and that the Bayesian update barely depends on the name itself (within reason), but on other information gained during the introduction.
hnfong 3y ago
I think most people working with hypotheses and theories should read this. All too often I see authoritative figures dismiss presented evidence because their theories suggest it is (close to) impossible. It often goes like this: A: "Hi my name is Mark Xu" B: "Our models say it's practically impossible that this is true, so it must be an unfortunate glitch, or you are a liar. You don't have a PhD, so I guess I'll assume the latter" Later... "Despite rumors that the person Mark Xu has been spotted near South Park, Colorado, scientists say there is no evidence to suggest he really exists. Xu continues to be a mystery confounding scientists till this day...."
- stavros 3y ago
  Sure, but if the odds of someone's name being Mark Xu are one in a million, the odds of someone seeing Bigfoot (or whatever) are many orders of magnitude lower than that.
  - hnfong 3y ago
    At some point the priors of any theory being wrong is probably more than one in a million. In fact, we know as a historical fact that our current scientific theories are likely to be wrong or inaccurate. Your priors might be different from mine, but positing that odds of Bigfoot (or whatever) is less than one in a trillion or quadrillion isn't that useful.
- epgui 3y ago
  I think you are mischaracterizing both the nature of Bayesian probability updates, and also the foundation underlying scientific expertise. In the first case, prior probabilities are more often than not weakly informative. In the latter case, experts usually rely on a huge volume of information, beyond what most laypeople could even imagine.
mbb70 3y ago
This seems like a reformulation of the birthday paradox. That someone has a specific name is incredibly unlikely, but that someone has a name is almost guaranteed. Situations/phrasing can trick out intuition into paying attention to former more than the latter.
- danbruc 3y ago
  This has barely anything to do with the name, the actual question is how many people would lie about their name in that situation. If nobody lied about their name, then you would know with certainty what that person's name is, if everybody lied about their name, then you would still have no idea, except that it probably is not Mark Xu. Note that this probability is highly context dependent, the chances that your new coworker lies about his name is probably orders of magnitude smaller than that someone you just randomly met in an online chat does so.
- singron 3y ago
  This is the dartboard paradox. If you throw an ideal dart at a dartboard that hits exactly 1 point, the probability that it hits any particular point is 0, but the probability that it hits a point is 1.
jstanley 3y ago
> I’m guessing they would have happily accepted a bet at 20:1 odds that my driver’s license would say “Mark Xu” on it. As soon as you present this bet, my confidence that you are actually called Mark Xu goes down so low that I would not accept it.
- zdragnar 3y ago
  You're assuming that the author was the person presenting the bet, which was not what was stated. Your confidence only dropped because of an intuitive recognition that the author presenting the bet is a signal the author previously lied about his name, as both the source of truth, the claim and motive to profit all lie with the author. None of this is really relevant to the article, so I think it is safer to assume the author meant that some third party presented the bet. Following that, the author's logic makes sense.
  - jstanley 3y ago
    Your confidence should still drop even if someone else presented the bet, because people are more likely to offer bets that they think have positive expected value to them. > the author presenting the bet is a signal the author previously lied about his name More generally: anyone presenting the bet is a signal that that person believes the author lied about his name.
PaulHoule 3y ago
Personally I think machine learning without probability calibration is like a car without tires. That said, you can have an uncalibrated model and write a paper about it in arXiv or you can have a calibrated model and hook it up to a Kelly better and you have an automated trading system. That said, it is frustrating to see something you know is a slam dunk and the model thinks it is 0.97 or 0.7 and if you look at how the calibration works it is often obvious why. The one that has frustrated me most is trying to calibrate full text search systems and never being able to get a confidence much better than 0.70, at least without special-purpose answerers.
- ta988 3y ago
  what's a Kelly in this context?
  - munch117 3y ago
    https://en.wikipedia.org/wiki/Kelly_criterion https://en.wikipedia.org/wiki/Kelly_criterion
  - kqr 3y ago
    A gentler introduction: https://two-wrongs.com/the-misunderstood-kelly-criterion.html https://two-wrongs.com/the-misunderstood-kelly-criterion.htm...
    - PaulHoule 3y ago
      Any gambling system that works is informed by the Kelly Criterion. Here is my favorite https://mitpressbookstore.mit.edu/book/9781635617207 https://mitpressbookstore.mit.edu/book/9781635617207 which is based on the fact that the odds to win are more accurate than the odds to place and show. If you go to the track you will see sometimes that the show odds for a horse are almost as good as the win odds but you have three to win so that is a great bet. Dr. Z’s system constructs a probability estimator for place and show and based on the win odds, then computes the return on investment for the various horses and uses Kelley to decide how much to bet to maximize your returns with a low risk of going broke. Card counters in Blackjack should use Kelly but either one of: (i) winning consistently or (ii) varying your bets a lot risks getting thrown out and doing both will get you a visit from a pit boss who, if you are lucky, will suggest you play roulette or craps instead.
wly_cdgr 3y ago
I still don't get why Sam always lies and tells people his first name is Mark
maartenscholl 3y ago
The reference to the Efficient Market Hypothesis (EMH) here is puzzling. Foremost, the strongest form of the EMH implies that consistent outperformance is not possible, contradicting the rest of the writing and what I believe is its main point: that evidence in the real world is often much stronger than in models. But also when switching from the model world of the EMH to real world markets this application is problematic because of the multiple testing problem. For a trader who recently had a short run of excess profits, the likelihood P(excess returns | skilled) would be high. However, because many diverse traders are in the market and some will show strong performance purely by chance, P(excess returns) or the marginal likelihood of observing such a winning streak would also be high. This dilutes the high likelihood term, leading to a less dramatic update in the posterior P(skilled | excess returns) . Then, the prior P(skilled) should be adjusted to be much more conservative.
- trashtester 3y ago
  Exactly. If you have already outperformed the market, then this is the same as p-hacking through survivorship bias. Which tend to result in hubris. > P(excess returns | skilled) This is not at all clear. In fact, unusually high returns may simply be a result of a string of lucky, possibly correlated, gambles. It may very well be that what separates a skilled investor has just as much or more to do with the ability to restrict the downside. Meaning that even when they lose money, they consistently lose less than the amateurs.
  - bloaf 3y ago
    It could also be illegal activity, e.g. insider trading.
    - trashtester 3y ago
      Absolutely it could.
more_corn 3y ago
There appears to be a missing therefor in the article. He seems to be hinting that the old adage “extraordinary claims require extraordinary evidence” isn’t as convincing as it sounds because strong (but really extraordinary) evidence is all around us. However, there is a decided lack of extraordinary evidence for extraordinary claims. That’s why they strike us as extraordinary. We’ve never seen them before. We’ve never seen anything like them. In fact all prior evidence has been strong evidence for ordinary claims.
bannedbybros 3y ago
[dead]
vannevar 3y ago
Essentially the author is saying that most people are honest most of the time. Maybe so, but truthfulness is only relevant in cases where there are stakes, and in those cases, there is much, much more dishonesty. Put another way, I'd say strong evidence is all around us when it doesn't matter, but considerably scarcer when we are actually interested in the truth.
- pphysch 3y ago
  Exactly, and this is an especially important point when taking about EMH. Markets are mostly noise/manipulation, whereas we assume that "statement of your given name" are mostly signal.
the_af 3y ago
Whenever I see an internet article (not written by a statistician and not a research paper) start with a reference to Bayes Theorem and its purported "perfection", I can safely bet it was written by a LessWrong'er and specifically a fan of Eliezer Yudkowsky. And behold! Just clicking on the Bayes link takes me to another article that immediately mentions Yudkowsky. I wonder what attracts internet people to Yudkowsky's brand of crackpottery, and specifically, what about Bayes elicits their quasi-religious awe.
- embwbam 3y ago
  I don’t identify with the religious zeal around bayes theorem, but can you elaborate on your criticisms of Yudkowsky? I’ve loved Harry Potter and the Methods of Rationality and read some of his essays, and Less Wrong’s coverage of cognitive bias helped me escape religion. I don’t agree with everything I’ve read of his, but I’m very curious at what point he becomes a crackpot.
  - _Algernon_ 3y ago
    I gave up halfway through MoR, but to me it read as a self-insert nerd power fantasy. All non main-characters always end up agreeing with the all knowing MC even though there are obvious gaps in the reasoning.
    - PaulHoule 3y ago
      "Sequences" and other Yudkowksy writings are long and incoherent not by accident but by design. The point is to drive away anybody who has faculties of critical thinking. It's like those ads you see on the internet where they tell you that "TINNITUS IS DESTROYING YOUR BRAIN" or tell you what intermittent fasting plan is right for your shoe size and then send you to some two and a half long video that is there to weed out anybody who isn't incredibly credulous and after all that you find out it is some lame-ass supplement. Or it is like those poorly spelled letters that tell you the widow of some African dictator wants to to send you $15 million by Western Union. Someone who's gotten through all that really is someone vulnerable who can be wrapped around someone's finger. People who are good at critical thinking aren't looking for the secret to become good critical thinkers.
pavlov 3y ago
The article shows examples of ordinary claims backed by ordinary evidence, then jumps to this conclusion: “Extraordinary claims require extraordinary evidence, but extraordinary evidence might be more common than you think.” I saw no evidence of that here. “Strong” is a very different qualifier than “extraordinary.”
kens 3y ago
I don't get the point of this article; is it a response to something else? It seems kind of obvious that there is a lot of strong evidence for many things, but why does that matter? Also, I hate to quibble about details, but someone saying their name is "Mark Xu" isn't really strong evidence that their driver's license says "Mark Xu". There are a lot of people who go by names other than their official name. In my case, I tell everyone my name is Ken, but it's actually Kenneth. In the case of Xu, many people from China use a Western name like Mark that isn't their real name. Edit: I just realized that the odds that someone's name matches their drivers license is at most 6:1, since only 84% of the driving-age US population has a drivers license at all. The paragraph about the Efficient Market Hypothesis is just wrong. Contrary to the article, it is easy to make money on the stock market: historically it has gone up an average of 10% a year. It is also easy to beat the market average: you have roughly a 50-50 chance. How difficult is it to get into the top 1% of traders? Uh, that would be exactly 1 in 100. And I really don't see any connection between this paragraph and the topic of the post. The post seems to have something to do with calibration training, but that link takes me to a signup page for some mystery site, which seems suspicious. I'm trying to interpret this post generously and understand its message, but it seems kind of random.
- the_af 3y ago
  It's typical of LessWrong'er drivel. Lots of words to say something either trivial or wrong, or both. Bayes must be mentioned, because it's their religion.
hn_throwaway_99 3y ago
Yikes, not sure I've seen a front page HN that was this blatantly wrong. The second half of this article is complete nonsense because on one hand, he's seeming to arguing that extreme evidence is "easy to get", but it obviously can't be that easy when it's in a competitive environment where the number of people who could get that info is absolutely limited to something like only 1 percent of people. Taking this quote: > One implication of the Efficient Market Hypothesis (EMH) is that is it difficult to make money on the stock market. Generously, maybe only the top 1% of traders will be profitable. How difficult is it to get into the top 1% of traders? To be 50% sure you’re in the top 1%, you only need 200:1 evidence. This seemingly large odds ratio might be easy to get. As other commenters pointed out, that is absolutely not what the EMH says - it doesn't say it's difficult to be profitable (it actually says the exact opposite, especially when taken together with modern portfolio theory), it says it's difficult (or, rather, impossible) to outperform the market. But the more important point is that the author is implying that it's easy to get enough evidence to put you in the top 1%, as opposed to getting enough evidence to conclude that someone is in the top 1%, or to conclude that they're in any particular percentage actually. In other words, if you take his argument at face value, it may be possible to gather evidence to determine who is in the top 1% of traders, but that's very different from getting enough data to put yourself in that top 1%.
- numbsafari 3y ago
  If you are a grifter influencer, then having Bs “evidence” is all that is needed to “put” you into a presumed category, which lends you an air of credibility sufficient to take money from suckers. Another read is that it doesn’t take much to exploit other people.
throwaway9274 3y ago
There is a particular flavor of writing common to those whose identity is bound up with the so-called “effective altruist” community and their various online forums. It usually combines a reference to Bayes Theorem, a deep hubris, and a significantly misunderstood concept from mathematics or technology. Here, for example, it is clear the author does not grok EMH which applies across the population of trades not traders. Usually this is done to provide the author or community with some claim to an inherent and unearned extraordinary nature, which justifies greater personal power. (You only need 200:1 evidence you’re extraordinary being the takeaway here.) It is unclear why certain SV cliques continue to grant these people power. Their intellectual insularity means that whatever value they once added in bringing logical rigor to conversations is now far outweighed by the train of collective bad logic powering their groupthink. It is increasingly dangerous to have such people steer important societal debates. These debates include crypto regulation where EAs famously associated with SBF, to “AI safety”, a field that appears to be invented out of whole cloth to allow LessWrongers to LARP as ML ethics and fairness experts.
- lusus_naturae 3y ago
  Part of the problem is how easy it is to logically and rationally convince yourself or others of something inaccurate, imprecise or just plain wrong.
  - kelseyfrog 3y ago
    There's a small, but important ego boost that comes along with thinking one has access to a clearer[1] view of reality. The LessWrong community gets this by way of "Bayesian thinking" and trying to subvert their own expectations. For the general population though, you have the Freakanomics effect. 1. The intellectual mind's biggest motivator is certainty. This isn't necessarily a bad thing when certainty aligns with objective reality whatever that means, but then these folks get quite certain about how uncertain they are.
- d0mine 3y ago
  Your comment had more weight if you'd dropped the ad-hominem and kept just the specific critique about EMH.
echohack5 3y ago
Strong evidence isn't common. There we go folks! Since I have now declared strong evidence to not be common you can believe on a 2,000,000:1 probability that in fact this is true!
bjourne 3y ago
Full article is better https://markxu.com/three-bayes-learnings https://markxu.com/three-bayes-learnings
ramblenode 3y ago
The author is asking the wrong question in the name example which is why the odds ratio is so huge. The correct question is what are the odds someone is telling the truth about their name given your expecation that they will tell you their true name when asked. The odds of the name being Mark Xu is irrelevant unless I have some shortlist of names I am comparing against, and then I would use the odds of the name being on the shortlist, not the global odds. It's obvious that the author's question is wrong because I would only be slightly surprised if I discovered that someone lied about their name, which is inconsistent with me having strong evidence for their name.

62 points by amadeuspagel 3y ago

This is a nice insight. I slightly disagree with the first example in a gambling context, although it makes sense in the context of information theory. I don’t think the choice of prior is right when giving 20:1 posterior odds you’d be willing to bet on. A better prior should be related to the probability of a random person lying about their name when introducing themselves. And the posterior then doesn’t really depend a lot on what name he says.

layer8 3y ago
It also depends on how much hinges on the assertion being true or false. If you lose a million dollars if it turns out his name isn’t “Mark Xu”, you’d take much more care to verify the claim.
nkurz 3y ago
I'm not sure. Given that there is an individual standing in front of you, does it ever make sense to use the odds of a random individual instead of your best estimate for that particular individual? At the least, it seems likely that the circumstance in which the name is being given would change things greatly: a police officer stopping someone in the dark versus a minister introducing themselves after a sermon. In neither case does a universal estimate of lying about a name seem applicable. And the choice of name does seem important. If someone with no visible appearance of being Asian gives you a very Asian name (or vice versa), you might have a lot more doubt. And if the name is otherwise humorous (Biggus Dickus) or stereotypical (John Doe) this also should affect your estimate. Why would it be a plus that "the posterior then doesn’t really depend a lot on what name he says"?
- conformist 3y ago
  Yeah, of course, if the name is clearly absurd you’d use the insights from that and update your posterior, and if you have more information about the person you can include it in the prior. The point was that 1/n over all possible names is far from a reasonable prior, and that the Bayesian update barely depends on the name itself (within reason), but on other information gained during the introduction.

hnfong 3y ago

I think most people working with hypotheses and theories should read this. All too often I see authoritative figures dismiss presented evidence because their theories suggest it is (close to) impossible. It often goes like this: A: "Hi my name is Mark Xu" B: "Our models say it's practically impossible that this is true, so it must be an unfortunate glitch, or you are a liar. You don't have a PhD, so I guess I'll assume the latter" Later... "Despite rumors that the person Mark Xu has been spotted near South Park, Colorado, scientists say there is no evidence to suggest he really exists. Xu continues to be a mystery confounding scientists till this day...."

stavros 3y ago
Sure, but if the odds of someone's name being Mark Xu are one in a million, the odds of someone seeing Bigfoot (or whatever) are many orders of magnitude lower than that.
- hnfong 3y ago
  At some point the priors of any theory being wrong is probably more than one in a million. In fact, we know as a historical fact that our current scientific theories are likely to be wrong or inaccurate. Your priors might be different from mine, but positing that odds of Bigfoot (or whatever) is less than one in a trillion or quadrillion isn't that useful.
epgui 3y ago
I think you are mischaracterizing both the nature of Bayesian probability updates, and also the foundation underlying scientific expertise. In the first case, prior probabilities are more often than not weakly informative. In the latter case, experts usually rely on a huge volume of information, beyond what most laypeople could even imagine.

mbb70 3y ago

This seems like a reformulation of the birthday paradox. That someone has a specific name is incredibly unlikely, but that someone has a name is almost guaranteed. Situations/phrasing can trick out intuition into paying attention to former more than the latter.

danbruc 3y ago
This has barely anything to do with the name, the actual question is how many people would lie about their name in that situation. If nobody lied about their name, then you would know with certainty what that person's name is, if everybody lied about their name, then you would still have no idea, except that it probably is not Mark Xu. Note that this probability is highly context dependent, the chances that your new coworker lies about his name is probably orders of magnitude smaller than that someone you just randomly met in an online chat does so.
singron 3y ago
This is the dartboard paradox. If you throw an ideal dart at a dartboard that hits exactly 1 point, the probability that it hits any particular point is 0, but the probability that it hits a point is 1.

jstanley 3y ago

> I’m guessing they would have happily accepted a bet at 20:1 odds that my driver’s license would say “Mark Xu” on it. As soon as you present this bet, my confidence that you are actually called Mark Xu goes down so low that I would not accept it.

zdragnar 3y ago
You're assuming that the author was the person presenting the bet, which was not what was stated. Your confidence only dropped because of an intuitive recognition that the author presenting the bet is a signal the author previously lied about his name, as both the source of truth, the claim and motive to profit all lie with the author. None of this is really relevant to the article, so I think it is safer to assume the author meant that some third party presented the bet. Following that, the author's logic makes sense.
- jstanley 3y ago
  Your confidence should still drop even if someone else presented the bet, because people are more likely to offer bets that they think have positive expected value to them. > the author presenting the bet is a signal the author previously lied about his name More generally: anyone presenting the bet is a signal that that person believes the author lied about his name.

PaulHoule 3y ago

Personally I think machine learning without probability calibration is like a car without tires. That said, you can have an uncalibrated model and write a paper about it in arXiv or you can have a calibrated model and hook it up to a Kelly better and you have an automated trading system. That said, it is frustrating to see something you know is a slam dunk and the model thinks it is 0.97 or 0.7 and if you look at how the calibration works it is often obvious why. The one that has frustrated me most is trying to calibrate full text search systems and never being able to get a confidence much better than 0.70, at least without special-purpose answerers.

ta988 3y ago
what's a Kelly in this context?
- munch117 3y ago
  https://en.wikipedia.org/wiki/Kelly_criterion https://en.wikipedia.org/wiki/Kelly_criterion
- kqr 3y ago
  A gentler introduction: https://two-wrongs.com/the-misunderstood-kelly-criterion.html https://two-wrongs.com/the-misunderstood-kelly-criterion.htm...
  - PaulHoule 3y ago
    Any gambling system that works is informed by the Kelly Criterion. Here is my favorite https://mitpressbookstore.mit.edu/book/9781635617207 https://mitpressbookstore.mit.edu/book/9781635617207 which is based on the fact that the odds to win are more accurate than the odds to place and show. If you go to the track you will see sometimes that the show odds for a horse are almost as good as the win odds but you have three to win so that is a great bet. Dr. Z’s system constructs a probability estimator for place and show and based on the win odds, then computes the return on investment for the various horses and uses Kelley to decide how much to bet to maximize your returns with a low risk of going broke. Card counters in Blackjack should use Kelly but either one of: (i) winning consistently or (ii) varying your bets a lot risks getting thrown out and doing both will get you a visit from a pit boss who, if you are lucky, will suggest you play roulette or craps instead.

wly_cdgr 3y ago

I still don't get why Sam always lies and tells people his first name is Mark

maartenscholl 3y ago

The reference to the Efficient Market Hypothesis (EMH) here is puzzling. Foremost, the strongest form of the EMH implies that consistent outperformance is not possible, contradicting the rest of the writing and what I believe is its main point: that evidence in the real world is often much stronger than in models. But also when switching from the model world of the EMH to real world markets this application is problematic because of the multiple testing problem. For a trader who recently had a short run of excess profits, the likelihood P(excess returns | skilled) would be high. However, because many diverse traders are in the market and some will show strong performance purely by chance, P(excess returns) or the marginal likelihood of observing such a winning streak would also be high. This dilutes the high likelihood term, leading to a less dramatic update in the posterior P(skilled | excess returns) . Then, the prior P(skilled) should be adjusted to be much more conservative.

trashtester 3y ago
Exactly. If you have already outperformed the market, then this is the same as p-hacking through survivorship bias. Which tend to result in hubris. > P(excess returns | skilled) This is not at all clear. In fact, unusually high returns may simply be a result of a string of lucky, possibly correlated, gambles. It may very well be that what separates a skilled investor has just as much or more to do with the ability to restrict the downside. Meaning that even when they lose money, they consistently lose less than the amateurs.
- bloaf 3y ago
  It could also be illegal activity, e.g. insider trading.
  - trashtester 3y ago
    Absolutely it could.

more_corn 3y ago

There appears to be a missing therefor in the article. He seems to be hinting that the old adage “extraordinary claims require extraordinary evidence” isn’t as convincing as it sounds because strong (but really extraordinary) evidence is all around us. However, there is a decided lack of extraordinary evidence for extraordinary claims. That’s why they strike us as extraordinary. We’ve never seen them before. We’ve never seen anything like them. In fact all prior evidence has been strong evidence for ordinary claims.

bannedbybros 3y ago

[dead]

vannevar 3y ago

Essentially the author is saying that most people are honest most of the time. Maybe so, but truthfulness is only relevant in cases where there are stakes, and in those cases, there is much, much more dishonesty. Put another way, I'd say strong evidence is all around us when it doesn't matter, but considerably scarcer when we are actually interested in the truth.

pphysch 3y ago
Exactly, and this is an especially important point when taking about EMH. Markets are mostly noise/manipulation, whereas we assume that "statement of your given name" are mostly signal.

the_af 3y ago

Whenever I see an internet article (not written by a statistician and not a research paper) start with a reference to Bayes Theorem and its purported "perfection", I can safely bet it was written by a LessWrong'er and specifically a fan of Eliezer Yudkowsky. And behold! Just clicking on the Bayes link takes me to another article that immediately mentions Yudkowsky. I wonder what attracts internet people to Yudkowsky's brand of crackpottery, and specifically, what about Bayes elicits their quasi-religious awe.

embwbam 3y ago
I don’t identify with the religious zeal around bayes theorem, but can you elaborate on your criticisms of Yudkowsky? I’ve loved Harry Potter and the Methods of Rationality and read some of his essays, and Less Wrong’s coverage of cognitive bias helped me escape religion. I don’t agree with everything I’ve read of his, but I’m very curious at what point he becomes a crackpot.
- _Algernon_ 3y ago
  I gave up halfway through MoR, but to me it read as a self-insert nerd power fantasy. All non main-characters always end up agreeing with the all knowing MC even though there are obvious gaps in the reasoning.
  - PaulHoule 3y ago
    "Sequences" and other Yudkowksy writings are long and incoherent not by accident but by design. The point is to drive away anybody who has faculties of critical thinking. It's like those ads you see on the internet where they tell you that "TINNITUS IS DESTROYING YOUR BRAIN" or tell you what intermittent fasting plan is right for your shoe size and then send you to some two and a half long video that is there to weed out anybody who isn't incredibly credulous and after all that you find out it is some lame-ass supplement. Or it is like those poorly spelled letters that tell you the widow of some African dictator wants to to send you $15 million by Western Union. Someone who's gotten through all that really is someone vulnerable who can be wrapped around someone's finger. People who are good at critical thinking aren't looking for the secret to become good critical thinkers.

pavlov 3y ago

The article shows examples of ordinary claims backed by ordinary evidence, then jumps to this conclusion: “Extraordinary claims require extraordinary evidence, but extraordinary evidence might be more common than you think.” I saw no evidence of that here. “Strong” is a very different qualifier than “extraordinary.”

kens 3y ago

I don't get the point of this article; is it a response to something else? It seems kind of obvious that there is a lot of strong evidence for many things, but why does that matter? Also, I hate to quibble about details, but someone saying their name is "Mark Xu" isn't really strong evidence that their driver's license says "Mark Xu". There are a lot of people who go by names other than their official name. In my case, I tell everyone my name is Ken, but it's actually Kenneth. In the case of Xu, many people from China use a Western name like Mark that isn't their real name. Edit: I just realized that the odds that someone's name matches their drivers license is at most 6:1, since only 84% of the driving-age US population has a drivers license at all. The paragraph about the Efficient Market Hypothesis is just wrong. Contrary to the article, it is easy to make money on the stock market: historically it has gone up an average of 10% a year. It is also easy to beat the market average: you have roughly a 50-50 chance. How difficult is it to get into the top 1% of traders? Uh, that would be exactly 1 in 100. And I really don't see any connection between this paragraph and the topic of the post. The post seems to have something to do with calibration training, but that link takes me to a signup page for some mystery site, which seems suspicious. I'm trying to interpret this post generously and understand its message, but it seems kind of random.

the_af 3y ago
It's typical of LessWrong'er drivel. Lots of words to say something either trivial or wrong, or both. Bayes must be mentioned, because it's their religion.

hn_throwaway_99 3y ago

Yikes, not sure I've seen a front page HN that was this blatantly wrong. The second half of this article is complete nonsense because on one hand, he's seeming to arguing that extreme evidence is "easy to get", but it obviously can't be that easy when it's in a competitive environment where the number of people who could get that info is absolutely limited to something like only 1 percent of people. Taking this quote: > One implication of the Efficient Market Hypothesis (EMH) is that is it difficult to make money on the stock market. Generously, maybe only the top 1% of traders will be profitable. How difficult is it to get into the top 1% of traders? To be 50% sure you’re in the top 1%, you only need 200:1 evidence. This seemingly large odds ratio might be easy to get. As other commenters pointed out, that is absolutely not what the EMH says - it doesn't say it's difficult to be profitable (it actually says the exact opposite, especially when taken together with modern portfolio theory), it says it's difficult (or, rather, impossible) to outperform the market. But the more important point is that the author is implying that it's easy to get enough evidence to put you in the top 1%, as opposed to getting enough evidence to conclude that someone is in the top 1%, or to conclude that they're in any particular percentage actually. In other words, if you take his argument at face value, it may be possible to gather evidence to determine who is in the top 1% of traders, but that's very different from getting enough data to put yourself in that top 1%.

numbsafari 3y ago
If you are a grifter influencer, then having Bs “evidence” is all that is needed to “put” you into a presumed category, which lends you an air of credibility sufficient to take money from suckers. Another read is that it doesn’t take much to exploit other people.

throwaway9274 3y ago

There is a particular flavor of writing common to those whose identity is bound up with the so-called “effective altruist” community and their various online forums. It usually combines a reference to Bayes Theorem, a deep hubris, and a significantly misunderstood concept from mathematics or technology. Here, for example, it is clear the author does not grok EMH which applies across the population of trades not traders. Usually this is done to provide the author or community with some claim to an inherent and unearned extraordinary nature, which justifies greater personal power. (You only need 200:1 evidence you’re extraordinary being the takeaway here.) It is unclear why certain SV cliques continue to grant these people power. Their intellectual insularity means that whatever value they once added in bringing logical rigor to conversations is now far outweighed by the train of collective bad logic powering their groupthink. It is increasingly dangerous to have such people steer important societal debates. These debates include crypto regulation where EAs famously associated with SBF, to “AI safety”, a field that appears to be invented out of whole cloth to allow LessWrongers to LARP as ML ethics and fairness experts.

lusus_naturae 3y ago
Part of the problem is how easy it is to logically and rationally convince yourself or others of something inaccurate, imprecise or just plain wrong.
- kelseyfrog 3y ago
  There's a small, but important ego boost that comes along with thinking one has access to a clearer[1] view of reality. The LessWrong community gets this by way of "Bayesian thinking" and trying to subvert their own expectations. For the general population though, you have the Freakanomics effect. 1. The intellectual mind's biggest motivator is certainty. This isn't necessarily a bad thing when certainty aligns with objective reality whatever that means, but then these folks get quite certain about how uncertain they are.
d0mine 3y ago
Your comment had more weight if you'd dropped the ad-hominem and kept just the specific critique about EMH.

echohack5 3y ago

Strong evidence isn't common. There we go folks! Since I have now declared strong evidence to not be common you can believe on a 2,000,000:1 probability that in fact this is true!

bjourne 3y ago

Full article is better https://markxu.com/three-bayes-learnings https://markxu.com/three-bayes-learnings

ramblenode 3y ago

The author is asking the wrong question in the name example which is why the odds ratio is so huge. The correct question is what are the odds someone is telling the truth about their name given your expecation that they will tell you their true name when asked. The odds of the name being Mark Xu is irrelevant unless I have some shortlist of names I am comparing against, and then I would use the odds of the name being on the shortlist, not the global odds. It's obvious that the author's question is wrong because I would only be slightly surprised if I discovered that someone lied about their name, which is inconsistent with me having strong evidence for their name.