15 ms·

What’s Wrong with Bayes

131 points by luu 7y ago

kylebenzle 7y ago
A good post, but the TL:DR. What's wrong with Bayes? Nothing.
- neonate 7y ago
  That is not what the article says.
signalsmith 7y ago
For me, I really appreciate the Bayesian approach because it makes it very explicit that you pick a prior. Perhaps my experience is limited, but every (supposedly non-Bayesian) model I've used in practice has been possible to re-express using Bayesian terms, priors and beliefs and so on. Then I get to look at the intitial assumptions (model/prior) and use suitable human hand-wavey judgement about whether they make sense. Bayes is a good way to _update_ models, but if you lose sight of the fact that the bottom of your chain of deduction was a hand-wavey guess, you're in trouble.
- eanzenberg 7y ago
  Yeah, no thanks though. I don't want every rando adding "priors" that "feel" right to their analysis. Frequentist is straight forward. Both can (and are) abused to prove bias.
  - nightski 7y ago
    Frequentist has a prior also though. The uniform distribution. In a sense this might be more biased as it doesn't always accurately describe the situation. Either way I believe the effect of a prior diminishes greatly pretty quickly as you acquire more data. It's only a factor if you have extremely small set of data.
    - zozbot234 7y ago
      The uniform distribution ("flat") prior lets you interpret a maximum likelihood result as a maximum-a-posteriori (MAP) Bayesian point-estimate (implying a 0-or-1 loss function). One could argue that if you refrain from doing this and just stick to a literal application of the likelihood principle, you're not really depending on a flat prior. For that matter, what is a "flat" prior over the parameters also depends on what parameterization you're using. Results that are 'intuitive' under one parameterization may not be under a different one.
    - btilly 7y ago
      Frequentist has a prior also though. The uniform distribution. No. Experimental design affect frequentist conclusions in a way that is inconsistent with _ANY_ prior. Here is a real life example. My aunt and uncle had 7 children. 6 boys and one girl. Were they biased towards having one gender over another? If the null hypothesis is that they aren't, the p-value that you get is easily calculated as 16/2^7 = 1/8 = 0.125. (There is 1 arrangement of 7 girls, 7 of 6 girls and a boy, 7 of 6 boys and a girl, and 1 of 7 boys for 16 equally likely arrangements.) If I add the fact that they planned to have children until they had a boy and a girl, then that changes the p-value. In fact there are only 4 ways that their first 7 children can come out to give evidence this strong against the prior. So the p-value is now 4/2^7 = 1/32 = 0.03125. However a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion. It literally has nowhere to go in the formula and can't make a difference. Therefore the frequentist's differing conclusions are not consistent with ANY prior, implicit or not.
      - pdonis 7y ago
        > a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion A Bayesian would say no such thing. A Bayesian would agree that the knowledge that they planned to have children until they had both a boy and a girl doesn't affect your prior: you still are picking how much probability mass you allocate to all of the possible odds of having a boy vs. a girl, and the couple's plans don't affect that. However, a Bayesian would also say that the knowledge that they planned to have children until they had both a boy and a girl significantly changes the likelihood ratio (or p-value, if you prefer to use that) associated with the observed data. And one of the advantages of Bayesianism is that it forces you to make that explicit as well. Notice, for example, that when you calculated the first p-value of 1/8, you implicitly assumed that the couple's plan was "have 7 children, no matter what gender each of them is". The sample space is therefore all possible arrangements of 7 children by gender, and the p-value is 1/8, as you say. But when you calculated the second p-value of 1/32, while you did change the count of arrangements, you failed to recognize that the sample space changed! Now the possibilities are not just all possible arrangements of 7 children (which is what you used), but all possible arrangements of up to 7 children (because the "stop condition" now is not when there are 7 children total, but when there is at least one child of each gender, and that could have happened at a number of children less than 7). So the correct p-value is not 4/2^7, but 4/(2^7 + 2^6 + 2^5 + 2^4 + 2^3 + 2^2) = 4/(2^8 - 2) = 2/127. A Bayesian, who has to calculate the p-value starting from the hypothesis, not the data, would not make that mistake. And Bayesianism does something else too: it forces you to recognize that the p-value is not actually the answer to the question you were asking! By the p-value criterion, at least with the typical threshold of 0.05, the null hypothesis (that your aunt and uncle are not biased towards having one gender) is rejected. But a Bayesian recognizes that the prior probability of the gender ratio, based on abundant previous evidence, is strongly peaked around 50-50, much more strongly peaked than data with a bias equivalent to a p-value of 2/127 can overcome. So the Bayesian is quite ready to accept that your aunt and uncle had no actual bias towards having boys, they just happened to be one of the statistical outliers that are to be expected given the huge number of humans who have children.
  - jules 7y ago
    The difference between a frequentist and a Bayesian is that the latter admits that he picks a prior. A frequentist smushes together (1) the statistical assumptions (2) the approximations that make the problem computationally tractable and (3) the mathematical derivations, into one big mess. Just because you're not stating your assumptions doesn't mean there are none. Consider maximum likelihood estimation. It is not invariant under coordinate transformations. So which coordinates you pick is an assumption. In fact, with Bayesian estimation you can do the same thing: picking a prior is equivalent to picking the uniform prior in a different coordinate system. So frequentist estimation does involve picking a prior by picking a coordinate system, even if the frequentist does not admit this. Frequentist methods are conceptually anything but straightforward. The advantage of frequentist methods is that they are computationally tractable. Usually they are best understood as approximations to Bayesian methods. For instance, MLE can be viewed as the variational approximation to Bayes where the family of probability distributions is the family of point masses, and the prior is uniform.
    - Akababa 7y ago
      What do you mean by coordinate transformation? MLE is invariant under parameter transformations because it's just the argmax of the likelihood.
      - jules 7y ago
        Indeed, it is the argmax of the likelihood, but the likelihood is not invariant under coordinate transformations. The quantity p(x)dx is invariant, not p(x). By picking a suitable coordinate transformation you can put the MLE on any value where the likelihood is not zero.
        deleted 7y ago
        [deleted]
      - knzhou 7y ago
        Adding to the other comments, you still have prior-dependence on a more subtle level, because it depends on what hypotheses are allowed. Here's an extreme example. Consider flipping an apparently fair coin and getting "THHT". The hypothesis that the coin is fair gives this result with likelihood 1/16. The hypothesis that a worldwide government conspiracy has been formed with the sole purpose of ensuring this result... has a likelihood of 1. But nobody would ever declare this the MLE, because "government conspiracy" isn't one of the allowed options. But it isn't precisely because it's unlikely, i.e. because of your prior. Of course this is an extreme example, but there are more innocuous prior-based assumptions baked in too.
  - jefft255 7y ago
    See my above comment for an example of a prior that doesn't just "feel" right.
  - Turing_Machine 7y ago
    > I don't want every rando adding "priors" that "feel" right to their analysis. Conversely, maybe others see value in a technique that doesn't start from square one every single time. I mean, knowledge is a thing, yes?
    - perl4ever 7y ago
      That's not the point. The point is there is no choice between having priors and not. There is only the choice between acknowledging priors versus doublethink, confusion, and deception. With how fashionable it is to talk about implicit bias, I wonder how those concerns intersect with the people attacking Bayesian approaches here.
  - wirrbel 7y ago
    Let's say you perform a maximum-likelihood estimate, you still have an assumption baked in, that maximizing the likelihood given the data is the right way to make your estimate. In fact, it's very interesting to reconstruct a Bayesian prior for a maximum likelihood estimate. For example when you calculate probabilities for a binary event, 10 head flips, 8 tail flips. The ML estimate is 8/18 = 4/9. A Beta-Binomial bayesian model leads to a posterior distribution of Beta(a=8, b=10) with a mean of (8+a0)/(10 + 8 + a0 + b0), with a0 and b0 representing the prior distribution Beta(a0, b0). Now you can see that the maximum likelihood estimate is identical in this case to assuming a bayesian Prior of Beta(0, 0). I am not saying by this that frequentism is Bayesian inference in disguise, rather, you cannot escape the assumptions. Also, frequentism is not that straightforward, there are many kinds of frequentist estimators and it can be complicated to choose among them.
    - zozbot234 7y ago
      The ML estimate is a posterior mode, assuming a flat prior. It's not immediately clear that it will always be possible to find a corresponding posterior mean. (From a Bayesian point of view, this is a difference in loss functions as opposed to priors over the parameters. With a posterior mean, you're making the optimal inference assuming a quadratic loss; a posterior mode is appropriate for a 0-or-1 loss.)
  - FeepingCreature 7y ago
    That's why your analysis shouldn't report posterior probability but degree of shift in confidence.
- madhadron 7y ago
  > it makes it very explicit that you pick a prior But you don't, in general, pick a prior. You pick a procedure that has an expected loss under various conditions. It's one player game theory. If you happen to have a prior, then you can use it to choose a unique procedure that has minimal expected risk for that prior given the loss function, but even so that may not be what you want. For example, you may want a minimax procedure, which may be quite different from the Bayes procedure.
  - jbay808 7y ago
    Minimax still requires a probability distribution, which means you need a prior. Edit: Based on the downvotes, I see my audience is not convinced. I'll repeat an explanation I posted a while ago. Probably should make this a blog post because I see this claim quite often. I'd love to know what book you read it in. -- In minimax regret, you have a set of available decisions D, and a set of possible states of nature N, and a utility U(D,N). Each state of nature also has a probability P(N) (which can be influenced by the decision too in some problems). States of nature include "interest rates rise 1%", "interest rates fall 1%", and "interest rates stay the same". Decisions include "invest in stocks" and "invest in bonds". Minimax regret proposes to ignore the probabilities P(N), instead suggesting a way to make a decision purely based on the utilities of the outcomes. But that is actually an illusion. Outside of math class word problems, we don't have N or U(D,N) handed to us on a silver platter. There is always an infinite range of possible states of nature, many of which have a probability approaching but never reaching zero, including states such as "win the lottery", "communist revolution", and "unexpected intergalactic nuclear war". In commonsense decision-making we don't include those states of nature in our decision matrix, because our common sense rules them out as being implausible before we even think about our options. You wouldn't choose to invest in bonds just because stocks have the most regret in the event of a communist takeover. So what actually happens is we intuitively apply some probability threshold that rules out states of nature falling below it from our consideration. Then we minimize max regret on the remaining "plausibly realistic" states of nature. Humans are so good at doing probability mentally that this step happens before we even realize it. But if you are writing code that makes decisions, you'll need to do it, and so you'll need to have at least a rough stab at the probability distributions.
- mikorym 7y ago
  Are all priors an application of Bayes's theorem? It is confusing to me that there is talk of Bayesian statistics vs. frequentist statistics when both are often used in conjunction. The classic example of a medical test with false positives and false negatives and the prior being incidence in the general population comes to mind. To me that is not just an example of Bayes, but a combination of frequentist statistics with Bayes's theorem. I also seem to recall that Bayes's theorem appears in a standard first year probability and statistics course.
  - eli_gottlieb 7y ago
    Bayesian statistics: the parameters you want to infer are modeled as random variables with a non-empirical prior, and Bayes' rule is used to draw inferences. Frequentist statistics: you construct estimators for the parameters you care about, subject to appropriate loss/risk criteria, but without any explicit "prior knowledge". Frequentist statistics with Bayes' theorem: you use available empirical data, plus some exponential-family distribution, to construct an informed prior, then use Bayes' rule to update the prior on evidence. You use this Bayesian approach only for unobservable hypotheses, rather than for parameters which can be estimated. Machine learning: you stack lots and lots of polynomial regressors onto each-other and train them with a loss function until they predict well on the validation set.
    - jbay808 7y ago
      What do you mean by a "non-empirical prior"?
      - chongli 7y ago
        Bayesian statistics is sometimes called subjectivist statistics. Probability in Bayesian statistics reflects your degree of belief in some potential outcome. If you conduct an experiment, you use Bayes’ theorem to update your degree of belief, which is now conditional on the outcome of your experiment. By quantifying your degree of belief in a prior, you give yourself some starting point (rather than just assuming 0 probability), even if that prior is only an educated guess and not some well researched position. This can be good because you might not have done the research yet.
jefft255 7y ago
In robotics, particularly in bayesian filtering (KFs and so on), I find the idea of a "prior" solid and I don't see any frequentist alternatives. Your prior is easy to understand: whatever your posterior for your state was at the previous timestep, updated using the actions you wanted your robot to accomplish. Inference is then refining this prior using the observation that the robot makes. There's nothing hand-wavy about that; if you do bayesian statistics with bad priors of course you're going to get bad inference. I guess the author just warns about being careful about your assumption which is always good.
- skybrian 7y ago
  I'm curious what happens when you reboot your robot. What's the first prior?
  - bonoboTP 7y ago
    You initialize it to something. Like if you have a robot for running around in a specific building or warehouse, you can initialize the location distribution to be uniform within the warehouse. After a few high-quality observations, the effect of the exact choice of initialization should be greatly reduced anyway.
  - jtolmar 7y ago
    For car GPS, which uses the same math, I used the first GPS signal the device receives. This ends up being equivalent to the uniform prior.
  - dTal 7y ago
    What do you do when you wake up? Assume you're in the same place as when you went to sleep. You won't be surprised to find yourself on the other side of the bed - slightly more surprised to find yourself on the floor, and very surprised to find yourself in another country. A large belief update is always a bit of a shock.
Akababa 7y ago
I don't know, this seems to be a really low-effort blog post. The given example is obviously contrived from the unreasonable improper (-\infty,\infty) prior and the low \sigma^2=1 likelihood. If it was really "pure noise" then you'd have \sigma^2=\infty which rightly gives you a flat posterior. For sure Bayesian gives you more flexibility with your assumptions, so it's easier to shoot yourself in the foot. But when used correctly it can be more powerful, and often easier to interpret.
- contravariant 7y ago
  Ironically the article that the example is from offers quite a nice rebuttal: > None of these examples are meant to shoot down Bayes. Indeed, if posterior inferences don’t make sense, that’s another way of saying that we have external (prior) information that was not included in the model. (“Doesn’t make sense” implies some source of knowledge about which claims make sense and which don’t.) When things don’t make sense, it’s time to improve the model. Bayes is cool with that.
olooney 7y ago
Just for context, Andrew Gelman is one of the creators of Stan[1], one of the most popular probabilistic programming platforms for Bayesian interference. He has written a popular textbook on Bayesian methods, Bayesian Data Analysis[2]. Everyone hates picking priors in Bayesian analysis. If you pick an informative prior, you can always be criticized for it (in peer review, for a business decision, etc.) The usual dodge is to use a non-informative prior (like the Jeffreys prior[3].) I interpret Gelman's point as saying this can also lead to bad decisions. Thus, Bayesian analysts must thread the needle between Scyllia and Charybdis when picking priors. That's certainly a real pain point when using Bayesian methods. However, it's pretty much the same pain point as choosing regularization parameters (or choosing not to use regularization) when doing frequentist statistics. For example, sklearn was recently criticized for turing on L2 regularization by default which could be viewed as a violation of the principle of least surprise, as well as causing practical problems when inputs are not standardized. But leaving regularization turned off is equivalent to choosing an non-informative or even improper prior. (informally in many cases, and formally identical for linear regression with normally distributed errors[4].) So Scyllia and Charybdis still loom on either side. My problem with Bayesian models, completely unrelated to Gelman's criticism, is that the partition function is usually intractable and really only amenable to probabilistic methods (MCMC with NUTS[5], for example.) This makes them computationally expensive to fit, and this in turn makes them suitable for (relatively) small data sets. But using a lot more data is the single best way to allow a model to get more accurate while avoiding over-fitting! That is why I live with the following contradiction: 1) I believe Bayesian models have better theoretical foundations, and 2) I almost always use non-Bayesian methods for practical problems. [1]: https://mc-stan.org/ https://mc-stan.org/ [2]: https://www.amazon.com/Bayesian-Analysis-Chapman-Statistical-Science/dp/1439840954 https://www.amazon.com/Bayesian-Analysis-Chapman-Statistical... [3]: https://en.wikipedia.org/wiki/Jeffreys_prior https://en.wikipedia.org/wiki/Jeffreys_prior [4]: https://stats.stackexchange.com/questions/163388/l2-regularization-is-equivalent-to-gaussian-prior https://stats.stackexchange.com/questions/163388/l2-regulari... [5]: http://www.stat.columbia.edu/~gelman/research/published/nuts.pdf http://www.stat.columbia.edu/~gelman/research/published/nuts...
- perl4ever 7y ago
  "Everyone hates picking priors in Bayesian analysis." Everybody hates searching for their keys in the dark.
knzhou 7y ago
But this isn't actually a criticism of Bayes at all. Yes, the result depends on your prior. But the result always depends on your preconceptions -- even in frequentist statistics, where it determines which statistical tests you use and which hypotheses you test and what p-value cutoff is reasonable. It's better to have this up front. Or, you can publish Bayesian update factors, which are prior-independent.
sunstone 7y ago
The human brain is the best Bayesian model builder that evolution has yet devised. A good place to start assessing its weaknesses is to observe your own brain messing up. This shouldn't be hard to do.
- madhadron 7y ago
  Why do you think that the human brain is Bayesian?
  - c2471 7y ago
    I bring out a coin; I tell you nothing, and ask you to guess what the probability of heads is. What do you guess? Unless you have reason to believe I am trying to deceive you, it will be able 50% because you have a lot of knowledge from other contexts that tells you this is true. The arrow is probably the other way round than you state -the brain probably isn't Bayesian; being Bayesian is modelled on how humans process and contextualise decisions. I'm not even sure how a frequentist would construct a model to estimate an outcome with no observations.
    - madhadron 7y ago
      > I'm not even sure how a frequentist would construct a model to estimate an outcome with no observations. The same way a Bayesian would, since it's a question about probabilities of hypothetical experiments, not about statistics. Or you go through decision theory instead of mucking about with half baked ideologies. > being Bayesian is modelled on how humans process and contextualise decisions. This is false.
Majromax 7y ago
> Example abridged: a draw from N(phi,1) for unknown phi is 1. Bayesian reasoning with a uniform prior gives an 86% posterior probability that phi > 0 I'm not sure I see the problem here? If it's counterintuitive, it's only because we treat N(0,1) as the normal distribution, so our true prior is that if we pick a distribution out of a hat we're more likely to have N(0,1) than anything else. Suppose I truly know nothing but what is given in the quote. On the basis of symmetry, I'd have to conclude that P(phi<0) is the same as P(phi>2). If the blogger had phrased this as "86% posterior probability that phi < 2", I don't think it would be so surprising. In fact, the blogger describes this draw as: > after seeing an observation that is statistically indistinguishable from noise. which to me presupposes a great deal of information about what 'noise' is supposed to look like.
j7ake 7y ago
The example should of course ring caution bells but at least in Bayes you can figure out why your inference is doing unreasonable things by examining each of your assumptions. In this case it’s the prior that needs fixing. Are there alternative methods that are better than the Bayes method for this toy example?
- TTPrograms 7y ago
  Seriously, as soon as he said "flat prior on theta" I had huge alarm bells go off. Garbage in garbage out.
roenxi 7y ago
There is a certain intellectual laziness in this perspective as might be expected from a short blog post - obviously Bayes' formula is theoretically sound because it is trivial to deduce and prove. So we know that if the conclusion is not acceptable then either the method, the prior or the evidence is not acceptable. Evidence and method can be ruled out; so the prior was not reasonable. Basically, he's saying that he doesn't believe the prior is flat. A reasonable thing to say too - as he says practically speaking if we suspect the distribution is probably random noise then the prior is we are probably looking at noise. So in practice the prior is heavily weighted towards 0. It isn't intellectually honest to use an uniformed prior unless you think the probability of a process being statistical noise is almost 0.
- 6gvONxR4sf7o 7y ago
  >obviously Bayes' formula is theoretically sound because it is trivial to deduce and prove. Quantum mechanics doesn't follow the usual probability rules, so you can't really say "obviously Bayes' formula is theoretically sound." It certainly seems like Bayes theorem should apply universally but apparently it doesn't. Or at least, the jury's still out. https://en.wikipedia.org/wiki/Quantum_probability https://en.wikipedia.org/wiki/Quantum_probability
howlin 7y ago
Bayesian modeling can be very powerful when it works but it can also be catastrophic when it fails. It helps to think about this in an adversarial decision theoretic context where you play a prediction game against an opponent (usually called Nature). We can think of the game as discovering the best model to explain a set of observations. The Bayesian believes that Nature picks the true model that generated the observations by sampling the prior. This is actually a huge assumption to make, which is why Bayesian methods work so well when the assumption is close to the truth. Frequentists make the assumption that Nature chooses the underlying true model from a set of possible models. Beyond restricting the set of models Nature can choose from, frequentists make no further assumptions about the selection process. This is a strictly weaker assumption than the Bayesian makes, which means frequentist methods will do better when the specified prior grossly misrepresents Nature's decision making process. There are even weaker assumptions that can be made about how Nature chooses the data. Regret-based model inference allows for a more adversarial game with Nature where the data may not come from the class of models considered at all. If Nature truly behaves this way, then Bayesian decision making can catastrophically fail.
- c2471 7y ago
  This ignores the main strength of a Bayesian workflow. You can straight forwardly quantify the effect of your prior choice on your inference - pick a different prior; how much does that change the inference, etc etc. A good Bayesian workflow does not assume a prior to be true; it should be based on available evidence, and then stressed. To be a bit more concrete, let's say we wish to model the height of kangaroos. We come up with a model form, say regression, and a bunch of potential features. If we are Bayesian we might say; "I think nature prefers simple stable solutions, so I'll put a N(0,d) prior on my weights. We then compute a posterior and get a range of credible values. We can then say, "hey, what if I'm wrong and actually it's a student t, or it's flat prior or X or y or z", and use principled tools like marginal likelihood to say which family of models works best, do prior posterior comparisons to see how observations changed our prior etc etc. If we do this under a frequentist framework we compute the regression coefficients, and can get some confidence bounds with some appeal to asymptotics (and nobody I've ever seen actually makes any attempt to validate these assumptions). And even when we are done, we get a confidence interval that has such a truly unintuitive definition that almost every person who is not a stats PhD fundamentally misinterprets. To say frequentists make less assumptions is not true- they are just less explicit, and I consider it a strength not a weakness to highlight choices made by the statistician.
selectionbias 7y ago
My problem with the 'Bayes=rationality' type of argument is that it ignores context and isn't really a case for reporting Bayesian vs frequentist estimates. If I am a researcher publishing results then I have an audience who interpret my results. If my audience is Bayesian and accept my model then all I need to do is report sufficient statistics and they can make their own Bayesian inferences given their priors, or better yet, I can just post my whole dataset. The very reason we need to report things like credible sets or confidence intervals rather than just sufficient statistics is because audiences in the real world want summary stats that they can easily interpret and are transparent. The best approach to inference is one that is the most useful to audiences, and that depends on context and practicalities rather than on some underlying philosophy of subjective vs objective probabilities.
abeppu 7y ago
I feel this post should be considered along with its sibling: https://statmodeling.stat.columbia.edu/2019/12/04/whats-wrong-with-null-hypothesis-significance-testing/ https://statmodeling.stat.columbia.edu/2019/12/04/whats-wron... I think reading either alone is prone to lead readers to a false understanding of Gelman's perspective.
metasj 7y ago
Many analyses of the world aren't bayesian /or/ frequentist, they use much simpler pattern-matching, with feedback loops that update the approach used as well as the conclusion. Problems start w/ assuming you have to choose one of those approaches to estimate the future...
bonoboTP 7y ago
"Bayesian" is an overloaded term. There's Bayes' theorem/rule, which basically everyone agrees with, since it's a theorem that's very simple to prove with a few high school math operations. Then there is the philosophical Bayesian interpretation of probability, that claims that probabilities are fundamentally about our own mental state of belief, as opposed to frequencies at the limit of infinite repetition of some experiment. Then there is the Bayesian methods of statistics / machine learning etc, which are about handling parameters as random variables and the observed data as fixed, as opposed to assuming that there's one fixed parameter (without a distribution to talk about) and the data should be modeled as random (from an oversimplified bird's eye view). And it was also oversold as a miracle cure for all our problems: for some time, before the deep learning era, you just had to have "Bayesian" in your ML paper title to make it sexy and interesting. Then there is the online Bayesian rationalist community, where Bayes is used to explain the meaning of life, the universe, it's the great grand explanation of everything, a self help tool, the key to seeing the light, a semi-religious experience, the way to enlightenment (they even call it the Way, capitalized - I guess a Buddhist reference?). As if being Bayesian was this secret club, that sets you apart from average people, a symbol of belonging to the in-group etc. [1] It's important to keep these apart. [1] For example: https://youtu.be/NEqHML98RgU?t=73 https://youtu.be/NEqHML98RgU?t=73 (it's explicitly not about the math but about self-help and intuition to benefit our lives etc...)
Gusen 7y ago
Nadler, Schiff, & Mueller each raped and killed boys in Buffalo on the night of January 14 2019, as Trump did earlier that morning. Impeachment is a vehicle for keeping power, they are all working together. Full audio proving this entirely here, all three admit collusion w/President. PDF updated. \\14 January 2019 11:23pm: Jerrold Nadler steps up to take his turn during the Illuminati "rape party". Nadler rapes and kills three boys in under a minute, however, there was a problem. One of the boys was already dead, so he requests a new one. Jerrold Nadler then requests another ten boys. Donald Reeves: "That's a million dollar request". Nadler responds: "...you guys will cover it. I'm gonna keep Trump in power" (Trump raped and killed a dozen boys 6:30-7:00am that morning). By the time Nadler is finished, he had raped and killed 24 boys. Audio pulled from the video linked below: 14JanCh4_2300-0000.mp3 - Nadler starts at about 20:00 in. https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU66GuU/view?usp=sharing https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU... 14JanCh2_2304-2359.mp3 https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9WLr1x1/view?usp=sharing https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9W... 14JanCh3_2302-2359.mp3 https://drive.google.com/file/d/1wdDIsxfsX7xTBIBZYzV4iE4xEdgmwy5l/view?usp=sharing https://drive.google.com/file/d/1wdDIsxfsX7xTBIBZYzV4iE4xEdg... Further disclosure by Porter about 24:47 in: Oblivious tenant Brian Schlenker comments on something unrelated to the ongoing events: "..call the fucking police...", to which is responded with: "...that's funny because we own the police (Buffalo Police Department), we pay them $6 million dollars a month." 19 minutes in Fred Norris, formerly of the Howard Stern show, is acknowledged on the system. At 19:47 in Porter admits that Brian Schlenker will be the owner of this footage should it be discovered. January 15, 2019 00:20: Special Counsel Robert Mueller takes his turn at the Illuminati "rape party" in Buffalo, New York. Mueller rapes and kills twelve boys. About roughly 00:55 Representative Adam Schiff who will also be leading an impeachment effort, also requests the same deal as Nadler, and then tries to make a case for getting more than Nadler and Mueller. Adam Schiff rapes and kills three boys. Mueller and Schiff all receive $3 billion dollars each for joing the group. Nadler came back to witness these two rape to make sure they were all bound together under one purpose: keep Trump in power, and also to confirm the payments to each, including his $10.5 billion dollar payment. Between Mueller and Schiff turns, the group issues orders for ten women to begin prepping more boys for rape. They are former friends and family of Brian Schlenker, and also some long standing Illuminati members who include Elsa Hosk, Gigi and Bella Hadid. Again, the "prep" these females engage means they perform oral sex on the boys’ penis and anus, as a child rapist like Henry Porter would, while trying to remove fecal matter from the boy prior to handing them over to be raped and subsequently murdered. Just a head's up, my voice is scattered throughout all of the footage within the links posted for this update, and is quite loud relative to the desired content at times. Audio links below: 15JanCh4_000-100.mp3 https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2OQnwZ/view?usp=sharing https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2... 15JanCh4_100-200.mp3 https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw746LeCL/view?usp=sharing https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw74... 15JanCh2_000-100.mp3 https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62tgCkj/view?usp=sharing https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62... 15JanCh2_100-200mp3 https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_rGSFH/view?usp=sharing https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_... 15JanCh3_000-100.mp3 https://drive.google.com/file/d/1zsEwYC875wQu6btsSbNznHsJinxYHxUL/view?usp=sharing https://drive.google.com/file/d/1zsEwYC875wQu6btsSbNznHsJinx... 15JanCh3_100-200.mp3 https://drive.google.com/file/d/15UAY2er2LdTViXG4azy_bc_oQUzvLpFh/view?usp=sharing https://drive.google.com/file/d/15UAY2er2LdTViXG4azy_bc_oQUz... //Full 110 page PDF [updated 2Dec2019]: FBI_FinalDraft_26Jul2019_BSchlenker.pdf https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJc9cGk/view?usp=sharing https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJ... //This post will be censored when this account logs off, the posts are "shadow banned". They try to make it look like the post is live, but it is not. Here is an example. https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJedYKz/view?usp=sharing https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJ... \\Previously: President of the United States Donald J. Trump rapes and kills 15 boys in Buffalo, NY on January 10, 2019. This is audio of the event from 10-Jan-2019. Download the mp3 and put on headphones, and turn the volume all the way up. 10JanCh3_1255-1557.mp3 https://drive.google.com/file/d/18lTt_YKFEtsV6YDNzXFVQYLkStY-kSg2/view?usp=sharing https://drive.google.com/file/d/18lTt_YKFEtsV6YDNzXFVQYLkStY... This is audio of the President, Donald J. Trump, demanding a $4 billion dollar bribe from child rapists to “take a blind eye” on January 3, 2019. Trump becomes one on January 14, 2019. Also, here is the big reason the major networks do not report any of it. //Download the video, turn the volume all the way up and put head phones on. Note: there is not much to see in the video, the audio is picked up from another [illegal surveillance] system. Trump is on a call from with Henry Porter and Gigi Hadid. See page 63. Bribe demand at 10:18am: 3JanCh3_900-1100.avi https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77YG0Vr/view?usp=sharing https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77... //President of the United States, Donald J. Trump, rapes and kills his first boy at 6:32am. Video link below: 14JanCh3_600.avi https://drive.google.com/file/d/154QvA5hwyHGYIVXtod1ZbsOHFUJNbqZW/view?usp=sharing https://drive.google.com/file/d/154QvA5hwyHGYIVXtod1ZbsOHFUJ... 14JanCh2_600-700.avi https://drive.google.com/file/d/19UkqmnMwZiWy7xxWngltqwoKLTJL-IZ_/view?usp=sharing https://drive.google.com/file/d/19UkqmnMwZiWy7xxWngltqwoKLTJ... //On January 18, 2019 at 8:31am (see page 8) Trump acknowledges the four billion dollar bribe and says: "Let's get it done and get to fucking some kids." Video link below: 18JanCh3_725-.avi https://drive.google.com/file/d/1bVTcGq5Z9oOSAiOQcKYrmuK4TwoNmY1h/view?usp=sharing https://drive.google.com/file/d/1bVTcGq5Z9oOSAiOQcKYrmuK4Two... //A big reason this has not been reported by the major news networks is right here. Lester Holt of NBC Nightly News, apparently a member of the Illuminati since the 1980's, along with ABC Nightly News lead anchor David Muir, stop over to the Porter studio in Buffalo on January 14, 2019 at 5:00am. They both rape and kill about two dozen boys by 6:00am. Muir starts around 5:15am, then Holt about 5:38am. Multi-billionaire Rupert Murdoch, owner of News Corp & Fox Corporation, takes his turn after Holt. Video links below: 14JanCh3_500-601.avi https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX3o7UL/view?usp=sharing https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX... 14JanCh2_530-600.avi https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1RwidV/view?usp=sharing https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1... Adding to the reason this is not picked up by the media, CBS & Viacom owner Sumner Redstone and Leslie Moonves rape boys following the President. 14JanCh3_700.avi https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXaB4aI0/view?usp=sharing https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXa... 14JanCh2_700-800.avi https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWYKfRDp/view?usp=sharing https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWY... //This is the tip of the ice berg. //Full 110 page PDF [updated 2Dec2019]: FBI_FinalDraft_26Jul2019_BSchlenker.pdf https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJc9cGk/view?usp=sharing https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJ... //This post will be censored when this account logs off, the posts are "shadow banned". They try to make it look like the post is live, but it is not. Here is an example. https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJedYKz/view?usp=sharing https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJ...
ummonk 7y ago
>Put a flat prior on theta and you end up with an 84% posterior probability that theta is greater than 0. Step back a bit, and it’s saying that you’ll offer 5-to-1 odds that theta>0 after seeing an observation that is statistically indistinguishable from noise. That can’t make sense. Go around offering 5:1 bets based on pure noise and you’ll go bankrupt real fast. If you think it's likely to be pure noise, why the hell would you put a flat prior on it? Note also that nonflat priors are implicit in significance testing - e.g. p95 significance is similar to putting a 95% prior on the null hypothesis, and p99 significance is similar to putting a 99% prior on the null hypothesis.
gweinberg 7y ago
The author has a major fundamental misconception as to how probability works. If I say "the probability that proposition X is true is 0.5", that means that based on the information available to me right now it's equally likely likely to be true as false. That's not even remotely similar to saying I would offer an even money bet.
- baron_harkonnen 7y ago
  Ignoring the fact that “the author” is one of the most respected statistians in the world today... there is no debate on how to translate probabilities into odds: odds(x) = p(x)/(1-p(x)) Thats the definition of “odds”. so in this case it is quite clear that the odds for X is 1, implying and even money bet.
syrrim 7y ago
If the goal is to avoid bankruptcy, then the probability needs to be interpreted differently. If you bet the house every time, you're guaranteed to go bankrupt eventually. Suppose instead you bet half your money on an event of 50% probability. If you take 1:1 odds on this, then when you lose, your money is divided by 2, but when you win it is only multiplied by 1.5. Your money will tend to decrease over time. You need to pick odds 1:a such that 1+a/2=2 => a=2. We recover our regular betting odds by betting a smaller portion of our money. If we bet a portion 1/d of our money on an event of probability 1/p, we needs odds 1:a such that 1+a/d=(d/(d-1))^(p-1). For large enough d we get a=p-1, as we would expect. Assume again you're betting half your money each round, but take a probability of winning of 84%, as in the article. You should take that bet at 1:1.14 odds, much less than the recommended 1:5 odds.
- ravar 7y ago
  For the curious, look up the kelly criterion, it formalizes this thinking.
- ikeboy 7y ago
  This has nothing to do with interpreting probability, but with a utility function that's not linear in terms of wealth. With decreasing marginal returns to wealth, the same bet becomes less attractive at lower wealth levels. Although this can't fully explain risk aversion, see https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.15.1.219 https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.15.1.219
mycall 7y ago
84% isn't that great for predictions compared to DNNs, RNNs or other modern ML algorithms.
pontusrehula 7y ago
To criticize is easy but it feels incomplete if one doesn't provide any clues of what the supposedly better alternatives would be.

131 points by luu 7y ago

kylebenzle 7y ago

A good post, but the TL:DR. What's wrong with Bayes? Nothing.

neonate 7y ago
That is not what the article says.

signalsmith 7y ago

For me, I really appreciate the Bayesian approach because it makes it very explicit that you pick a prior. Perhaps my experience is limited, but every (supposedly non-Bayesian) model I've used in practice has been possible to re-express using Bayesian terms, priors and beliefs and so on. Then I get to look at the intitial assumptions (model/prior) and use suitable human hand-wavey judgement about whether they make sense. Bayes is a good way to _update_ models, but if you lose sight of the fact that the bottom of your chain of deduction was a hand-wavey guess, you're in trouble.

eanzenberg 7y ago
Yeah, no thanks though. I don't want every rando adding "priors" that "feel" right to their analysis. Frequentist is straight forward. Both can (and are) abused to prove bias.
- nightski 7y ago
  Frequentist has a prior also though. The uniform distribution. In a sense this might be more biased as it doesn't always accurately describe the situation. Either way I believe the effect of a prior diminishes greatly pretty quickly as you acquire more data. It's only a factor if you have extremely small set of data.
  - zozbot234 7y ago
    The uniform distribution ("flat") prior lets you interpret a maximum likelihood result as a maximum-a-posteriori (MAP) Bayesian point-estimate (implying a 0-or-1 loss function). One could argue that if you refrain from doing this and just stick to a literal application of the likelihood principle, you're not really depending on a flat prior. For that matter, what is a "flat" prior over the parameters also depends on what parameterization you're using. Results that are 'intuitive' under one parameterization may not be under a different one.
  - btilly 7y ago
    Frequentist has a prior also though. The uniform distribution. No. Experimental design affect frequentist conclusions in a way that is inconsistent with _ANY_ prior. Here is a real life example. My aunt and uncle had 7 children. 6 boys and one girl. Were they biased towards having one gender over another? If the null hypothesis is that they aren't, the p-value that you get is easily calculated as 16/2^7 = 1/8 = 0.125. (There is 1 arrangement of 7 girls, 7 of 6 girls and a boy, 7 of 6 boys and a girl, and 1 of 7 boys for 16 equally likely arrangements.) If I add the fact that they planned to have children until they had a boy and a girl, then that changes the p-value. In fact there are only 4 ways that their first 7 children can come out to give evidence this strong against the prior. So the p-value is now 4/2^7 = 1/32 = 0.03125. However a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion. It literally has nowhere to go in the formula and can't make a difference. Therefore the frequentist's differing conclusions are not consistent with ANY prior, implicit or not.
    - pdonis 7y ago
      > a Bayesian looks at this and says that no matter what prior you pick, the knowledge that they planned to have children until they had both a boy and a girl does not affect your posterior conclusion A Bayesian would say no such thing. A Bayesian would agree that the knowledge that they planned to have children until they had both a boy and a girl doesn't affect your prior: you still are picking how much probability mass you allocate to all of the possible odds of having a boy vs. a girl, and the couple's plans don't affect that. However, a Bayesian would also say that the knowledge that they planned to have children until they had both a boy and a girl significantly changes the likelihood ratio (or p-value, if you prefer to use that) associated with the observed data. And one of the advantages of Bayesianism is that it forces you to make that explicit as well. Notice, for example, that when you calculated the first p-value of 1/8, you implicitly assumed that the couple's plan was "have 7 children, no matter what gender each of them is". The sample space is therefore all possible arrangements of 7 children by gender, and the p-value is 1/8, as you say. But when you calculated the second p-value of 1/32, while you did change the count of arrangements, you failed to recognize that the sample space changed! Now the possibilities are not just all possible arrangements of 7 children (which is what you used), but all possible arrangements of up to 7 children (because the "stop condition" now is not when there are 7 children total, but when there is at least one child of each gender, and that could have happened at a number of children less than 7). So the correct p-value is not 4/2^7, but 4/(2^7 + 2^6 + 2^5 + 2^4 + 2^3 + 2^2) = 4/(2^8 - 2) = 2/127. A Bayesian, who has to calculate the p-value starting from the hypothesis, not the data, would not make that mistake. And Bayesianism does something else too: it forces you to recognize that the p-value is not actually the answer to the question you were asking! By the p-value criterion, at least with the typical threshold of 0.05, the null hypothesis (that your aunt and uncle are not biased towards having one gender) is rejected. But a Bayesian recognizes that the prior probability of the gender ratio, based on abundant previous evidence, is strongly peaked around 50-50, much more strongly peaked than data with a bias equivalent to a p-value of 2/127 can overcome. So the Bayesian is quite ready to accept that your aunt and uncle had no actual bias towards having boys, they just happened to be one of the statistical outliers that are to be expected given the huge number of humans who have children.
- jules 7y ago
  The difference between a frequentist and a Bayesian is that the latter admits that he picks a prior. A frequentist smushes together (1) the statistical assumptions (2) the approximations that make the problem computationally tractable and (3) the mathematical derivations, into one big mess. Just because you're not stating your assumptions doesn't mean there are none. Consider maximum likelihood estimation. It is not invariant under coordinate transformations. So which coordinates you pick is an assumption. In fact, with Bayesian estimation you can do the same thing: picking a prior is equivalent to picking the uniform prior in a different coordinate system. So frequentist estimation does involve picking a prior by picking a coordinate system, even if the frequentist does not admit this. Frequentist methods are conceptually anything but straightforward. The advantage of frequentist methods is that they are computationally tractable. Usually they are best understood as approximations to Bayesian methods. For instance, MLE can be viewed as the variational approximation to Bayes where the family of probability distributions is the family of point masses, and the prior is uniform.
  - Akababa 7y ago
    What do you mean by coordinate transformation? MLE is invariant under parameter transformations because it's just the argmax of the likelihood.
    - jules 7y ago
      Indeed, it is the argmax of the likelihood, but the likelihood is not invariant under coordinate transformations. The quantity p(x)dx is invariant, not p(x). By picking a suitable coordinate transformation you can put the MLE on any value where the likelihood is not zero.
      - deleted 7y ago
        [deleted]
    - knzhou 7y ago
      Adding to the other comments, you still have prior-dependence on a more subtle level, because it depends on what hypotheses are allowed. Here's an extreme example. Consider flipping an apparently fair coin and getting "THHT". The hypothesis that the coin is fair gives this result with likelihood 1/16. The hypothesis that a worldwide government conspiracy has been formed with the sole purpose of ensuring this result... has a likelihood of 1. But nobody would ever declare this the MLE, because "government conspiracy" isn't one of the allowed options. But it isn't precisely because it's unlikely, i.e. because of your prior. Of course this is an extreme example, but there are more innocuous prior-based assumptions baked in too.
- jefft255 7y ago
  See my above comment for an example of a prior that doesn't just "feel" right.
- Turing_Machine 7y ago
  > I don't want every rando adding "priors" that "feel" right to their analysis. Conversely, maybe others see value in a technique that doesn't start from square one every single time. I mean, knowledge is a thing, yes?
  - perl4ever 7y ago
    That's not the point. The point is there is no choice between having priors and not. There is only the choice between acknowledging priors versus doublethink, confusion, and deception. With how fashionable it is to talk about implicit bias, I wonder how those concerns intersect with the people attacking Bayesian approaches here.
- wirrbel 7y ago
  Let's say you perform a maximum-likelihood estimate, you still have an assumption baked in, that maximizing the likelihood given the data is the right way to make your estimate. In fact, it's very interesting to reconstruct a Bayesian prior for a maximum likelihood estimate. For example when you calculate probabilities for a binary event, 10 head flips, 8 tail flips. The ML estimate is 8/18 = 4/9. A Beta-Binomial bayesian model leads to a posterior distribution of Beta(a=8, b=10) with a mean of (8+a0)/(10 + 8 + a0 + b0), with a0 and b0 representing the prior distribution Beta(a0, b0). Now you can see that the maximum likelihood estimate is identical in this case to assuming a bayesian Prior of Beta(0, 0). I am not saying by this that frequentism is Bayesian inference in disguise, rather, you cannot escape the assumptions. Also, frequentism is not that straightforward, there are many kinds of frequentist estimators and it can be complicated to choose among them.
  - zozbot234 7y ago
    The ML estimate is a posterior mode, assuming a flat prior. It's not immediately clear that it will always be possible to find a corresponding posterior mean. (From a Bayesian point of view, this is a difference in loss functions as opposed to priors over the parameters. With a posterior mean, you're making the optimal inference assuming a quadratic loss; a posterior mode is appropriate for a 0-or-1 loss.)
- FeepingCreature 7y ago
  That's why your analysis shouldn't report posterior probability but degree of shift in confidence.
madhadron 7y ago
> it makes it very explicit that you pick a prior But you don't, in general, pick a prior. You pick a procedure that has an expected loss under various conditions. It's one player game theory. If you happen to have a prior, then you can use it to choose a unique procedure that has minimal expected risk for that prior given the loss function, but even so that may not be what you want. For example, you may want a minimax procedure, which may be quite different from the Bayes procedure.
- jbay808 7y ago
  Minimax still requires a probability distribution, which means you need a prior. Edit: Based on the downvotes, I see my audience is not convinced. I'll repeat an explanation I posted a while ago. Probably should make this a blog post because I see this claim quite often. I'd love to know what book you read it in. -- In minimax regret, you have a set of available decisions D, and a set of possible states of nature N, and a utility U(D,N). Each state of nature also has a probability P(N) (which can be influenced by the decision too in some problems). States of nature include "interest rates rise 1%", "interest rates fall 1%", and "interest rates stay the same". Decisions include "invest in stocks" and "invest in bonds". Minimax regret proposes to ignore the probabilities P(N), instead suggesting a way to make a decision purely based on the utilities of the outcomes. But that is actually an illusion. Outside of math class word problems, we don't have N or U(D,N) handed to us on a silver platter. There is always an infinite range of possible states of nature, many of which have a probability approaching but never reaching zero, including states such as "win the lottery", "communist revolution", and "unexpected intergalactic nuclear war". In commonsense decision-making we don't include those states of nature in our decision matrix, because our common sense rules them out as being implausible before we even think about our options. You wouldn't choose to invest in bonds just because stocks have the most regret in the event of a communist takeover. So what actually happens is we intuitively apply some probability threshold that rules out states of nature falling below it from our consideration. Then we minimize max regret on the remaining "plausibly realistic" states of nature. Humans are so good at doing probability mentally that this step happens before we even realize it. But if you are writing code that makes decisions, you'll need to do it, and so you'll need to have at least a rough stab at the probability distributions.
mikorym 7y ago
Are all priors an application of Bayes's theorem? It is confusing to me that there is talk of Bayesian statistics vs. frequentist statistics when both are often used in conjunction. The classic example of a medical test with false positives and false negatives and the prior being incidence in the general population comes to mind. To me that is not just an example of Bayes, but a combination of frequentist statistics with Bayes's theorem. I also seem to recall that Bayes's theorem appears in a standard first year probability and statistics course.
- eli_gottlieb 7y ago
  Bayesian statistics: the parameters you want to infer are modeled as random variables with a non-empirical prior, and Bayes' rule is used to draw inferences. Frequentist statistics: you construct estimators for the parameters you care about, subject to appropriate loss/risk criteria, but without any explicit "prior knowledge". Frequentist statistics with Bayes' theorem: you use available empirical data, plus some exponential-family distribution, to construct an informed prior, then use Bayes' rule to update the prior on evidence. You use this Bayesian approach only for unobservable hypotheses, rather than for parameters which can be estimated. Machine learning: you stack lots and lots of polynomial regressors onto each-other and train them with a loss function until they predict well on the validation set.
  - jbay808 7y ago
    What do you mean by a "non-empirical prior"?
    - chongli 7y ago
      Bayesian statistics is sometimes called subjectivist statistics. Probability in Bayesian statistics reflects your degree of belief in some potential outcome. If you conduct an experiment, you use Bayes’ theorem to update your degree of belief, which is now conditional on the outcome of your experiment. By quantifying your degree of belief in a prior, you give yourself some starting point (rather than just assuming 0 probability), even if that prior is only an educated guess and not some well researched position. This can be good because you might not have done the research yet.

jefft255 7y ago

In robotics, particularly in bayesian filtering (KFs and so on), I find the idea of a "prior" solid and I don't see any frequentist alternatives. Your prior is easy to understand: whatever your posterior for your state was at the previous timestep, updated using the actions you wanted your robot to accomplish. Inference is then refining this prior using the observation that the robot makes. There's nothing hand-wavy about that; if you do bayesian statistics with bad priors of course you're going to get bad inference. I guess the author just warns about being careful about your assumption which is always good.

skybrian 7y ago
I'm curious what happens when you reboot your robot. What's the first prior?
- bonoboTP 7y ago
  You initialize it to something. Like if you have a robot for running around in a specific building or warehouse, you can initialize the location distribution to be uniform within the warehouse. After a few high-quality observations, the effect of the exact choice of initialization should be greatly reduced anyway.
- jtolmar 7y ago
  For car GPS, which uses the same math, I used the first GPS signal the device receives. This ends up being equivalent to the uniform prior.
- dTal 7y ago
  What do you do when you wake up? Assume you're in the same place as when you went to sleep. You won't be surprised to find yourself on the other side of the bed - slightly more surprised to find yourself on the floor, and very surprised to find yourself in another country. A large belief update is always a bit of a shock.

Akababa 7y ago

I don't know, this seems to be a really low-effort blog post. The given example is obviously contrived from the unreasonable improper (-\infty,\infty) prior and the low \sigma^2=1 likelihood. If it was really "pure noise" then you'd have \sigma^2=\infty which rightly gives you a flat posterior. For sure Bayesian gives you more flexibility with your assumptions, so it's easier to shoot yourself in the foot. But when used correctly it can be more powerful, and often easier to interpret.

contravariant 7y ago
Ironically the article that the example is from offers quite a nice rebuttal: > None of these examples are meant to shoot down Bayes. Indeed, if posterior inferences don’t make sense, that’s another way of saying that we have external (prior) information that was not included in the model. (“Doesn’t make sense” implies some source of knowledge about which claims make sense and which don’t.) When things don’t make sense, it’s time to improve the model. Bayes is cool with that.

olooney 7y ago

Just for context, Andrew Gelman is one of the creators of Stan[1], one of the most popular probabilistic programming platforms for Bayesian interference. He has written a popular textbook on Bayesian methods, Bayesian Data Analysis[2]. Everyone hates picking priors in Bayesian analysis. If you pick an informative prior, you can always be criticized for it (in peer review, for a business decision, etc.) The usual dodge is to use a non-informative prior (like the Jeffreys prior[3].) I interpret Gelman's point as saying this can also lead to bad decisions. Thus, Bayesian analysts must thread the needle between Scyllia and Charybdis when picking priors. That's certainly a real pain point when using Bayesian methods. However, it's pretty much the same pain point as choosing regularization parameters (or choosing not to use regularization) when doing frequentist statistics. For example, sklearn was recently criticized for turing on L2 regularization by default which could be viewed as a violation of the principle of least surprise, as well as causing practical problems when inputs are not standardized. But leaving regularization turned off is equivalent to choosing an non-informative or even improper prior. (informally in many cases, and formally identical for linear regression with normally distributed errors[4].) So Scyllia and Charybdis still loom on either side. My problem with Bayesian models, completely unrelated to Gelman's criticism, is that the partition function is usually intractable and really only amenable to probabilistic methods (MCMC with NUTS[5], for example.) This makes them computationally expensive to fit, and this in turn makes them suitable for (relatively) small data sets. But using a lot more data is the single best way to allow a model to get more accurate while avoiding over-fitting! That is why I live with the following contradiction: 1) I believe Bayesian models have better theoretical foundations, and 2) I almost always use non-Bayesian methods for practical problems. [1]: https://mc-stan.org/ https://mc-stan.org/ [2]: https://www.amazon.com/Bayesian-Analysis-Chapman-Statistical-Science/dp/1439840954 https://www.amazon.com/Bayesian-Analysis-Chapman-Statistical... [3]: https://en.wikipedia.org/wiki/Jeffreys_prior https://en.wikipedia.org/wiki/Jeffreys_prior [4]: https://stats.stackexchange.com/questions/163388/l2-regularization-is-equivalent-to-gaussian-prior https://stats.stackexchange.com/questions/163388/l2-regulari... [5]: http://www.stat.columbia.edu/~gelman/research/published/nuts.pdf http://www.stat.columbia.edu/~gelman/research/published/nuts...

perl4ever 7y ago
"Everyone hates picking priors in Bayesian analysis." Everybody hates searching for their keys in the dark.

knzhou 7y ago

But this isn't actually a criticism of Bayes at all. Yes, the result depends on your prior. But the result always depends on your preconceptions -- even in frequentist statistics, where it determines which statistical tests you use and which hypotheses you test and what p-value cutoff is reasonable. It's better to have this up front. Or, you can publish Bayesian update factors, which are prior-independent.

sunstone 7y ago

The human brain is the best Bayesian model builder that evolution has yet devised. A good place to start assessing its weaknesses is to observe your own brain messing up. This shouldn't be hard to do.

madhadron 7y ago
Why do you think that the human brain is Bayesian?
- c2471 7y ago
  I bring out a coin; I tell you nothing, and ask you to guess what the probability of heads is. What do you guess? Unless you have reason to believe I am trying to deceive you, it will be able 50% because you have a lot of knowledge from other contexts that tells you this is true. The arrow is probably the other way round than you state -the brain probably isn't Bayesian; being Bayesian is modelled on how humans process and contextualise decisions. I'm not even sure how a frequentist would construct a model to estimate an outcome with no observations.
  - madhadron 7y ago
    > I'm not even sure how a frequentist would construct a model to estimate an outcome with no observations. The same way a Bayesian would, since it's a question about probabilities of hypothetical experiments, not about statistics. Or you go through decision theory instead of mucking about with half baked ideologies. > being Bayesian is modelled on how humans process and contextualise decisions. This is false.

Majromax 7y ago

> Example abridged: a draw from N(phi,1) for unknown phi is 1. Bayesian reasoning with a uniform prior gives an 86% posterior probability that phi > 0 I'm not sure I see the problem here? If it's counterintuitive, it's only because we treat N(0,1) as the normal distribution, so our true prior is that if we pick a distribution out of a hat we're more likely to have N(0,1) than anything else. Suppose I truly know nothing but what is given in the quote. On the basis of symmetry, I'd have to conclude that P(phi<0) is the same as P(phi>2). If the blogger had phrased this as "86% posterior probability that phi < 2", I don't think it would be so surprising. In fact, the blogger describes this draw as: > after seeing an observation that is statistically indistinguishable from noise. which to me presupposes a great deal of information about what 'noise' is supposed to look like.

j7ake 7y ago

The example should of course ring caution bells but at least in Bayes you can figure out why your inference is doing unreasonable things by examining each of your assumptions. In this case it’s the prior that needs fixing. Are there alternative methods that are better than the Bayes method for this toy example?

TTPrograms 7y ago
Seriously, as soon as he said "flat prior on theta" I had huge alarm bells go off. Garbage in garbage out.

roenxi 7y ago

There is a certain intellectual laziness in this perspective as might be expected from a short blog post - obviously Bayes' formula is theoretically sound because it is trivial to deduce and prove. So we know that if the conclusion is not acceptable then either the method, the prior or the evidence is not acceptable. Evidence and method can be ruled out; so the prior was not reasonable. Basically, he's saying that he doesn't believe the prior is flat. A reasonable thing to say too - as he says practically speaking if we suspect the distribution is probably random noise then the prior is we are probably looking at noise. So in practice the prior is heavily weighted towards 0. It isn't intellectually honest to use an uniformed prior unless you think the probability of a process being statistical noise is almost 0.

6gvONxR4sf7o 7y ago
>obviously Bayes' formula is theoretically sound because it is trivial to deduce and prove. Quantum mechanics doesn't follow the usual probability rules, so you can't really say "obviously Bayes' formula is theoretically sound." It certainly seems like Bayes theorem should apply universally but apparently it doesn't. Or at least, the jury's still out. https://en.wikipedia.org/wiki/Quantum_probability https://en.wikipedia.org/wiki/Quantum_probability

howlin 7y ago

Bayesian modeling can be very powerful when it works but it can also be catastrophic when it fails. It helps to think about this in an adversarial decision theoretic context where you play a prediction game against an opponent (usually called Nature). We can think of the game as discovering the best model to explain a set of observations. The Bayesian believes that Nature picks the true model that generated the observations by sampling the prior. This is actually a huge assumption to make, which is why Bayesian methods work so well when the assumption is close to the truth. Frequentists make the assumption that Nature chooses the underlying true model from a set of possible models. Beyond restricting the set of models Nature can choose from, frequentists make no further assumptions about the selection process. This is a strictly weaker assumption than the Bayesian makes, which means frequentist methods will do better when the specified prior grossly misrepresents Nature's decision making process. There are even weaker assumptions that can be made about how Nature chooses the data. Regret-based model inference allows for a more adversarial game with Nature where the data may not come from the class of models considered at all. If Nature truly behaves this way, then Bayesian decision making can catastrophically fail.

c2471 7y ago
This ignores the main strength of a Bayesian workflow. You can straight forwardly quantify the effect of your prior choice on your inference - pick a different prior; how much does that change the inference, etc etc. A good Bayesian workflow does not assume a prior to be true; it should be based on available evidence, and then stressed. To be a bit more concrete, let's say we wish to model the height of kangaroos. We come up with a model form, say regression, and a bunch of potential features. If we are Bayesian we might say; "I think nature prefers simple stable solutions, so I'll put a N(0,d) prior on my weights. We then compute a posterior and get a range of credible values. We can then say, "hey, what if I'm wrong and actually it's a student t, or it's flat prior or X or y or z", and use principled tools like marginal likelihood to say which family of models works best, do prior posterior comparisons to see how observations changed our prior etc etc. If we do this under a frequentist framework we compute the regression coefficients, and can get some confidence bounds with some appeal to asymptotics (and nobody I've ever seen actually makes any attempt to validate these assumptions). And even when we are done, we get a confidence interval that has such a truly unintuitive definition that almost every person who is not a stats PhD fundamentally misinterprets. To say frequentists make less assumptions is not true- they are just less explicit, and I consider it a strength not a weakness to highlight choices made by the statistician.

selectionbias 7y ago

My problem with the 'Bayes=rationality' type of argument is that it ignores context and isn't really a case for reporting Bayesian vs frequentist estimates. If I am a researcher publishing results then I have an audience who interpret my results. If my audience is Bayesian and accept my model then all I need to do is report sufficient statistics and they can make their own Bayesian inferences given their priors, or better yet, I can just post my whole dataset. The very reason we need to report things like credible sets or confidence intervals rather than just sufficient statistics is because audiences in the real world want summary stats that they can easily interpret and are transparent. The best approach to inference is one that is the most useful to audiences, and that depends on context and practicalities rather than on some underlying philosophy of subjective vs objective probabilities.

abeppu 7y ago

I feel this post should be considered along with its sibling: https://statmodeling.stat.columbia.edu/2019/12/04/whats-wrong-with-null-hypothesis-significance-testing/ https://statmodeling.stat.columbia.edu/2019/12/04/whats-wron... I think reading either alone is prone to lead readers to a false understanding of Gelman's perspective.

metasj 7y ago

Many analyses of the world aren't bayesian /or/ frequentist, they use much simpler pattern-matching, with feedback loops that update the approach used as well as the conclusion. Problems start w/ assuming you have to choose one of those approaches to estimate the future...

bonoboTP 7y ago

"Bayesian" is an overloaded term. There's Bayes' theorem/rule, which basically everyone agrees with, since it's a theorem that's very simple to prove with a few high school math operations. Then there is the philosophical Bayesian interpretation of probability, that claims that probabilities are fundamentally about our own mental state of belief, as opposed to frequencies at the limit of infinite repetition of some experiment. Then there is the Bayesian methods of statistics / machine learning etc, which are about handling parameters as random variables and the observed data as fixed, as opposed to assuming that there's one fixed parameter (without a distribution to talk about) and the data should be modeled as random (from an oversimplified bird's eye view). And it was also oversold as a miracle cure for all our problems: for some time, before the deep learning era, you just had to have "Bayesian" in your ML paper title to make it sexy and interesting. Then there is the online Bayesian rationalist community, where Bayes is used to explain the meaning of life, the universe, it's the great grand explanation of everything, a self help tool, the key to seeing the light, a semi-religious experience, the way to enlightenment (they even call it the Way, capitalized - I guess a Buddhist reference?). As if being Bayesian was this secret club, that sets you apart from average people, a symbol of belonging to the in-group etc. [1] It's important to keep these apart. [1] For example: https://youtu.be/NEqHML98RgU?t=73 https://youtu.be/NEqHML98RgU?t=73 (it's explicitly not about the math but about self-help and intuition to benefit our lives etc...)

Gusen 7y ago

Nadler, Schiff, & Mueller each raped and killed boys in Buffalo on the night of January 14 2019, as Trump did earlier that morning. Impeachment is a vehicle for keeping power, they are all working together. Full audio proving this entirely here, all three admit collusion w/President. PDF updated. \\14 January 2019 11:23pm: Jerrold Nadler steps up to take his turn during the Illuminati "rape party". Nadler rapes and kills three boys in under a minute, however, there was a problem. One of the boys was already dead, so he requests a new one. Jerrold Nadler then requests another ten boys. Donald Reeves: "That's a million dollar request". Nadler responds: "...you guys will cover it. I'm gonna keep Trump in power" (Trump raped and killed a dozen boys 6:30-7:00am that morning). By the time Nadler is finished, he had raped and killed 24 boys. Audio pulled from the video linked below: 14JanCh4_2300-0000.mp3 - Nadler starts at about 20:00 in. https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU66GuU/view?usp=sharing https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU... 14JanCh2_2304-2359.mp3 https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9WLr1x1/view?usp=sharing https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9W... 14JanCh3_2302-2359.mp3 https://drive.google.com/file/d/1wdDIsxfsX7xTBIBZYzV4iE4xEdgmwy5l/view?usp=sharing https://drive.google.com/file/d/1wdDIsxfsX7xTBIBZYzV4iE4xEdg... Further disclosure by Porter about 24:47 in: Oblivious tenant Brian Schlenker comments on something unrelated to the ongoing events: "..call the fucking police...", to which is responded with: "...that's funny because we own the police (Buffalo Police Department), we pay them $6 million dollars a month." 19 minutes in Fred Norris, formerly of the Howard Stern show, is acknowledged on the system. At 19:47 in Porter admits that Brian Schlenker will be the owner of this footage should it be discovered. January 15, 2019 00:20: Special Counsel Robert Mueller takes his turn at the Illuminati "rape party" in Buffalo, New York. Mueller rapes and kills twelve boys. About roughly 00:55 Representative Adam Schiff who will also be leading an impeachment effort, also requests the same deal as Nadler, and then tries to make a case for getting more than Nadler and Mueller. Adam Schiff rapes and kills three boys. Mueller and Schiff all receive $3 billion dollars each for joing the group. Nadler came back to witness these two rape to make sure they were all bound together under one purpose: keep Trump in power, and also to confirm the payments to each, including his $10.5 billion dollar payment. Between Mueller and Schiff turns, the group issues orders for ten women to begin prepping more boys for rape. They are former friends and family of Brian Schlenker, and also some long standing Illuminati members who include Elsa Hosk, Gigi and Bella Hadid. Again, the "prep" these females engage means they perform oral sex on the boys’ penis and anus, as a child rapist like Henry Porter would, while trying to remove fecal matter from the boy prior to handing them over to be raped and subsequently murdered. Just a head's up, my voice is scattered throughout all of the footage within the links posted for this update, and is quite loud relative to the desired content at times. Audio links below: 15JanCh4_000-100.mp3 https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2OQnwZ/view?usp=sharing https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2... 15JanCh4_100-200.mp3 https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw746LeCL/view?usp=sharing https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw74... 15JanCh2_000-100.mp3 https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62tgCkj/view?usp=sharing https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62... 15JanCh2_100-200mp3 https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_rGSFH/view?usp=sharing https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_... 15JanCh3_000-100.mp3 https://drive.google.com/file/d/1zsEwYC875wQu6btsSbNznHsJinxYHxUL/view?usp=sharing https://drive.google.com/file/d/1zsEwYC875wQu6btsSbNznHsJinx... 15JanCh3_100-200.mp3 https://drive.google.com/file/d/15UAY2er2LdTViXG4azy_bc_oQUzvLpFh/view?usp=sharing https://drive.google.com/file/d/15UAY2er2LdTViXG4azy_bc_oQUz... //Full 110 page PDF [updated 2Dec2019]: FBI_FinalDraft_26Jul2019_BSchlenker.pdf https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJc9cGk/view?usp=sharing https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJ... //This post will be censored when this account logs off, the posts are "shadow banned". They try to make it look like the post is live, but it is not. Here is an example. https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJedYKz/view?usp=sharing https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJ... \\Previously: President of the United States Donald J. Trump rapes and kills 15 boys in Buffalo, NY on January 10, 2019. This is audio of the event from 10-Jan-2019. Download the mp3 and put on headphones, and turn the volume all the way up. 10JanCh3_1255-1557.mp3 https://drive.google.com/file/d/18lTt_YKFEtsV6YDNzXFVQYLkStY-kSg2/view?usp=sharing https://drive.google.com/file/d/18lTt_YKFEtsV6YDNzXFVQYLkStY... This is audio of the President, Donald J. Trump, demanding a $4 billion dollar bribe from child rapists to “take a blind eye” on January 3, 2019. Trump becomes one on January 14, 2019. Also, here is the big reason the major networks do not report any of it. //Download the video, turn the volume all the way up and put head phones on. Note: there is not much to see in the video, the audio is picked up from another [illegal surveillance] system. Trump is on a call from with Henry Porter and Gigi Hadid. See page 63. Bribe demand at 10:18am: 3JanCh3_900-1100.avi https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77YG0Vr/view?usp=sharing https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77... //President of the United States, Donald J. Trump, rapes and kills his first boy at 6:32am. Video link below: 14JanCh3_600.avi https://drive.google.com/file/d/154QvA5hwyHGYIVXtod1ZbsOHFUJNbqZW/view?usp=sharing https://drive.google.com/file/d/154QvA5hwyHGYIVXtod1ZbsOHFUJ... 14JanCh2_600-700.avi https://drive.google.com/file/d/19UkqmnMwZiWy7xxWngltqwoKLTJL-IZ_/view?usp=sharing https://drive.google.com/file/d/19UkqmnMwZiWy7xxWngltqwoKLTJ... //On January 18, 2019 at 8:31am (see page 8) Trump acknowledges the four billion dollar bribe and says: "Let's get it done and get to fucking some kids." Video link below: 18JanCh3_725-.avi https://drive.google.com/file/d/1bVTcGq5Z9oOSAiOQcKYrmuK4TwoNmY1h/view?usp=sharing https://drive.google.com/file/d/1bVTcGq5Z9oOSAiOQcKYrmuK4Two... //A big reason this has not been reported by the major news networks is right here. Lester Holt of NBC Nightly News, apparently a member of the Illuminati since the 1980's, along with ABC Nightly News lead anchor David Muir, stop over to the Porter studio in Buffalo on January 14, 2019 at 5:00am. They both rape and kill about two dozen boys by 6:00am. Muir starts around 5:15am, then Holt about 5:38am. Multi-billionaire Rupert Murdoch, owner of News Corp & Fox Corporation, takes his turn after Holt. Video links below: 14JanCh3_500-601.avi https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX3o7UL/view?usp=sharing https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX... 14JanCh2_530-600.avi https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1RwidV/view?usp=sharing https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1... Adding to the reason this is not picked up by the media, CBS & Viacom owner Sumner Redstone and Leslie Moonves rape boys following the President. 14JanCh3_700.avi https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXaB4aI0/view?usp=sharing https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXa... 14JanCh2_700-800.avi https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWYKfRDp/view?usp=sharing https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWY... //This is the tip of the ice berg. //Full 110 page PDF [updated 2Dec2019]: FBI_FinalDraft_26Jul2019_BSchlenker.pdf https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJc9cGk/view?usp=sharing https://drive.google.com/file/d/1Sj9EN_pHmicKS6rFQlmk67knMdJ... //This post will be censored when this account logs off, the posts are "shadow banned". They try to make it look like the post is live, but it is not. Here is an example. https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJedYKz/view?usp=sharing https://drive.google.com/file/d/1zxS8JESoIg7uxRkUgdptMsF6SuJ...

ummonk 7y ago

>Put a flat prior on theta and you end up with an 84% posterior probability that theta is greater than 0. Step back a bit, and it’s saying that you’ll offer 5-to-1 odds that theta>0 after seeing an observation that is statistically indistinguishable from noise. That can’t make sense. Go around offering 5:1 bets based on pure noise and you’ll go bankrupt real fast. If you think it's likely to be pure noise, why the hell would you put a flat prior on it? Note also that nonflat priors are implicit in significance testing - e.g. p95 significance is similar to putting a 95% prior on the null hypothesis, and p99 significance is similar to putting a 99% prior on the null hypothesis.

gweinberg 7y ago

The author has a major fundamental misconception as to how probability works. If I say "the probability that proposition X is true is 0.5", that means that based on the information available to me right now it's equally likely likely to be true as false. That's not even remotely similar to saying I would offer an even money bet.

baron_harkonnen 7y ago
Ignoring the fact that “the author” is one of the most respected statistians in the world today... there is no debate on how to translate probabilities into odds: odds(x) = p(x)/(1-p(x)) Thats the definition of “odds”. so in this case it is quite clear that the odds for X is 1, implying and even money bet.

syrrim 7y ago

If the goal is to avoid bankruptcy, then the probability needs to be interpreted differently. If you bet the house every time, you're guaranteed to go bankrupt eventually. Suppose instead you bet half your money on an event of 50% probability. If you take 1:1 odds on this, then when you lose, your money is divided by 2, but when you win it is only multiplied by 1.5. Your money will tend to decrease over time. You need to pick odds 1:a such that 1+a/2=2 => a=2. We recover our regular betting odds by betting a smaller portion of our money. If we bet a portion 1/d of our money on an event of probability 1/p, we needs odds 1:a such that 1+a/d=(d/(d-1))^(p-1). For large enough d we get a=p-1, as we would expect. Assume again you're betting half your money each round, but take a probability of winning of 84%, as in the article. You should take that bet at 1:1.14 odds, much less than the recommended 1:5 odds.

ravar 7y ago
For the curious, look up the kelly criterion, it formalizes this thinking.
ikeboy 7y ago
This has nothing to do with interpreting probability, but with a utility function that's not linear in terms of wealth. With decreasing marginal returns to wealth, the same bet becomes less attractive at lower wealth levels. Although this can't fully explain risk aversion, see https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.15.1.219 https://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.15.1.219

mycall 7y ago

84% isn't that great for predictions compared to DNNs, RNNs or other modern ML algorithms.

pontusrehula 7y ago

To criticize is easy but it feels incomplete if one doesn't provide any clues of what the supposedly better alternatives would be.