11 ms·

How Bayes’ Rule Emerged Triumphant from Two Centuries of Controversy

152 points by evanb 10y ago

arcanus 10y ago
Bayesian here. 'The theory that would not die' is a wonderful read, and notes how many scientists used Bayesian techniques (subjective probability) in a variety of contexts while the field was still unpopular (arguably, heretical) in the mainstream statistical community. Bayes was nevertheless used to inform ballistics calculations, help crack the enigma code, or inform search patterns for lost nuclear weapons. I don't think we have seen the end of Bayes, either, as it is very useful for uncertainty quantification in the engineering Sciences, machine learning techniques, or even discovering the Higgs Boson.
- krastanov 10y ago
  To your last paragraph: most of my fellow physicists do not care much about the distinction between bayesian and frequentist approaches, they just use a prepackaged tool to process their data. But of the few that learn those tools intimately, cosmologists are always bayesians (because we have only one Universe), and experimental particle physicists are always frequentist (because generating a ton of repeated test events is "cheap").
  - jessriedel 10y ago
    The particle physics do sometimes make Bayesian plots and end up arguing over priors. See this, for example: https://cds.cern.ch/record/1375842/files/ATL-PHYS-PUB-2011-011.pdf https://cds.cern.ch/record/1375842/files/ATL-PHYS-PUB-2011-0... And I've heard more than one confused argument between particle physicists about whether Bayesianism or frequentism is "right"...
    - asher_ 10y ago
      Given the high level of statistical certainty that particle physicists strive for from experimental data, how much of a difference does the approach matter by the point where they are confident enough to endorse the theory? For instance, if you are going for five sigma (like in the Higgs Boson discovery), how would the different techniques change when the data is considered sufficient, if at all?
      - jessriedel 10y ago
        The five-sigma threshold is a limit for making big public announcements, or claiming definitive discrete discoveries (like new particles) that non-particle physicists can take for granted. But there's lots of intermediate work done at accelerators that can't reach that threshold but still informs further work, and even influences the construction and design of the next multi-billion dollar machines. The bi-annual Review of Particle Physics is as thick as a phone book and filled with (usually) 95%-confidence intervals for measured values. For instance, here is the 89-page summary of what is known about the tau particle: http://pdg.lbl.gov/2015/listings/rpp2015-list-tau.pdf http://pdg.lbl.gov/2015/listings/rpp2015-list-tau.pdf
- j_s 10y ago
  The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy https://amzn.com/dp/B0050QB3EQ https://amzn.com/dp/B0050QB3EQ
- amasad 10y ago
  As someone new to Bayesianism I'd be interested in hearing your experience applying it in day-to-day life. How useful do you think it is to the ordinary Joe? From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found that they had fatal incidents in the past. My friends were convinced that given that they had incidents either the probability that we would have an incident using them doesn't change. Or worse, that since it has already happened, it's now less likely that an incident would happen. Wat.
  - Houshalter 10y ago
    I think your anecdote is good. Knowing the formula for Bayes theorem isn't necessary, and I've never needed to try to do the math in my head or anything. But understanding probability theory, and having an intuitive feel for what probability is, is really important.
  - mikekchar 10y ago
    The one place I've used Bayes (hopefully properly!) is in a spaced repetition flash card program. Usually spaced repetition algorithms wait a certain amount of time based on how many times you have seen and remembered a card. The more times you have remembered it, the longer you wait. It then creates a schedule for each day. You review the cards that have "expired" their wait time. I wanted to turn this upside down. Instead I sorted the cards by how likely you would be to remember them (a function of the number of times you have already remembered it and the amount of time that has passed since you last saw it). I put the least likely to be remembered cards first and the most likely to be remembered cards last. As you reviewed the cards you would either remember them or not remember them. I supposed that cards grouped together had a similar probability of being remembered (only valid if my sort algorithm was correct, but I had some confidence on that since I based it on somebody else's research ;-) ). I then used Bayes to estimate the probability of getting the card correct. So instead of scheduling the cards, I simply had the user keep reviewing until I had a certain confidence that there was a 90% or more of getting the cards correct. At which point I left the rest for another day (the probability goes down over time). Interestingly as the test is binary (remembered/not remembered) Bayes could be simplified down to getting it right a certain number of times in a row. This was simpler for the user to understand without significantly reducing the accuracy of the estimate. Gratifyingly the system worked incredibly well. I added an option to continue drilling the cards after you hit the 90% mark and I very rarely got into a situation where the estimate was wrong.
- hacker42 10y ago
  As a Bayesian you can maybe answer my question here: https://news.ycombinator.com/item?id=11985312 https://news.ycombinator.com/item?id=11985312 I would appreciate it!
dschiptsov 10y ago
Brexit is the best example so far. That painful dissonance between so called reality and these probabilistic models.
- dschiptsov 10y ago
  Probabilities makes sense only with absolutely certain things like a fair coin or a dice. In cases where there is no absolute certainty about how many sides or dimensions your "dice" has and that it is not biased and that there is no other forces or factors in play probability ceases to make sense. Probability of A, given B becomes meaningless when either A or B aren't precisely defined (like in the case of a "fair coin") and so is the relationship between the two. Application of the Bayesian rule to "estimated" probabilities is just wrong and unscientific (in the face of ambiguity avoid the temptation to guess). Multiplying and dividing nonsense by nonsense yields nonsense. The global financial crisis and recent cock-sure consensus about outcome of the brexit referendum the day before voting are good evidences.
  - greenshackle 10y ago
    I disagree. There's a large grey area between 'completely unknown' and 'scientific certainty'. I prefer guessing and doing computations with my guesses than throwing my hands up in the air and calling it unknowable. When presented with new evidence it's better to write a number down for your degree in belief in X, ask how much you should change that belief based on the new data, and update your probability estimate, than to just go with your feelings. You're not doing actual bayesian computations, that's totally untractable for anything that's not a very well defined problem, but doing 'pseudo-bayesian' updating is better than not. I think of it as the fermi approximation of probabilities. You won't get accurate numbers that way but you'll get better numbers than if you just invent the answer. EDIT to add: most of the time you should then throw out the number. Just like a fermi estimate, you get a ballpark sense for the answer, not a precise answer. In the superforecasting experiment by Tetlock the best forecasters did this. They were writing down probability estimates and methodically updating them based on new data (news articles, data, etc.). They were forecasting geopolitical events, not dice rolls, and it worked (better than the alternative, obviously no one can forecast geopolitical events with high certainty).
dkbrk 10y ago
I found this talk extremely interesting despite being already somwhat familiar with the history of Bayesian methods; I heartily recommend watching it in its entirety regardless of prior familiarity with the subject. The talk was also presented very well. It's almost novel nowadays for something like this to not be some sort of powerpoint presentation, and I can't say it suffered for it.
moon_of_moon 10y ago
If you are in London you can visit the tombstone of the good Reverend Bayes at the Bunhill Fields burial grounds, a short walk from the Old Street tube station, and pay your respects. There is also a bench facing a spacious lawn that you can sit on and contemplate Bayes in silence. He probably sat there at some point, come to think of it.
danbruc 10y ago
Is there any (uncontroversial) theory that rigorously defines what a 50 % probability for heads and tails means? It certainly doesn't mean that in the long run you will obtain the same number of heads and tails because there is a (vanishing) chance that you will always get heads even though the coin is actually fair. And just saying that you will obtain the same or at least similar number of heads and tails with high probability is a circular definition at best. Does Bayesian thinking just deny the existence of intrinsic probabilities or considers them out of scope? Assuming there is such a thing as true randomness, for example in quantum physics (and it is not due to our ignorance of hidden variables like in Bohemian mechanics), could a Bayesian assign probabilities to the outcomes? Would a Bayesian be mislead into assigning something other than 50/50 for spin up and spin down if he, by chance, only observes spin up although he is really confident that this is just a statistical fluke? If he is mislead, does that mean that uncertainty about the truth of his assumptions creeps in in case the system momentarily deviates from the expected behavior?
- paol 10y ago
  > doesn't mean that in the long run you will obtain the same number of heads and tails IANAM, but in my understanding that's exactly what it means, if you express "in the long run" as "the ratio tends to 0.5 as the number of tosses tends toward ∞"
  - danbruc 10y ago
    IANAM either, but that seems to be wrong to me. For any finite number of tosses you may certainly not converge to 0.5 even if it becomes very unlikely pretty quickly not to do so. In order for this to be true, it would have to be literally impossible to always get heads on every toss no matter how often you try. But it seems counter-intuitive at the very least that it is inevitable to get tails eventually. What rules out the possibility to get heads over and over again, an infinite number of times? [1] But if it is possible to obtain an infinite number of heads and not a single tails, then it is not true that ratio always converges towards 0.5. In consequence one would have to modify the definition and say that it is only very likely to converge towards 0.5 but that obviously raises the problem that »very likely« sounds a lot like »with high probability« which is the thing we are trying to define, hence this would become a somewhat circular definition. [1] Actually it must not only be impossible to only ever get heads or tails, it must be impossible to obtain any sequence of outcomes that has not exactly the same number of heads and tails. There is only one sequence of outcomes with only heads and only one with only tails but there is already an infinite number of sequences with all heads but one tails or vice versa. And there is an enormous number of possible sequences with 40 % heads and 60 % tails or any other ratio other than 50/50.
hacker42 10y ago
I could never quite understand the divide between Bayesian statistics and frequentist statistics. Both seem to be ultimately about counting the frequency by which something occurs and normalizing this frequency with respect to the number of all possible outcomes. Bayesian statistics essentially is concerned with the application of the Bayesian updating technique by which one can iteratively improve a distribution over the values a parameter toward the true distribution using Bayes' rule. One can prove that given sufficiently many updates, the initial distribution does not matter as long it is non-zero for all possible values. What I am not quite seeing is how this has philosophical implications which divide the field into Frequentists and Bayesians. It rather seems that Bayesian statistics is just a frequentist method that is helpful for dealing with noisy measurements as one can always recover from a bad distributions using more updates.
- twanvl 10y ago
  Bayesian and frequentist approaches ultimately have a different notion of probability. In the frequentist approach, a probability of 10% means that if you repeat an experiment many times, roughly 1 out of 10 times you will observe an event. In Baysian statistics, a probability of 10% means that you are that certain about the event happening. So you would be willing to bet at 10 to 1 odds on the event happening. There doesn't have to be any repetition of the experiment for the probability to make sense. And as you can hopefully see, there is always a prior, that is, your belief about the event before doing any experiments.
  - hacker42 10y ago
    But it seems to me one can still define this in terms of frequencies using a more general definition of what is meant by repeating an experiment. For example, when you use a probability P as a degree of belief or certainty about whether a patient X with symptoms W has a particular disease Y, one would define the universe as containing all possible realities in which X has the same symptoms W but with different underlying causal factors that lead to the same symptoms. An observation is a uniform sample from this universe and the belief is that a fraction of P of these realities has the cause Y.
  - 10y ago
jordigh 10y ago
The biggest historical obstacle to Bayesian stats was low amounts of data available and difficulty of computation. Frequentist stats is optimised around these. With 30 samples and very easy computations, you are able to produce a reasonable frequentist confidence interval, sometimes even with less data. On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requires some interesting integration and the somewhat obscure Beta distribution. Now that we have lots of data and lots of computing power, Bayesian stats can show its results, after getting rebranded as "machine learning".
- tgb 10y ago
  > On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requires some interesting integration and the somewhat obscure Beta distribution. Isn't this kind of misleading? The end result of the Beta distribution, etc. is just the extremely simple-to-compute Rule of Succession [1]. [1] https://en.wikipedia.org/wiki/Rule_of_succession https://en.wikipedia.org/wiki/Rule_of_succession
  - jordigh 10y ago
    A bit, but in general, computing posterior distributions tends to very quickly lead to more complicated integrals. It just so happens that the Beta distribution is somewhat nice and symmetrical, if a bit obscure.
    - XFrequentist 10y ago
      More generally, this is why there's (still) a mild obsession with "conjugate" prior/likelihood distribution pairs - i.e. combinations of prior and likelihood that give analytically tractable posteriors[1] - despite the ability to get easily get results with MCMC. [1] Yes, everyone wants a nice posterior. You're very clever.
- eli_gottlieb 10y ago
  >Now that we have lots of data and lots of computing power, Bayesian stats can show its results, after getting rebranded as "machine learning". Partly lots of computing power, and partly the Monte Carlo revolution that enabled us to replace intractable integrals with lots of cheap CPU time.

152 points by evanb 10y ago

Bayesian here. 'The theory that would not die' is a wonderful read, and notes how many scientists used Bayesian techniques (subjective probability) in a variety of contexts while the field was still unpopular (arguably, heretical) in the mainstream statistical community. Bayes was nevertheless used to inform ballistics calculations, help crack the enigma code, or inform search patterns for lost nuclear weapons. I don't think we have seen the end of Bayes, either, as it is very useful for uncertainty quantification in the engineering Sciences, machine learning techniques, or even discovering the Higgs Boson.

krastanov 10y ago
To your last paragraph: most of my fellow physicists do not care much about the distinction between bayesian and frequentist approaches, they just use a prepackaged tool to process their data. But of the few that learn those tools intimately, cosmologists are always bayesians (because we have only one Universe), and experimental particle physicists are always frequentist (because generating a ton of repeated test events is "cheap").
- jessriedel 10y ago
  The particle physics do sometimes make Bayesian plots and end up arguing over priors. See this, for example: https://cds.cern.ch/record/1375842/files/ATL-PHYS-PUB-2011-011.pdf https://cds.cern.ch/record/1375842/files/ATL-PHYS-PUB-2011-0... And I've heard more than one confused argument between particle physicists about whether Bayesianism or frequentism is "right"...
  - asher_ 10y ago
    Given the high level of statistical certainty that particle physicists strive for from experimental data, how much of a difference does the approach matter by the point where they are confident enough to endorse the theory? For instance, if you are going for five sigma (like in the Higgs Boson discovery), how would the different techniques change when the data is considered sufficient, if at all?
    - jessriedel 10y ago
      The five-sigma threshold is a limit for making big public announcements, or claiming definitive discrete discoveries (like new particles) that non-particle physicists can take for granted. But there's lots of intermediate work done at accelerators that can't reach that threshold but still informs further work, and even influences the construction and design of the next multi-billion dollar machines. The bi-annual Review of Particle Physics is as thick as a phone book and filled with (usually) 95%-confidence intervals for measured values. For instance, here is the 89-page summary of what is known about the tau particle: http://pdg.lbl.gov/2015/listings/rpp2015-list-tau.pdf http://pdg.lbl.gov/2015/listings/rpp2015-list-tau.pdf
j_s 10y ago
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy https://amzn.com/dp/B0050QB3EQ https://amzn.com/dp/B0050QB3EQ
amasad 10y ago
As someone new to Bayesianism I'd be interested in hearing your experience applying it in day-to-day life. How useful do you think it is to the ordinary Joe? From my brief experience, after learning Bayes, my intuition about things involving probabilities grew very different than the people around me. For example, my friends were planning a skydiving trip and I googled the name of the skydiving business and found that they had fatal incidents in the past. My friends were convinced that given that they had incidents either the probability that we would have an incident using them doesn't change. Or worse, that since it has already happened, it's now less likely that an incident would happen. Wat.
- Houshalter 10y ago
  I think your anecdote is good. Knowing the formula for Bayes theorem isn't necessary, and I've never needed to try to do the math in my head or anything. But understanding probability theory, and having an intuitive feel for what probability is, is really important.
- mikekchar 10y ago
  The one place I've used Bayes (hopefully properly!) is in a spaced repetition flash card program. Usually spaced repetition algorithms wait a certain amount of time based on how many times you have seen and remembered a card. The more times you have remembered it, the longer you wait. It then creates a schedule for each day. You review the cards that have "expired" their wait time. I wanted to turn this upside down. Instead I sorted the cards by how likely you would be to remember them (a function of the number of times you have already remembered it and the amount of time that has passed since you last saw it). I put the least likely to be remembered cards first and the most likely to be remembered cards last. As you reviewed the cards you would either remember them or not remember them. I supposed that cards grouped together had a similar probability of being remembered (only valid if my sort algorithm was correct, but I had some confidence on that since I based it on somebody else's research ;-) ). I then used Bayes to estimate the probability of getting the card correct. So instead of scheduling the cards, I simply had the user keep reviewing until I had a certain confidence that there was a 90% or more of getting the cards correct. At which point I left the rest for another day (the probability goes down over time). Interestingly as the test is binary (remembered/not remembered) Bayes could be simplified down to getting it right a certain number of times in a row. This was simpler for the user to understand without significantly reducing the accuracy of the estimate. Gratifyingly the system worked incredibly well. I added an option to continue drilling the cards after you hit the 90% mark and I very rarely got into a situation where the estimate was wrong.
hacker42 10y ago
As a Bayesian you can maybe answer my question here: https://news.ycombinator.com/item?id=11985312 https://news.ycombinator.com/item?id=11985312 I would appreciate it!

dschiptsov 10y ago

Brexit is the best example so far. That painful dissonance between so called reality and these probabilistic models.

dschiptsov 10y ago
Probabilities makes sense only with absolutely certain things like a fair coin or a dice. In cases where there is no absolute certainty about how many sides or dimensions your "dice" has and that it is not biased and that there is no other forces or factors in play probability ceases to make sense. Probability of A, given B becomes meaningless when either A or B aren't precisely defined (like in the case of a "fair coin") and so is the relationship between the two. Application of the Bayesian rule to "estimated" probabilities is just wrong and unscientific (in the face of ambiguity avoid the temptation to guess). Multiplying and dividing nonsense by nonsense yields nonsense. The global financial crisis and recent cock-sure consensus about outcome of the brexit referendum the day before voting are good evidences.
- greenshackle 10y ago
  I disagree. There's a large grey area between 'completely unknown' and 'scientific certainty'. I prefer guessing and doing computations with my guesses than throwing my hands up in the air and calling it unknowable. When presented with new evidence it's better to write a number down for your degree in belief in X, ask how much you should change that belief based on the new data, and update your probability estimate, than to just go with your feelings. You're not doing actual bayesian computations, that's totally untractable for anything that's not a very well defined problem, but doing 'pseudo-bayesian' updating is better than not. I think of it as the fermi approximation of probabilities. You won't get accurate numbers that way but you'll get better numbers than if you just invent the answer. EDIT to add: most of the time you should then throw out the number. Just like a fermi estimate, you get a ballpark sense for the answer, not a precise answer. In the superforecasting experiment by Tetlock the best forecasters did this. They were writing down probability estimates and methodically updating them based on new data (news articles, data, etc.). They were forecasting geopolitical events, not dice rolls, and it worked (better than the alternative, obviously no one can forecast geopolitical events with high certainty).

dkbrk 10y ago

I found this talk extremely interesting despite being already somwhat familiar with the history of Bayesian methods; I heartily recommend watching it in its entirety regardless of prior familiarity with the subject. The talk was also presented very well. It's almost novel nowadays for something like this to not be some sort of powerpoint presentation, and I can't say it suffered for it.

moon_of_moon 10y ago

If you are in London you can visit the tombstone of the good Reverend Bayes at the Bunhill Fields burial grounds, a short walk from the Old Street tube station, and pay your respects. There is also a bench facing a spacious lawn that you can sit on and contemplate Bayes in silence. He probably sat there at some point, come to think of it.

danbruc 10y ago

Is there any (uncontroversial) theory that rigorously defines what a 50 % probability for heads and tails means? It certainly doesn't mean that in the long run you will obtain the same number of heads and tails because there is a (vanishing) chance that you will always get heads even though the coin is actually fair. And just saying that you will obtain the same or at least similar number of heads and tails with high probability is a circular definition at best. Does Bayesian thinking just deny the existence of intrinsic probabilities or considers them out of scope? Assuming there is such a thing as true randomness, for example in quantum physics (and it is not due to our ignorance of hidden variables like in Bohemian mechanics), could a Bayesian assign probabilities to the outcomes? Would a Bayesian be mislead into assigning something other than 50/50 for spin up and spin down if he, by chance, only observes spin up although he is really confident that this is just a statistical fluke? If he is mislead, does that mean that uncertainty about the truth of his assumptions creeps in in case the system momentarily deviates from the expected behavior?

paol 10y ago
> doesn't mean that in the long run you will obtain the same number of heads and tails IANAM, but in my understanding that's exactly what it means, if you express "in the long run" as "the ratio tends to 0.5 as the number of tosses tends toward ∞"
- danbruc 10y ago
  IANAM either, but that seems to be wrong to me. For any finite number of tosses you may certainly not converge to 0.5 even if it becomes very unlikely pretty quickly not to do so. In order for this to be true, it would have to be literally impossible to always get heads on every toss no matter how often you try. But it seems counter-intuitive at the very least that it is inevitable to get tails eventually. What rules out the possibility to get heads over and over again, an infinite number of times? [1] But if it is possible to obtain an infinite number of heads and not a single tails, then it is not true that ratio always converges towards 0.5. In consequence one would have to modify the definition and say that it is only very likely to converge towards 0.5 but that obviously raises the problem that »very likely« sounds a lot like »with high probability« which is the thing we are trying to define, hence this would become a somewhat circular definition. [1] Actually it must not only be impossible to only ever get heads or tails, it must be impossible to obtain any sequence of outcomes that has not exactly the same number of heads and tails. There is only one sequence of outcomes with only heads and only one with only tails but there is already an infinite number of sequences with all heads but one tails or vice versa. And there is an enormous number of possible sequences with 40 % heads and 60 % tails or any other ratio other than 50/50.

hacker42 10y ago

I could never quite understand the divide between Bayesian statistics and frequentist statistics. Both seem to be ultimately about counting the frequency by which something occurs and normalizing this frequency with respect to the number of all possible outcomes. Bayesian statistics essentially is concerned with the application of the Bayesian updating technique by which one can iteratively improve a distribution over the values a parameter toward the true distribution using Bayes' rule. One can prove that given sufficiently many updates, the initial distribution does not matter as long it is non-zero for all possible values. What I am not quite seeing is how this has philosophical implications which divide the field into Frequentists and Bayesians. It rather seems that Bayesian statistics is just a frequentist method that is helpful for dealing with noisy measurements as one can always recover from a bad distributions using more updates.

twanvl 10y ago
Bayesian and frequentist approaches ultimately have a different notion of probability. In the frequentist approach, a probability of 10% means that if you repeat an experiment many times, roughly 1 out of 10 times you will observe an event. In Baysian statistics, a probability of 10% means that you are that certain about the event happening. So you would be willing to bet at 10 to 1 odds on the event happening. There doesn't have to be any repetition of the experiment for the probability to make sense. And as you can hopefully see, there is always a prior, that is, your belief about the event before doing any experiments.
- hacker42 10y ago
  But it seems to me one can still define this in terms of frequencies using a more general definition of what is meant by repeating an experiment. For example, when you use a probability P as a degree of belief or certainty about whether a patient X with symptoms W has a particular disease Y, one would define the universe as containing all possible realities in which X has the same symptoms W but with different underlying causal factors that lead to the same symptoms. An observation is a uniform sample from this universe and the belief is that a fraction of P of these realities has the cause Y.
- 10y ago

jordigh 10y ago

The biggest historical obstacle to Bayesian stats was low amounts of data available and difficulty of computation. Frequentist stats is optimised around these. With 30 samples and very easy computations, you are able to produce a reasonable frequentist confidence interval, sometimes even with less data. On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requires some interesting integration and the somewhat obscure Beta distribution. Now that we have lots of data and lots of computing power, Bayesian stats can show its results, after getting rebranded as "machine learning".

tgb 10y ago
> On the other hand, even the simplest Bayesian analysis of determining the probability of heads in a coin flip requires some interesting integration and the somewhat obscure Beta distribution. Isn't this kind of misleading? The end result of the Beta distribution, etc. is just the extremely simple-to-compute Rule of Succession [1]. [1] https://en.wikipedia.org/wiki/Rule_of_succession https://en.wikipedia.org/wiki/Rule_of_succession
- jordigh 10y ago
  A bit, but in general, computing posterior distributions tends to very quickly lead to more complicated integrals. It just so happens that the Beta distribution is somewhat nice and symmetrical, if a bit obscure.
  - XFrequentist 10y ago
    More generally, this is why there's (still) a mild obsession with "conjugate" prior/likelihood distribution pairs - i.e. combinations of prior and likelihood that give analytically tractable posteriors[1] - despite the ability to get easily get results with MCMC. [1] Yes, everyone wants a nice posterior. You're very clever.
eli_gottlieb 10y ago
>Now that we have lots of data and lots of computing power, Bayesian stats can show its results, after getting rebranded as "machine learning". Partly lots of computing power, and partly the Monte Carlo revolution that enabled us to replace intractable integrals with lots of cheap CPU time.