Let me take a stab at answering. And I'll answer 2 questions, not 1. 1. Why do we need Bayesian estimation? Can't we just all use MLE, and live in harmony? If

Let me take a stab at answering. And I'll answer 2 questions, not 1. 1. Why do we need Bayesian estimation? Can't we just all use MLE, and live in harmony? If MLE was good for Gauss and Laplace and Fisher, why isn't it good for me? Answer: for many problems the likelihood function is unbound. In fact the cases where the likelihood function are bounded, so the MLE makes sense, are few and far between. But if you think of the likelihood as a function with a hump somewhere, you have several choices for a single number that best describes the function: mode, mean, median for example. MLE corresponds to mode, can we use the mean instead? Also, does it not even make better sense to use mean rather than mode? Sure it does, and if you use mean you become a Bayesian. But can the likelihood be unbounded and have a finite mean, you ask? Yes, it happens quite often, sufficiently often that in practice people don't worry about that. Once you start using mean instead of mode (and therefore you can proudly call yourself a Bayesian), you get a number of side benefits: you get parameter uncertainty estimation for free, and you can start multiplying your likelihoods with various functions that make your life easier. You can interpret those functions as regularization functions, or prior, you can even start talking about expert knowledge, and delve into philosophical meanderings. No need to do that , there are enough people on the internet who do that. 2. Why Monte Carlo? Because calculating means of high-dimensional distributions is not easy. In most cases you are hit with the curse of dimensionality. Separately, the distributions that you work with are not known only up to a multiplicative constant. So the problem is how do you estimate the mean of a vector (v1, v2, ...., vn) when you know its distribution, but only up to a constant? The answer is Markov Chain Monte Carlo (MCMC) and if you learn that, you get special powers. So why do Bayesians need Monte Carlo? Because they like to be like super-heroes and have special powers.