3 ms·

Yeah I suppose in the context of MLE it makes more sense to talk about your choice of variables. Which does matter unfortunately (which is kind of obvious when

Yeah I suppose in the context of MLE it makes more sense to talk about your choice of variables. Which does matter unfortunately (which is kind of obvious when you note that you can convert any distribution into any other by chaging its coordinates [*]). Using a prior gives you an 'out' by picking the Radon-Nikodym derivative w.r.t. that prior, since this definition of probability density is independent of your choice of variables. In most applications of MLE people implicitly use the (improper) uniform prior in which case you end up with the usual density. However this is usually done without justification, which is a bit dangerous. [*]: For a rather extreme example consider that if X is exponentially distributed with mean 1 then so is -log(1 - e^-X) which you get by using the CDF, flipping the distribution and then using the inverse CDF, this transformation swaps 0 (the mode) with positive infinity.

radford-neal 5y ago
No. Maximum likelihood estimation does not involve a prior over parameters, and the MLE does not depend on how the parameters are represented, nor does it depend on the way the variables are represented. For example, the maximum likelihood estimate for a variance parameter is just the square of the maximum likelihood estimate for the corresponding standard deviation parameter, as you'd expect. And the maximum likelihood estimate for the parameters of a log-normal distribution for positive data are related in the obvious way to the maximum likelihood estimate for the parameters of a normal distribution for the logs of those data points. Doing a 1-to-1 continuous transformation of the data causes the probability density (for any given parameter values) to be multiplied by a Jacobian factor, but this factor depends only on the data values, NOT on the probability density (determined by the parameters), so the MLE for the transformed data is the same as for the untransformed data.
- whimsicalism 5y ago
  Agreed - great reply.

radford-neal 5y ago

No. Maximum likelihood estimation does not involve a prior over parameters, and the MLE does not depend on how the parameters are represented, nor does it depend on the way the variables are represented. For example, the maximum likelihood estimate for a variance parameter is just the square of the maximum likelihood estimate for the corresponding standard deviation parameter, as you'd expect. And the maximum likelihood estimate for the parameters of a log-normal distribution for positive data are related in the obvious way to the maximum likelihood estimate for the parameters of a normal distribution for the logs of those data points. Doing a 1-to-1 continuous transformation of the data causes the probability density (for any given parameter values) to be multiplied by a Jacobian factor, but this factor depends only on the data values, NOT on the probability density (determined by the parameters), so the MLE for the transformed data is the same as for the untransformed data.

whimsicalism 5y ago
Agreed - great reply.