I think the correct answer is that it is mostly bogus, but likelihood based statistical methods mostly work for well-behaving distributions, especially for Gaus

I think the correct answer is that it is mostly bogus, but likelihood based statistical methods mostly work for well-behaving distributions, especially for Gaussian. Maximum likelihood estimation has some weird cases when the distribution is not "well behaving".