3 ms·
First, model averaging is a bit different than when you describe it as (at least in my reading, but maybe you mean "single probability" differently than I read
by p4wnc6 11y ago
First, model averaging is a bit different than when you describe it as (at least in my reading, but maybe you mean "single probability" differently than I read it):
> even Bayesian model averaging collapses a distribution over probabilities into a single probability
as I mentioned in the child comment to the other reply to this comment, model averaging creates an average value, but the type of that value is a distribution. That is, you have a distribution over distributions (each coming from a different model) and the effect of averaging does not reduce you down to a point estimate, rather it reduces you down to just one distribution.
This is why it's totally fair to say a posterior can be an average. It is the average of a bunch of other distributions. I think if I had said it that way in my first comment, it would have removed some confusion.
But it is important, because the criticism that "a posterior isn't an average" is very wrong. A posterior most certainly can be the average of some other stuff, if that other stuff was itself a bunch of distributions -- and that's exactly what I am trying to talk about.
But to your other point, about the 'sensible' vs. 'not sensible' readings, I mostly agree. However, the problem is who gets to decide when two ideas fall into the "knife-vs-spoon-clearly-exclusive" category, or when it's more gray than that, and the choice is not so black and white, and there is not a need to over-commit to just one approach?
The reason the OP post strikes me as problematic is that it seems like a matter of opinion, or in the worst case a matter of bureaucratic/dictatorial mandate, as to when ideas are of the type that can be averaged vs. when they are not.
I'd generally like people to be more humble about it and tend to believe that conventional wisdom and model averaging are better, at least as a first heuristic, than deeply committing to just one thing. That way there might be less urgency to rush into the claim that some debate is "knife-vs-spoon".
I sort of see the whole "knife-vs-spoon" thing as a kind of Godwin's law of brainstorming. Once you invoke the "knife-vs-spoon-so-we-can't-average" claim, it's like game over and all useful intellectual discussion dies and everyone just either picks Team Knife or Team Spoon and then the political battles start. Unless it's really mutually exclusive, I'd rather that doesn't happen.