This is an area where Bayesian methods really come to the fore - dealing with non-normal, even non-parametric, distributions. That’s not to say they can’t be tr

This is an area where Bayesian methods really come to the fore - dealing with non-normal, even non-parametric, distributions. That’s not to say they can’t be treated in frequentists stats. I’ve never read that book but perhaps it stops before dealing with how to handle all that stuff. Regarding your specific point about how do you quantify error regarding your chosen prior? The answer in a Bayesian framework lies in things like credibility intervals, posterior distributions, posterior predictive distributions, depending on what you’re actually trying to quantify. Very long story short, the latter allows you to sample from a distribution of predictions - for which you can then form a variety of error estimates (and the distribution doesn’t have to be parametric). That will quantify uncertainty for the model you’ve made using the prior you chose (and the data). But usually we wouldn’t recommend changing your prior just to get lower uncertainty unless you have some rationality behind changing the prior. Priors should really be chosen based on domain knowledge etc etc etc.