3 ms·

> I don't understand how we can reason with "probabilities of failure" about such fundamental things such as laws of physics. A sketch: Suppose we have two the

by tbabb 7y ago

> I don't understand how we can reason with "probabilities of failure" about such fundamental things such as laws of physics. A sketch: Suppose we have two theories, one ("A") with assumptions (a, b); the other ("B") with assumptions (a, p, q, r, s, t); and our evidence 'e'. Pr(A) ~= Pr(a|e) * Pr(b|e) Pr(B) ~= Pr(a|e) * Pr(p|e) * Pr(q|e) * Pr(r|e) * Pr(s|e) * Pr(t|e) That is, the probability of each theory being right is the probability that each of its assumptions are simultaneously true. It might be difficult to come up with a specific number for each of those component terms, but we'd do well to estimate the total probability of each theory simply by counting the terms, since in the limit that will matter more than the probability of each term (assuming we think none of them are obviously low). Also note that when counting, we can trivially factor out the common assumptions "a" (a.k.a. the things we don't wish to doubt or vary between theories, e.g. QM in "ordinary" regimes, GR, the Newtonian approximation to GR, etc.) Yes, there is a "ground truth" theory which is absolutely true or false, but we don't have access to it. And I don't see how it's more problematic to use probability here than on any other classical event on which we have imperfect information, like a specific dice roll, or a baseball game, or the outcome of an election— One specific thing will happen, and no other outcome was possible, but we can still use probability to model our incomplete knowledge. How is physics different? Example stress tests of this idea: e = the motion of the planets; A := "G = m1*m2/r^2"; B := [long list of epicycle parameters] --> pick "A". e = the varied appearance/adaptations of animals/species; A := [reproduction, inheritance, variation, selection]; B := foreach animal x {"God zotted $x into existence like that because just_so_story($x)" --> pick "A". e = my empty garage; A := "there's nothing in it"; B := "there's a dragon in it; the dragon is invisible; the dragon dodges your touch; the dragon has no heat signature; the dragon floats and leaves no footprints; ..." --> pick "A". e = the behavior of the universe; A := [the standard model]; B := [the standard model; also it's a simulation; there are intelligent beings who set up the simulation; there is an external universe in which the simulation is occurring; the number of simulations happening in this universe is large; ...] --> pick "A".

vladf 7y ago
I think that you may have missed my point. First off, why are assumptions independent? Why are you allowed to factor p(a|e) * p(b|e) = p(a,b|e)? Assumptions aren't just independent binary variables -- it's not necessarily true that you can have some product measure over the set of assumptions (a, b, p, q, r, s, t) simultaneously. More importantly, you dived write into some notation (Pr) without telling me what it _means_, which is what my OP was about. > And I don't see how it's more problematic to use probability here than on any other classical event on which we have imperfect information, like a specific dice roll, or a baseball game, or the outcome of an election— One specific thing will happen, and no other outcome was possible, but we can still use probability to model our incomplete knowledge. How is physics different? Here's a crucial difference. We can roll a die 100 times, 1000 times, 10K times, and the frequency of a six landing as we increase the number of trials will tend to 1/6. That's what we mean (if we're frequentists) when we say the probability of a six landing is 1/6. We can't "roll" universes with physics models.
- tbabb 7y ago
  > First off, why are assumptions independent? Because I've defined them that way. I mean them to be independent choices you could make when designing your model that could be varied to fit the data. If two aspects of the model are not independent; i.e. they are covariant in some way, then there is some common parameter that explains them both, and that parameter is the one that should be seen as an input to the model. > We can roll a die 100 times, 1000 times, 10K times [...] That's what we mean (if we're frequentists) We're not frequentists. You can't "re-roll" the 2016 election 10K times, either. There was only one, and there was only one way it could come out; we just didn't know enough to say what it would be before it happened. All the particles in all the voters were obeying the laws of physics at every moment; never was there any freedom for a different outcome. Nonetheless, even though there was/is only one "ground truth" that could ever be, we assigned probabilities to each possible outcome, given our incomplete knowledge. This is a pretty standard application of probability. State estimators (e.g. the Kalman filter) are doing the same thing— you have some noisy readings of reality, and you use Bayesian logic on some assumed probability distributions to pick the estimate from the space of possible "ground truths" that has the highest probability of being the right one. Concretely: I'm measuring roll rate, local acceleration, compass heading, barometric pressure, and GPS, all with significant error, and I want to know where my quadcopter is most likely to be at the current moment. There is only one true answer to that question, the quadcopter is in one place, not 10,000 places (or 10,000 flights), there is a single ground truth. But Bayes will give me a probability, given my readings, that any given estimate is the true ground truth (and some math will help me solve for the highest one). In this case, instead of assigning probabilities to possible election outcomes or system state "ground truths", the "configuration space" is models of reality. But all we've changed is the domain of our probability distribution; the math doesn't care what kind of thing our "ground truth" represents. And it doesn't matter if reality contains only one "ground truth" or many; the fact is that we are choosing between many options (and we are ranking them by likelihood).
  - vladf 7y ago
    Re independence: identifying whether or not these physical assumptions covary is not that easy. That's my point: assumptions a, b, c, d could easily have some mutual incompatibility that makes them non-independent. It's an active area of research. Re probability, I'm glad you committed to the Bayesian interpretation. Bayes gives you a _degree of belief_, based on your priors. It's quite fortuitous that you mention the 2016 election. As you say, there's only one instance here. Which is why the prior matters a lot. We can incorporate (partial) evidence from past elections, but it's going to be very sensitive to the priors that we place, since the net amount of evidence we're working with is very small. As we found out in 2016, that means these beliefs aren't worth much in such low data scenarios, since the prior has a large impact! https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/ This brings me to my original point: > In the limit of evidence, this prior matters less, but constants matter here! How much evidence do we need before we can be confident the "belief levels" we're throwing around aren't that subjective anymore? We don't really have a good sense for what the structure over this space of "assumptions of physical models" is, so we can't really answer this question.

by tbabb 7y ago

vladf 7y ago

I think that you may have missed my point. First off, why are assumptions independent? Why are you allowed to factor p(a|e) * p(b|e) = p(a,b|e)? Assumptions aren't just independent binary variables -- it's not necessarily true that you can have some product measure over the set of assumptions (a, b, p, q, r, s, t) simultaneously. More importantly, you dived write into some notation (Pr) without telling me what it _means_, which is what my OP was about. > And I don't see how it's more problematic to use probability here than on any other classical event on which we have imperfect information, like a specific dice roll, or a baseball game, or the outcome of an election— One specific thing will happen, and no other outcome was possible, but we can still use probability to model our incomplete knowledge. How is physics different? Here's a crucial difference. We can roll a die 100 times, 1000 times, 10K times, and the frequency of a six landing as we increase the number of trials will tend to 1/6. That's what we mean (if we're frequentists) when we say the probability of a six landing is 1/6. We can't "roll" universes with physics models.

tbabb 7y ago
> First off, why are assumptions independent? Because I've defined them that way. I mean them to be independent choices you could make when designing your model that could be varied to fit the data. If two aspects of the model are not independent; i.e. they are covariant in some way, then there is some common parameter that explains them both, and that parameter is the one that should be seen as an input to the model. > We can roll a die 100 times, 1000 times, 10K times [...] That's what we mean (if we're frequentists) We're not frequentists. You can't "re-roll" the 2016 election 10K times, either. There was only one, and there was only one way it could come out; we just didn't know enough to say what it would be before it happened. All the particles in all the voters were obeying the laws of physics at every moment; never was there any freedom for a different outcome. Nonetheless, even though there was/is only one "ground truth" that could ever be, we assigned probabilities to each possible outcome, given our incomplete knowledge. This is a pretty standard application of probability. State estimators (e.g. the Kalman filter) are doing the same thing— you have some noisy readings of reality, and you use Bayesian logic on some assumed probability distributions to pick the estimate from the space of possible "ground truths" that has the highest probability of being the right one. Concretely: I'm measuring roll rate, local acceleration, compass heading, barometric pressure, and GPS, all with significant error, and I want to know where my quadcopter is most likely to be at the current moment. There is only one true answer to that question, the quadcopter is in one place, not 10,000 places (or 10,000 flights), there is a single ground truth. But Bayes will give me a probability, given my readings, that any given estimate is the true ground truth (and some math will help me solve for the highest one). In this case, instead of assigning probabilities to possible election outcomes or system state "ground truths", the "configuration space" is models of reality. But all we've changed is the domain of our probability distribution; the math doesn't care what kind of thing our "ground truth" represents. And it doesn't matter if reality contains only one "ground truth" or many; the fact is that we are choosing between many options (and we are ranking them by likelihood).
- vladf 7y ago
  Re independence: identifying whether or not these physical assumptions covary is not that easy. That's my point: assumptions a, b, c, d could easily have some mutual incompatibility that makes them non-independent. It's an active area of research. Re probability, I'm glad you committed to the Bayesian interpretation. Bayes gives you a _degree of belief_, based on your priors. It's quite fortuitous that you mention the 2016 election. As you say, there's only one instance here. Which is why the prior matters a lot. We can incorporate (partial) evidence from past elections, but it's going to be very sensitive to the priors that we place, since the net amount of evidence we're working with is very small. As we found out in 2016, that means these beliefs aren't worth much in such low data scenarios, since the prior has a large impact! https://projects.fivethirtyeight.com/2016-election-forecast/ https://projects.fivethirtyeight.com/2016-election-forecast/ This brings me to my original point: > In the limit of evidence, this prior matters less, but constants matter here! How much evidence do we need before we can be confident the "belief levels" we're throwing around aren't that subjective anymore? We don't really have a good sense for what the structure over this space of "assumptions of physical models" is, so we can't really answer this question.