3 ms·

I'm not sure what your example is supposed to prove. Besides the fact that your likelihood function isn't an "unbiased algorithm" like your original comment was

I'm not sure what your example is supposed to prove. Besides the fact that your likelihood function isn't an "unbiased algorithm" like your original comment was talking about, it assumes the bias of the input is known and accepted rather than heavily contested. We certainly shouldn't mandate specific outcomes. We should analyze processes, outcomes, and inputs alike for bias.

yummyfajitas 11y ago
The algorithm would be Bayesian inference using the aforementioned likelihood. The example is a simple illustration of how to fix bias. If bias is unknown then treat it as another unknown variable and use bayesian inference to identify it. If this is unfamiliar, I'm actually publishing a blog post about it tomorrow (how to find the bias in your phone's compass) which you might find helpful.
- pron 11y ago
  Yes, I used that when I worked on sensor fusion algorithms and we had to overcome both random and bias errors in measurements provided by radars. In this case, however, your math is wrong. Radars and compasses don't change the actual position of aircraft or the north, but social predictions really do change future real outcomes (we know that thanks to research in history, social psychology and sociology). The system is dynamic, not static, and has a very strong feedback[1]. Neglecting that feedback makes you miss other stable states; or, in other words: gives you a wrong answer. Just to show you how tricky that feedback can be, I can tell you that in our tracking algorithms we had to carefully track the error's growth with time, as well as allow for unpredictable change in the model (how the aircraft behaves). But if your system has self-reinforcing bias -- i.e. the feedback is stabilizing (as in my example of the {-1, 1} dynamic system) -- you can neglect either of those things and your correct predictions would lead you to the conclusion that your model is actually simpler than the radar case, when, in fact it is far more complicated. Our mistakes are rarely arithmetic errors when applying formulas. They are almost always in the hidden (or not) assumptions we make about the model, and in how we interpret the results[2]. In the case of 99.9% of social statistics, the results mean, "this is the likely outcome provided nothing changes, including not using this result to take any action". Interpreting the results any differently would be making assumptions completely unjustified by math. ---- [1]: Trying to correct biases in sensor measurement also has a feedback effect on future measurements, but it is a predictable and simple feedback that we can usually fix (with gradient techniques as we know the derivatives, though drift may still occur) but that is still not at all the kind of feedback we have in social systems. [2]: Of course, it is often the same thing: how we interpret the result of any mathematical statement depends on the assumptions we've made to arrive at it. In order to apply a proven theorem you must make sure that its assumptions are held in your case. If you've ever used any mechanical provers like proof assistants, you know how unforgiving they are in that regard; you must spend as much care and effort on the assumptions as on the proof.
- yummyfajitas 11y ago
  Published the aforementioned post, which illustrates exactly how biased measurements are just a stats problem: https://www.chrisstucchio.com/blog/2016/bayesian_calibration_of_mobile_phone_compass.html https://www.chrisstucchio.com/blog/2016/bayesian_calibration...

yummyfajitas 11y ago

The algorithm would be Bayesian inference using the aforementioned likelihood. The example is a simple illustration of how to fix bias. If bias is unknown then treat it as another unknown variable and use bayesian inference to identify it. If this is unfamiliar, I'm actually publishing a blog post about it tomorrow (how to find the bias in your phone's compass) which you might find helpful.

pron 11y ago
Yes, I used that when I worked on sensor fusion algorithms and we had to overcome both random and bias errors in measurements provided by radars. In this case, however, your math is wrong. Radars and compasses don't change the actual position of aircraft or the north, but social predictions really do change future real outcomes (we know that thanks to research in history, social psychology and sociology). The system is dynamic, not static, and has a very strong feedback[1]. Neglecting that feedback makes you miss other stable states; or, in other words: gives you a wrong answer. Just to show you how tricky that feedback can be, I can tell you that in our tracking algorithms we had to carefully track the error's growth with time, as well as allow for unpredictable change in the model (how the aircraft behaves). But if your system has self-reinforcing bias -- i.e. the feedback is stabilizing (as in my example of the {-1, 1} dynamic system) -- you can neglect either of those things and your correct predictions would lead you to the conclusion that your model is actually simpler than the radar case, when, in fact it is far more complicated. Our mistakes are rarely arithmetic errors when applying formulas. They are almost always in the hidden (or not) assumptions we make about the model, and in how we interpret the results[2]. In the case of 99.9% of social statistics, the results mean, "this is the likely outcome provided nothing changes, including not using this result to take any action". Interpreting the results any differently would be making assumptions completely unjustified by math. ---- [1]: Trying to correct biases in sensor measurement also has a feedback effect on future measurements, but it is a predictable and simple feedback that we can usually fix (with gradient techniques as we know the derivatives, though drift may still occur) but that is still not at all the kind of feedback we have in social systems. [2]: Of course, it is often the same thing: how we interpret the result of any mathematical statement depends on the assumptions we've made to arrive at it. In order to apply a proven theorem you must make sure that its assumptions are held in your case. If you've ever used any mechanical provers like proof assistants, you know how unforgiving they are in that regard; you must spend as much care and effort on the assumptions as on the proof.
yummyfajitas 11y ago
Published the aforementioned post, which illustrates exactly how biased measurements are just a stats problem: https://www.chrisstucchio.com/blog/2016/bayesian_calibration_of_mobile_phone_compass.html https://www.chrisstucchio.com/blog/2016/bayesian_calibration...