5 ms·

Just out of curiosity, what kinds of statistical problems do you tend to solve, or are you just a fanboy? I saw you posted a link to something on lesswrong, wh

Just out of curiosity, what kinds of statistical problems do you tend to solve, or are you just a fanboy? I saw you posted a link to something on lesswrong, which is confusing an application of Bayes Rule with Bayesian statistics. Maybe you should read what an actual statistician has to say on the matter: http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesianfrequentist-inference/ http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesi... instead of a Harry Potter fanfic author.

loup-vaillant 13y ago
Fanboy. I have read half of E.T. Jaynes' Probability Theory: the Logic of Science so far. I have read the link you speak of. I have already responded here: http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesianfrequentist-inference/#comment-2793 http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesi... I don't think my LessWrong link confuses Bayes' Rule with Bayesian statistics. Why do you think Eliezer responded 1/2 to the brain teaser? He does not say it, but I'm pretty sure he just assumed that a mathematician who has 2 boys is twice as likely to spontaneously say "I have at least one boy", compared to a mathematician who has only one boy. The disagreement between his inference (which he did not know was "Bayesian" at the time) and "Orthodox" statistics didn't come from Bayes' Rule. It came from the use of a non-uniform prior to begin with. Which Frequentism rejects, because it's "subjective". So, instead of using this highly relevant prior information, it just uses a uniform prior. (By the way, this is nuts. A scientist should never throw away relevant information.) Both methods then use Bayes' Rule. They just start with different priors: best guess and objective-looking, respectively. (I'm still wondering why anyone would use antything but one's best guess.)
- sanskritabelt 13y ago
  So you're basically just Monday morning quarterbacking here. It's kind of telling you didn't understand Wasserman's reply to you, it was perfectly clear to any practitioner. Here's an example of a real-world problem I had to solve in my last job. We wanted to compare two treatments on an object, OLD and NEW, and evaluate which was better. We got a bunch of volunteers and showed them each a random sample of objects, the OLD treatment, and the NEW treatment (suitably blinded of course) and asked them to rate each on a scale of 1-5. The goal was to determine if the OLD treatment should be replaced by the NEW treatment. What's your Bayesian approach to solving this? I was the closest thing to a domain expert, and I certainly didn't have any prior beliefs about what the ratings would be (except for a weak expectation that OLD>NEW). Tell me the tests you would have run, and then I'll tell you the non-Bayesian thing I did that gave a very solid answer and took well under a second of computation.
  - loup-vaillant 13y ago
    > It's kind of telling you didn't understand Wasserman's reply to you, it was perfectly clear to any practitioner. Condescension doesn't impress me. Explanations do. Could you please explain his "perfectly clear" reply to me? > What's your Bayesian approach to solving this? Applying probability theory as best I could. If it turns out to be too complicated, or computationally intractable, I'll resort to approximations. For this particular case… Well, first, we don't have 2 treatments, we have as many treatments as we have objects. It's like flipping a coin. When you flip a coin in the obvious way, you have no idea which side will eventually come up, because even if you know it came up heads the previous time, and you tried to perform the same move again: in fact, you don't perform the same move, and you don't know how exactly your second move differed from the first. So first, there is an unpredictable variability in the way the treatment will be performed. This is the first source of uncertainty. Second, the objects aren't all the same. I expect them to be more or less clean to begin with, and other characteristics such as sharp angles may more or less hinder the treatment. This is the second source of uncertainty. Once you perform a treatment on an object, it had a definite effect. Unfortunately, it is hard to define how well it actually went. Maybe you can come up with a definite criterion, but apparently, since you needed to ask a sample of human volunteers, that criterion is either not well understood, or hard to measure properly. Anyway, you have access to an uncertain measure, which in this case is a human giving you a scale from 1 to 5. This is the third and fourth sources of uncertainty (akin to the first and second one respectively: variability in how a given human will assess an object; and variability across humans). From what I have understood of what you told me, there were 2 sets of objects, and 2 groups of people. One group looked at every object before any treatment. Then you applied OLD to half objects, and NEW to half the other. You then showed the newly cleaned up objects to the second group. Of course, you don't tell the group which objects they get to examine. An alternative would be to photograph every objects before and after the treatment, then show both sets of photos to everyone, and ask them to rank the objects (both before and after), and the perceived efficiency of the treatment. I don't like it much however, because photos add an additional source of uncertainty. This looks like a complicated version of a compound estimation problem, described in chapter 6 of Jayne's PT:TLOS (elementary parameter estimation). Pretty basic stuff. We're not finished yet. You wanted to know which treatment should be chosen: OLD or NEW. You basically have 3 alternatives: always apply OLD, always apply NEW, or look at the object then decide which treatment you apply. For each alternative, you should have a probability distribution over the "cleanness" distribution of the objects. Decide a utility function for the cleanness distribution, and chose the method that maximizes expected utility. And now I'm stuck because I don't know probability theory deeply enough to actually give you a non-ambiguous procedure in less than a couple days. Not to mention that I don't have your prior information about OLD and NEW (why you expect OLD to be better etc.) Anyway, I believe a specialist would only need a couple minutes, or a couple hours tops. (To find the procedure, that is. Actually running the numbers in a computer may or may not be expensive. I personally have no idea.)

loup-vaillant 13y ago

Fanboy. I have read half of E.T. Jaynes' Probability Theory: the Logic of Science so far. I have read the link you speak of. I have already responded here: http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesianfrequentist-inference/#comment-2793 http://normaldeviate.wordpress.com/2012/11/17/what-is-bayesi... I don't think my LessWrong link confuses Bayes' Rule with Bayesian statistics. Why do you think Eliezer responded 1/2 to the brain teaser? He does not say it, but I'm pretty sure he just assumed that a mathematician who has 2 boys is twice as likely to spontaneously say "I have at least one boy", compared to a mathematician who has only one boy. The disagreement between his inference (which he did not know was "Bayesian" at the time) and "Orthodox" statistics didn't come from Bayes' Rule. It came from the use of a non-uniform prior to begin with. Which Frequentism rejects, because it's "subjective". So, instead of using this highly relevant prior information, it just uses a uniform prior. (By the way, this is nuts. A scientist should never throw away relevant information.) Both methods then use Bayes' Rule. They just start with different priors: best guess and objective-looking, respectively. (I'm still wondering why anyone would use antything but one's best guess.)

sanskritabelt 13y ago
So you're basically just Monday morning quarterbacking here. It's kind of telling you didn't understand Wasserman's reply to you, it was perfectly clear to any practitioner. Here's an example of a real-world problem I had to solve in my last job. We wanted to compare two treatments on an object, OLD and NEW, and evaluate which was better. We got a bunch of volunteers and showed them each a random sample of objects, the OLD treatment, and the NEW treatment (suitably blinded of course) and asked them to rate each on a scale of 1-5. The goal was to determine if the OLD treatment should be replaced by the NEW treatment. What's your Bayesian approach to solving this? I was the closest thing to a domain expert, and I certainly didn't have any prior beliefs about what the ratings would be (except for a weak expectation that OLD>NEW). Tell me the tests you would have run, and then I'll tell you the non-Bayesian thing I did that gave a very solid answer and took well under a second of computation.
- loup-vaillant 13y ago
  > It's kind of telling you didn't understand Wasserman's reply to you, it was perfectly clear to any practitioner. Condescension doesn't impress me. Explanations do. Could you please explain his "perfectly clear" reply to me? > What's your Bayesian approach to solving this? Applying probability theory as best I could. If it turns out to be too complicated, or computationally intractable, I'll resort to approximations. For this particular case… Well, first, we don't have 2 treatments, we have as many treatments as we have objects. It's like flipping a coin. When you flip a coin in the obvious way, you have no idea which side will eventually come up, because even if you know it came up heads the previous time, and you tried to perform the same move again: in fact, you don't perform the same move, and you don't know how exactly your second move differed from the first. So first, there is an unpredictable variability in the way the treatment will be performed. This is the first source of uncertainty. Second, the objects aren't all the same. I expect them to be more or less clean to begin with, and other characteristics such as sharp angles may more or less hinder the treatment. This is the second source of uncertainty. Once you perform a treatment on an object, it had a definite effect. Unfortunately, it is hard to define how well it actually went. Maybe you can come up with a definite criterion, but apparently, since you needed to ask a sample of human volunteers, that criterion is either not well understood, or hard to measure properly. Anyway, you have access to an uncertain measure, which in this case is a human giving you a scale from 1 to 5. This is the third and fourth sources of uncertainty (akin to the first and second one respectively: variability in how a given human will assess an object; and variability across humans). From what I have understood of what you told me, there were 2 sets of objects, and 2 groups of people. One group looked at every object before any treatment. Then you applied OLD to half objects, and NEW to half the other. You then showed the newly cleaned up objects to the second group. Of course, you don't tell the group which objects they get to examine. An alternative would be to photograph every objects before and after the treatment, then show both sets of photos to everyone, and ask them to rank the objects (both before and after), and the perceived efficiency of the treatment. I don't like it much however, because photos add an additional source of uncertainty. This looks like a complicated version of a compound estimation problem, described in chapter 6 of Jayne's PT:TLOS (elementary parameter estimation). Pretty basic stuff. We're not finished yet. You wanted to know which treatment should be chosen: OLD or NEW. You basically have 3 alternatives: always apply OLD, always apply NEW, or look at the object then decide which treatment you apply. For each alternative, you should have a probability distribution over the "cleanness" distribution of the objects. Decide a utility function for the cleanness distribution, and chose the method that maximizes expected utility. And now I'm stuck because I don't know probability theory deeply enough to actually give you a non-ambiguous procedure in less than a couple days. Not to mention that I don't have your prior information about OLD and NEW (why you expect OLD to be better etc.) Anyway, I believe a specialist would only need a couple minutes, or a couple hours tops. (To find the procedure, that is. Actually running the numbers in a computer may or may not be expensive. I personally have no idea.)