5 ms·

I get that you're probably trying to leave it as an exercise to the reader, but the word "frequentist" doesn't even appear on the wikipedia article for Decision

I get that you're probably trying to leave it as an exercise to the reader, but the word "frequentist" doesn't even appear on the wikipedia article for Decision Theory, nor the Stanford Encyclopedia of Philosophy. I can't tell if you're being hyperbolic, or if it requires deep study to grasp how bayesian and frequentist statistics are rendered irrelevant by decision theory. Anyone care to take a stab at a layman's summary for the benefit of the under-educated folks around here?

James_Henry 7y ago
Googling <statistical decision theory> might help you out. My opinion is the Wikipedia page for decision theory could use a good rewrite. Someday I may get around to it. Also, madhadron didn't quite say that Bayesian and frequentist statistics are rendered irrelevant by decision theory. Rather, statistical decision theory includes Bayesian and frequentist statistics as possible statistical decision rules. You might want to not use Bayesian or frequentist rules and instead use minimax regret or something else. I do think that it is fine that people argue about Bayes vs frequentist. I wish they'd consider everything else though.
- jbay808 7y ago
  >> You might want to not use Bayesian or frequentist rules and instead use minimax regret or something else. ... What? Any such decision algorithm at least in theory still takes a probability distribution as an input. You still need to approximately follow the proper rules of probability to get that distribution, there's no way around it.
  - madhadron 7y ago
    How do minimax criteria not follow the"proper rules of probability"?
    - James_Henry 7y ago
      Or minimax regret
  - James_Henry 7y ago
    You don't need to have a distribution to make a decision. I don't think I understand what you are saying.
    - jbay808 7y ago
      Sorry for the super late reply. I don't know if anyone will see this... But... In minimax regret, you have a set of available decisions D, and a set of possible states of nature N, and a utility U(D,N). Each state of nature also has a probability P(N) (which can be influenced by the decision too in some problems). States of nature include "interest rates rise 1%", "interest rates fall 1%", and "interest rates stay the same". Decisions include "invest in stocks" and "invest in bonds". Minimax regret proposes to ignore the probabilities P(N), instead suggesting a way to make a decision purely based on the utilities of the outcomes. But that is actually an illusion. Outside of math class word problems, we don't have N or U(D,N) handed to us on a silver platter. There is always an infinite range of possible states of nature, many of which have a probability approaching but never reaching zero, including states such as "win the lottery", "communist revolution", and "unexpected intergalactic nuclear war". In commonsense decision-making we don't include those states of nature in our decision matrix, because our common sense rules them out as being implausible before we even think about our options. You wouldn't choose to invest in bonds just because stocks have the most regret in the event of a communist takeover. So what actually happens is we intuitively apply some probability threshold that rules out states of nature falling below it from our consideration. Then we minimize max regret on the remaining "plausibly realistic" states of nature. Humans are so good at doing probability mentally that this step happens before we even realize it. But if you are writing code that makes decisions, you'll need to do it, and so you'll need to have at least a rough stab at the probability distributions.
madhadron 7y ago
The sibling gave some pointers. It's not that they're irrelevant. It's that they're historically important special cases, and decision theory provides the framework for thinking about when they make sense. The general setup is this: you have a set of possible states of nature, a random variable on that set which will produce some observations, a set of decisions you may make, and a loss function defined over tuples of (state of nature, decision). The task is to produce a "good" function that maps from the observations to a decision. It turns out that "good" isn't unique. You start by defining the expected loss for a given state of nature and decision function. Then you generally narrow your decision functions of interest to those for which there is no other function that has a lower expected loss for all states of nature (we call such ones "inadmissible" and ones that aren't uniformly dominated like this "admissible"). But then you end up with a whole collection of decision functions that are better under different conditions, and you need a way of choosing among them. Unbiasedness is one such criterion, as is maximum likelihood (though in many situations the maximum likelihood criterion may single out decision functions that are inadmissible), Bayes under a given prior, or minimax. For some of these, particularly minimax, you also run into the fact that there aren't "enough" deterministic decision functions to select a best one...but if you allow randomized decision functions, there are. Bayes procedures also enter in an interesting way: the class of Bayes procedures over all sensible priors is a superset of the set of admissible procedures...but usually not a much bigger set, so you can sometimes prove properties of all admissible procedures by proving them for all Bayes procedures. For those who have studied game theory, this also leads to an immediate extension of repeated procedures where the decision rule at each step is "stop and accept a decision" or "sample again." These sometimes go under the name of "online trials" and are really useful. And since any admissible procedure is Bayes under some prior, you can often get away with doing the math for these online trials in a Bayesian framework, adding a hard coded stopping rule to correspond to your underlying decision procedure, even if you are a died in the wool frequentist. The other, really important result in Bayesian statistics is that for a well behaved prior, data eventually drowns it out. For my part, I don't take Bayesian statistics very seriously as a philosophical point of view. In the 1960's there was a lot of interest in it as a unified system of inference because there was a theorem showing that rational actors with a prior are Bayesian. When you start really digging into Bayesian statistics, though, you realize that a single prior is overspecified, and the more correct class of objects to work with is classes of priors. As soon as you do that, though, the rational actor is Bayesian theorem collapses. I don't take the frequentist approach that seriously, either. Honestly, this is a space where I don't feel that there is a truly satisfactory underpinning besides decision theory plus "shut up and calculate." But that's true of so much of science that I'm not that bothered by it anymore.

James_Henry 7y ago

Googling <statistical decision theory> might help you out. My opinion is the Wikipedia page for decision theory could use a good rewrite. Someday I may get around to it. Also, madhadron didn't quite say that Bayesian and frequentist statistics are rendered irrelevant by decision theory. Rather, statistical decision theory includes Bayesian and frequentist statistics as possible statistical decision rules. You might want to not use Bayesian or frequentist rules and instead use minimax regret or something else. I do think that it is fine that people argue about Bayes vs frequentist. I wish they'd consider everything else though.

jbay808 7y ago
>> You might want to not use Bayesian or frequentist rules and instead use minimax regret or something else. ... What? Any such decision algorithm at least in theory still takes a probability distribution as an input. You still need to approximately follow the proper rules of probability to get that distribution, there's no way around it.
- madhadron 7y ago
  How do minimax criteria not follow the"proper rules of probability"?
  - James_Henry 7y ago
    Or minimax regret
- James_Henry 7y ago
  You don't need to have a distribution to make a decision. I don't think I understand what you are saying.
  - jbay808 7y ago
    Sorry for the super late reply. I don't know if anyone will see this... But... In minimax regret, you have a set of available decisions D, and a set of possible states of nature N, and a utility U(D,N). Each state of nature also has a probability P(N) (which can be influenced by the decision too in some problems). States of nature include "interest rates rise 1%", "interest rates fall 1%", and "interest rates stay the same". Decisions include "invest in stocks" and "invest in bonds". Minimax regret proposes to ignore the probabilities P(N), instead suggesting a way to make a decision purely based on the utilities of the outcomes. But that is actually an illusion. Outside of math class word problems, we don't have N or U(D,N) handed to us on a silver platter. There is always an infinite range of possible states of nature, many of which have a probability approaching but never reaching zero, including states such as "win the lottery", "communist revolution", and "unexpected intergalactic nuclear war". In commonsense decision-making we don't include those states of nature in our decision matrix, because our common sense rules them out as being implausible before we even think about our options. You wouldn't choose to invest in bonds just because stocks have the most regret in the event of a communist takeover. So what actually happens is we intuitively apply some probability threshold that rules out states of nature falling below it from our consideration. Then we minimize max regret on the remaining "plausibly realistic" states of nature. Humans are so good at doing probability mentally that this step happens before we even realize it. But if you are writing code that makes decisions, you'll need to do it, and so you'll need to have at least a rough stab at the probability distributions.

madhadron 7y ago

The sibling gave some pointers. It's not that they're irrelevant. It's that they're historically important special cases, and decision theory provides the framework for thinking about when they make sense. The general setup is this: you have a set of possible states of nature, a random variable on that set which will produce some observations, a set of decisions you may make, and a loss function defined over tuples of (state of nature, decision). The task is to produce a "good" function that maps from the observations to a decision. It turns out that "good" isn't unique. You start by defining the expected loss for a given state of nature and decision function. Then you generally narrow your decision functions of interest to those for which there is no other function that has a lower expected loss for all states of nature (we call such ones "inadmissible" and ones that aren't uniformly dominated like this "admissible"). But then you end up with a whole collection of decision functions that are better under different conditions, and you need a way of choosing among them. Unbiasedness is one such criterion, as is maximum likelihood (though in many situations the maximum likelihood criterion may single out decision functions that are inadmissible), Bayes under a given prior, or minimax. For some of these, particularly minimax, you also run into the fact that there aren't "enough" deterministic decision functions to select a best one...but if you allow randomized decision functions, there are. Bayes procedures also enter in an interesting way: the class of Bayes procedures over all sensible priors is a superset of the set of admissible procedures...but usually not a much bigger set, so you can sometimes prove properties of all admissible procedures by proving them for all Bayes procedures. For those who have studied game theory, this also leads to an immediate extension of repeated procedures where the decision rule at each step is "stop and accept a decision" or "sample again." These sometimes go under the name of "online trials" and are really useful. And since any admissible procedure is Bayes under some prior, you can often get away with doing the math for these online trials in a Bayesian framework, adding a hard coded stopping rule to correspond to your underlying decision procedure, even if you are a died in the wool frequentist. The other, really important result in Bayesian statistics is that for a well behaved prior, data eventually drowns it out. For my part, I don't take Bayesian statistics very seriously as a philosophical point of view. In the 1960's there was a lot of interest in it as a unified system of inference because there was a theorem showing that rational actors with a prior are Bayesian. When you start really digging into Bayesian statistics, though, you realize that a single prior is overspecified, and the more correct class of objects to work with is classes of priors. As soon as you do that, though, the rational actor is Bayesian theorem collapses. I don't take the frequentist approach that seriously, either. Honestly, this is a space where I don't feel that there is a truly satisfactory underpinning besides decision theory plus "shut up and calculate." But that's true of so much of science that I'm not that bothered by it anymore.