5 ms·

For continuous data it is exactly a probability density evaluated on your data (for discrete it's PMF instead). L(params)=P(D|params)

kgwgk 3y ago
The point is that L(params) is not a probability density. The integral of L(params) over params is not one.
- vermarish 3y ago
  Right. But if you make the notation slightly more explicit, then the integral of L(data, params) over data is 1. This follows from the independence assumption. So we ARE working with a probability function. Its output can be interpreted as probabilities. It's just that we're maximizing L = P(events | params) with respect to params.
  - 331c8c71 3y ago
    The independence has nothing do with the integral being 1 to be honest. You could write a model where the observations are not independent but the (multivariate) integral over their domain will still be 1.
    - vermarish 3y ago
      But for such a model, the joint pdf would not be written simply as a product of each individual pdf. That's what independence provides.
      - kgwgk 3y ago
        If by “joint probability” you mean function(params, data) there is no joint probability here in general. L(params, data) is constructed from a family density functions p(data) for each possible value of param. The integral of L(params, data) may be anything or diverge. You don’t need any extra independence assumption either. Or maybe you mean “joint probability” as p(data1, data2) when data is composed of two observations, for example. But you don’t need any independence assumption for that probability density to integrate to one! It necessarily does that - whether you can factorize it as p’(data1)p’’(data2) or not.
  - kgwgk 3y ago
    The likelihood function is a function of params for a fixed value of data and it is not a probability function. There is another function - a function of data for fixed params - which is a probability density. That doesn’t change the fact that the likelihood function isn’t.
- 331c8c71 3y ago
  That's exactly the reason why frequentist approach sucks by the way;) Parameters are treated specially and there is no internal consistency - to have it you need to introduce priors...
  - 3abiton 3y ago
    It's the bayes vs frequentist war again.

kgwgk 3y ago

The point is that L(params) is not a probability density. The integral of L(params) over params is not one.

vermarish 3y ago
Right. But if you make the notation slightly more explicit, then the integral of L(data, params) over data is 1. This follows from the independence assumption. So we ARE working with a probability function. Its output can be interpreted as probabilities. It's just that we're maximizing L = P(events | params) with respect to params.
- 331c8c71 3y ago
  The independence has nothing do with the integral being 1 to be honest. You could write a model where the observations are not independent but the (multivariate) integral over their domain will still be 1.
  - vermarish 3y ago
    But for such a model, the joint pdf would not be written simply as a product of each individual pdf. That's what independence provides.
    - kgwgk 3y ago
      If by “joint probability” you mean function(params, data) there is no joint probability here in general. L(params, data) is constructed from a family density functions p(data) for each possible value of param. The integral of L(params, data) may be anything or diverge. You don’t need any extra independence assumption either. Or maybe you mean “joint probability” as p(data1, data2) when data is composed of two observations, for example. But you don’t need any independence assumption for that probability density to integrate to one! It necessarily does that - whether you can factorize it as p’(data1)p’’(data2) or not.
- kgwgk 3y ago
  The likelihood function is a function of params for a fixed value of data and it is not a probability function. There is another function - a function of data for fixed params - which is a probability density. That doesn’t change the fact that the likelihood function isn’t.
331c8c71 3y ago
That's exactly the reason why frequentist approach sucks by the way;) Parameters are treated specially and there is no internal consistency - to have it you need to introduce priors...
- 3abiton 3y ago
  It's the bayes vs frequentist war again.