3 ms·
> understanding the nuances between words like probability and likelihood Go into a lab, do an experiment, call that one trial, measure a number, call that num
by graycat 10y ago
> understanding the nuances between words
like probability and likelihood
Go into a lab, do an experiment, call that
one trial, measure a number, call that
number (the value at this trial of)
random variable X.
We might want the average value, expected
value, or expectation of X denoted by
E[X].
Under meager assumptions, if we take a
sequence of independent samples of X, then
their average will converge to E[X]; this
is the law of large numbers.
We might be interested in the event,
call it A, when X > 1.
We might want the probability of A, that
is, P(A) = P(X > 1).
For random variable X, we can define its
cumulative distribution: For real
number x,
F_X(x) = P(X <= x)
[Here are using TeX notation where F_X is
F with a subscript X.]
Then with calculus and meager assumptions,
the probability density of X is
f_X(x) = d/dx F_X(x)
With meager assumptions, calculus and
f_X(x) can give us the expectation E[X].
The likelihood of X = x is just f_X(x),
that is, the value of the density at x.
For the Gaussian distribution, the maximum
likelihood is at the central peak of the
density which is also the expectation.
In some approaches to statistical
estimation, we have some data and seek
estimate x that maximizes the likelihood
of getting the data we actually did get.
Given events A and B,
we can define the conditional probability
of event A given event B by
P(A|B) = P(A and B) / P(B)
So, if we think of events as geometric
regions and their probabilities as their
areas (actually part of a serious
approach), then P(A|B) is the fraction of
B that is also A.
Then
P(A|B) = P(A and B) / P(B)
is Bayes Rule.
If we do experiments and believe from
whatever prior to the experiment that we
have a meaningful estimate of P(B) or
P(A|B), then maybe we are being Bayesian.
More generally knowing that event B
occurred we can regard that as
information we have obtained, and what
that information says about event A is
just P(A|B).
Then, if events A and B are independent,
event B gives us no more information about
event A and we have
P(A|B) = P(A)
So, if we are interested in event A and
its probability P(A) and suddenly are told
that event B occurred, then for event A we
now want the updated view P(A|B).
Using the measure theory foundations of
probability and the Radon-Nikodym theorem
of measure theory, under meager
assumptions we can define for random
variables X and Y
E[Y|X]
which is a function, say, f(X), of random
variable X and the best non-linear least
squares estimate of Y for any function of
X.
This measure theory approach also lets us
define
E[Y|Z]
for an infinite set Z of random variables.
This definition is useful, e.g., in the
Poisson process where each increment of
time to the next arrival is independent of
all previous increments, Markov processes,
a stochastic process adapted to a
history, etc.
- BeetleB 10y agoYou're being downvoted because everything you say applies to frequentist statistics.
- SubiculumCode 10y agoIt would be nice if more HN users responded with a comment making that point rather than using anonymous downvotes...
- graycat 10y agoI just answered one of the user's questions. The user's question I answered didn't get involved in frequentist versus Bayesian and, thus, neither did my answer. I started my post by quoting the user's question; that's the question I answered. I never used the word frequentist and made only minimal use of the word Bayesian. I avoided all political fights. Note: Possibly of special interests to Bayesian, I touched on E[X|Z] for an infinite collection of random variables Z. This will be important in conditioning (the core of Bayesian) in statistics of stochastic processes. Also of interest in conditioning, I did mention that if events A and B are independent, then B gives no more information about A because P(A|B) = P(A) Anyone working with conditioning needs to know this concept. More generally, I mentioned the sense in which conditioning gives the best possible non-linear least squares estimate; so, here we begin to see the power of conditioning, of which Bayes Rule is the most elementary case. Also you may find that my touching on the Radon-Nikodym theorem is a first step to high end versions of Bayesian, e.g., the old idea of sequential testing (A. Wald) and the concepts of stopping times, optimal stopping, the strong Markov property, etc. I wrote out an earlier response, longer, I didn't post, that did go back to sigma algebras, measurability, etc. I did omit measurable selection, sufficient statistics, etc. For such concepts, the Radon-Nikodym theorem and sigma algebras are crucial, and my post may be the only one here that mentioned either. Also, comparing my response to others, you may find that my response was comparatively clear, precise, understandable, for such a short post without poorly defined or undefined terms, correct, and from a mature view. By the way, I hold a Ph.D. in applied math from one of the world's best and best known research universities. My research was on stochastic optimal control and passed an oral exam from a Member, US National Academy of Engineering. I've published as sole author peer-reviewed original research in mathematical statistics. Once I did a statistical estimation of expected revenue growth for the BoD of FedEx; my work got two Board representatives from investor General Dynamics to change their mind and stay and, thus, saved FedEx. I've worked in statistical consulting in finance, marketing, etc., including in computing and statistical consulting for the faculty at Georgetown University. My work in statistical power spectral estimation got my company sole source on an important contract from the US Navy. Once I did a Monte Carlo estimation of a statistical estimation I did of the survivability of the US SSBN fleet under a special scenario of global nuclear war limited to sea -- the US Navy was pleased. My work passed review from J. Keilson, a world class expert in statistics. Maybe, instead of what I wrote, some readers were looking for something else. Sorry some people were offended.