5 ms·

I usually enjoy his posts a lot, but this one let me down. He didn't really put much thought into applications of statistics. Maybe the academic perspective i

by mturmon 12y ago

I usually enjoy his posts a lot, but this one let me down. He didn't really put much thought into applications of statistics. Maybe the academic perspective is too limiting. There are lots of non-trivial problems that depend heavily on statistics for solutions. Some examples that come to mind -- signal processing (optimal detectors, say for radar or modems) medical image reconstruction (Poisson counting statistics, regularization) wireless channel allocation source/channel coding (for all radio comms) gzip (and all universal source coding) robotics (localization, object detection) Kalman filters (navigation for planes, etc.) hidden Markov models (for audio/speech processing) weather prediction, data assimilation I'm just getting warmed up. Weird post.

nn3 12y ago
gzip (and all universal source coding) I don't think LZW uses anything statistical. Neither Huffman encoding.
- mturmon 12y ago
  The aim of universal source coding is to get the code rate as close to the source entropy as possible. The proof that this entropy is indeed an absolute limit is statistical (this will be in any introductory text, like Cover and Thomas, that discusses entropy of a stochastic process, i.e. a stream of symbols). And the method of analysis that shows the LZ algorithm itself approaches this limit is also statistical (http://www.cs.duke.edu/courses/spring03/cps296.5/papers/ziv_lempel_1978_variable-rate.pdf http://www.cs.duke.edu/courses/spring03/cps296.5/papers/ziv_...). One of the neat things about LZ is that the algorithm itself really doesn't have a statistical flavor, it's about building dictionaries/codebooks (as you say). But the analysis that shows it actually works is all statistical, or probabilistic, depending on how you define terms.
  - dragontamer 12y ago
    Precisely. Statistics were used in showing that LZ and Huffman Encoding were good from an entropy point of view. There were dozens of compression schemes proposed. How are you to tell which ones are better than other ones? Well... you experiment, and then measure with statistics.
ggambetta 12y ago
And we wouldn't have discovered the Hibbs boson, whatever it is.
- mturmon 12y ago
  Well, you could see the code for the Kalman filter embedded in the Apollo 11 code dump the other day. "Bucy recalls that "no one was very interested in the papers originally." But then researchers at NASA latched onto the Kalman filter as a way of dealing with problems in satellite orbit determination. [...] It was used, for example, in the Ranger, Mariner, and Apollo missions of the 1960s. "In particular, the on-board computer that guided the descent of the Apollo 11 lunar module to the moon had a Kalman filter. ..." From http://www.cs.unc.edu/~welch/kalman/siam_cipra.html http://www.cs.unc.edu/~welch/kalman/siam_cipra.html
notahacker 12y ago
He also missed the entire industries that work on statistics (insurance, finance), which seems a bit like writing an article about "the world without computers" and not referencing the internet. The Incas ran a sophisticated and extensive civilization based on statistics (they had a tax system and central planning functions like grain redistribution policies) and maintained meticulous records of quantitative data without bothering to devise a way of writing words or managing to invent the wheel
- rquantz 12y ago
  You could still have finance without interest. It would consist of lending money at interest based on a subjective estimation of the debtor's ability to repay. Like it used to be. Insurance is harder to imagine...
cjslep 12y ago
Radiation applications (medical, isotopic dating, radio, etc) Nuclear reactors The entire life of Boltzmann and the entire application of his statistical mechanics, such as molecular dynamics and chemistry Solving an engineering problem with uncertainties in data or models (ex: Monte Carlo) Some big ones I immediately thought of. Boltzmann especially would roll in his grave, since this kind of dismissal of his work at the time contributed to his suicide.

by mturmon 12y ago

nn3 12y ago

gzip (and all universal source coding) I don't think LZW uses anything statistical. Neither Huffman encoding.

mturmon 12y ago
The aim of universal source coding is to get the code rate as close to the source entropy as possible. The proof that this entropy is indeed an absolute limit is statistical (this will be in any introductory text, like Cover and Thomas, that discusses entropy of a stochastic process, i.e. a stream of symbols). And the method of analysis that shows the LZ algorithm itself approaches this limit is also statistical (http://www.cs.duke.edu/courses/spring03/cps296.5/papers/ziv_lempel_1978_variable-rate.pdf http://www.cs.duke.edu/courses/spring03/cps296.5/papers/ziv_...). One of the neat things about LZ is that the algorithm itself really doesn't have a statistical flavor, it's about building dictionaries/codebooks (as you say). But the analysis that shows it actually works is all statistical, or probabilistic, depending on how you define terms.
- dragontamer 12y ago
  Precisely. Statistics were used in showing that LZ and Huffman Encoding were good from an entropy point of view. There were dozens of compression schemes proposed. How are you to tell which ones are better than other ones? Well... you experiment, and then measure with statistics.

ggambetta 12y ago

And we wouldn't have discovered the Hibbs boson, whatever it is.

mturmon 12y ago
Well, you could see the code for the Kalman filter embedded in the Apollo 11 code dump the other day. "Bucy recalls that "no one was very interested in the papers originally." But then researchers at NASA latched onto the Kalman filter as a way of dealing with problems in satellite orbit determination. [...] It was used, for example, in the Ranger, Mariner, and Apollo missions of the 1960s. "In particular, the on-board computer that guided the descent of the Apollo 11 lunar module to the moon had a Kalman filter. ..." From http://www.cs.unc.edu/~welch/kalman/siam_cipra.html http://www.cs.unc.edu/~welch/kalman/siam_cipra.html

notahacker 12y ago

He also missed the entire industries that work on statistics (insurance, finance), which seems a bit like writing an article about "the world without computers" and not referencing the internet. The Incas ran a sophisticated and extensive civilization based on statistics (they had a tax system and central planning functions like grain redistribution policies) and maintained meticulous records of quantitative data without bothering to devise a way of writing words or managing to invent the wheel

rquantz 12y ago
You could still have finance without interest. It would consist of lending money at interest based on a subjective estimation of the debtor's ability to repay. Like it used to be. Insurance is harder to imagine...

cjslep 12y ago

Radiation applications (medical, isotopic dating, radio, etc) Nuclear reactors The entire life of Boltzmann and the entire application of his statistical mechanics, such as molecular dynamics and chemistry Solving an engineering problem with uncertainties in data or models (ex: Monte Carlo) Some big ones I immediately thought of. Boltzmann especially would roll in his grave, since this kind of dismissal of his work at the time contributed to his suicide.