4 ms·

A couple of years ago I wrote a book about a new philosophy of science based on the following methodology: - Obtain a large dataset of observations related to

A couple of years ago I wrote a book about a new philosophy of science based on the following methodology: - Obtain a large dataset of observations related to a phenomenon of interest (eg English text, bioimaging data, economic reports, car-mounted camera recordings, etc) - Develop a theory of the phenomenon, or revise a previous theory - Build the theory into a lossless data compression program - Score the theory by invoking the compressor on the dataset, taking into account the size of the compressor itself; lower codelength is better. - Adopt or discard the theory as appropriate and return to step #2. I believe that this is a valid variant of the traditional scientific method, in which data compression takes the place of experimental prediction. In Graham's terms, this idea is a small delta of novelty, but since it refers to a predecessor idea of tremendous significance and generality, it could be very important. The book is available here: https://arxiv.org/abs/1104.5466 https://arxiv.org/abs/1104.5466 I've spent a few years pursuing this methodology in the domain of English text. The result is a sentence parser, which works quite well, but was built completely without the use of hand-labelled training data (eg the Penn Treebank). You can check out the parser here: http://ozoraresearch.com/crm/public/parseview/UserParseView.jsp http://ozoraresearch.com/crm/public/parseview/UserParseView.... I'm always happy to talk to people about these ideas, ping me at daniel dot burfoot at gmail if you're interested.

jsweojtj 9y ago
This is interesting. I don't have an intuition on how well this could work on different problems. Do you think this approach would work for something, let's start simple, like ballistics neglecting rotation of the Earth and drag? If you fed in thousands of starting velocities and angles, and the resulting distance traveled, will this process end up with equations of motion that are as compact as can be found in first year physics?
- indiv0 9y ago
  I think the problem here is that you are still the one that needs to develop the equations of motion that will be tested for compression. What this describes is a method for testing theories, not generating them. You still need to study the data to try and properly analyze it so you can understand the rules yourself, or at least be able to see the edges of them. Now if you could find a way to generate moderately accurate equations automatically by analyzing the dataset, I think that would be a real leap forward. Is there a related mechanism to the one presented by the parent which could be used to generate theories?
  - d_burfoot 9y ago
    Yes, it is a method for testing theories; it does not give you a procedure for developing the theories. It is agnostic about whether the theories are developed by humans or algorithms. There is not going to be any universal and practical algorithm for automatically generating the theories, at least in the short term. But you can think about algorithms that apply to particular domains. For example in my field of text analysis, you can imagine techniques for automatically inferring grammar rules from a large quantity of text. Some such methods already exist, but have not been pursued extensively. We could also develop algorithms for inferring equations of motion from astronomical and celestial observations, as the parent mentioned. The compression principle wouldn't give you the real Newtonian equations, but it would correctly identify them as being superior to other candidates. But this would be a bit of an academic exercise, since we already have the Newtonian equations and relativity. What's more interesting to me is an attempt to build modern astronomical theories (eg about black holes, the Big Bang, cosmic inflation, dark energy, etc) into a compressor that will apply to an astronomical data set like the Sloan Digital Sky Survey. My guess is that such an attempt would reveal important shortcomings in the theories, or perhaps highlight regions of space that confound the theories.
jamiek88 9y ago
Interesting, and I just read the abstract from your book, why do you think size of compression code indicates strength of theory? Regularities in data/predictability of data?
- d_burfoot 9y ago
  Simplifying drastically, fully general lossless data compression is impossible because of a simple No Free Lunch theorem. To achieve compression in practice, a compressor must exploit empirical regularities in the data. For example, PNG can achieve compression on most real world images because it exploits a regularity that they exhibit, which is that adjacent pixels tend to be highly correlated. So lossless data compression is about documenting and describing empirical regularities in real world observations; exactly the same as empirical science.

jsweojtj 9y ago

This is interesting. I don't have an intuition on how well this could work on different problems. Do you think this approach would work for something, let's start simple, like ballistics neglecting rotation of the Earth and drag? If you fed in thousands of starting velocities and angles, and the resulting distance traveled, will this process end up with equations of motion that are as compact as can be found in first year physics?

indiv0 9y ago
I think the problem here is that you are still the one that needs to develop the equations of motion that will be tested for compression. What this describes is a method for testing theories, not generating them. You still need to study the data to try and properly analyze it so you can understand the rules yourself, or at least be able to see the edges of them. Now if you could find a way to generate moderately accurate equations automatically by analyzing the dataset, I think that would be a real leap forward. Is there a related mechanism to the one presented by the parent which could be used to generate theories?
- d_burfoot 9y ago
  Yes, it is a method for testing theories; it does not give you a procedure for developing the theories. It is agnostic about whether the theories are developed by humans or algorithms. There is not going to be any universal and practical algorithm for automatically generating the theories, at least in the short term. But you can think about algorithms that apply to particular domains. For example in my field of text analysis, you can imagine techniques for automatically inferring grammar rules from a large quantity of text. Some such methods already exist, but have not been pursued extensively. We could also develop algorithms for inferring equations of motion from astronomical and celestial observations, as the parent mentioned. The compression principle wouldn't give you the real Newtonian equations, but it would correctly identify them as being superior to other candidates. But this would be a bit of an academic exercise, since we already have the Newtonian equations and relativity. What's more interesting to me is an attempt to build modern astronomical theories (eg about black holes, the Big Bang, cosmic inflation, dark energy, etc) into a compressor that will apply to an astronomical data set like the Sloan Digital Sky Survey. My guess is that such an attempt would reveal important shortcomings in the theories, or perhaps highlight regions of space that confound the theories.

jamiek88 9y ago

Interesting, and I just read the abstract from your book, why do you think size of compression code indicates strength of theory? Regularities in data/predictability of data?

d_burfoot 9y ago
Simplifying drastically, fully general lossless data compression is impossible because of a simple No Free Lunch theorem. To achieve compression in practice, a compressor must exploit empirical regularities in the data. For example, PNG can achieve compression on most real world images because it exploits a regularity that they exhibit, which is that adjacent pixels tend to be highly correlated. So lossless data compression is about documenting and describing empirical regularities in real world observations; exactly the same as empirical science.