3 ms·

The site itself says that this is the general idea, proposing that intelligence can actually be defined as the ability to compress information as much as possib

The site itself says that this is the general idea, proposing that intelligence can actually be defined as the ability to compress information as much as possible and.. I guess either just rehydrate it or also derive inferences from it. So basically like enthalpy Interesting - if you play the universe in reverse (reversing entropy) you wind up at the Big Bang, which is the final boss of compressing everything, into a singularity.

DaiPlusPlus 3y ago
> The site itself says that this is the general idea, proposing that intelligence can actually be defined as the ability to compress information as much as possible I don't understand how anyone can define "intelligence" like that...
- GolDDranks 3y ago
  It's not an unreasonable definition, if you are aware of Kolmogorov complexity and Solomonoff induction. Intelligence is intimately connected to the ability to predict and model things, and it turns out that data compression is _also_ connected to the ability of predict and model things.
  - lIIllIIllIIllII 3y ago
    It makes sense intuitively right? If I tell you that this week it's raining on Monday, raining on Tuesday, raining on Wednesday, etc.. Compressing that information into "it's raining every day this week" requires creating an abstraction around it. Finding a pattern. Producing more order from something more chaotic.
    - vjerancrnjak 3y ago
      Interestingly, compression increases entropy. So the first description is more "ordered", or less random.
    - DaiPlusPlus 3y ago
      > Compressing that information into "it's raining every day this week" requires creating an abstraction around it. Finding a pattern. Producing more order from something more chaotic. Is there any way to determine if that process of abstraction-finding and pattern-finding will always/actually/sometimes/never result in a more compressed output than simpler approaches though?
      - GolDDranks 3y ago
        A hunch of mine is that's undecidable in the general case. In simple cases like this, there's likely to be a way to prove that yes, this is tho most compressed thing out there.
  - DaiPlusPlus 3y ago
    > It's not an unreasonable definition, if you are aware of Kolmogorov complexity and Solomonoff induction I was already familiar with Kolmogorov Complexity, but not _Solomonoff's theory of inductive inference_ - I skimmed the Wikipedia article just now and I think/hope I get the general gist of it - so thank you for that. > Intelligence is intimately connected to the ability to predict and model things I agree with that statement - but I am unsure how this relates to other aspects of intelligence - or even other notions of intelligence entirely. If we're redefining exactly what the word "Intelligence" means aren't we just moving the goalposts? ...so I'm unsure if the word "Intelligence" in your post can/should be read as a layperson would understand it, or if it's a specific, well-defined term in this field? (Like how "complexity" in algorithms does not refer to how convoluted and unmaintainable a program is, which is what a layperson would think of when an SWE person says "this program has high computational complexity" to them). > and it turns out that data compression is _also_ connected to the ability of predict and model things. This is where I have difficulty following your argument: As a preface: my academic understanding of data compression only covers things like algorithms in the LZ family, Entropy coding, and (lossy and lossless) domain-specific compression schemes (like JPEG/DCT, MP3/FLAC, etc). I am aware of very interesting results coming from present-day AI approaches, like using LLMs fed only on compressed data - but these are completely outside my scope of understanding (and LLMs are still very spooky to me). Does it matter if a scheme is lossless or lossy? In a lossless system, surely a data compression system needs to be deterministic and mechanical? If so, what room is there for "intelligence" in a rigid, statically-defined, system? Take entropy coding for example - or even simpler probability-based compression schemes like Morse Code (e.g. "E" has a high probability, so it has a very short symbol length). I just can't see the connection from things like entropy-encoding to "modelling": supposing I have a large file (CSV?) of meterological data for a specific region over time - if it's plaintext then I expect I can compress it better using gzip than by using a system that can (somehow!) identify some structure to the weather sensor data (the "model", right?) and then use that as the basis for a better domain-specific compression - but doing this means having to add additional metadata to the source data to describe the structure that the system identified, and then hope that this approach is better than a comparatively "dumb" approach like DEFLATE - and even then, assuming that employing that "model" really does result in smaller compressed output, how is that an example of the system having a general "intelligence"?
eru 3y ago
> Interesting - if you play the universe in reverse (reversing entropy) you wind up at the Big Bang, which is the final boss of compressing everything, into a singularity. Only if you believe that running the universe forward did not add any extra information. Ie everything is deterministic, and no randomness occurs. (That's actually a reasonable stance to take. Quantum mechanics is probabilistic in the Copenhagen interpretation, but completely deterministic in the Many World interpretation.)
- vjerancrnjak 3y ago
  A non-local formulation of QM is deterministic. [0] An addition to the "many world" interpretation. Still one universe but the underlying reality is fully connected (to go forward you need to take the effects of everything). 0: https://en.wikipedia.org/wiki/De_Broglie–Bohm_theory https://en.wikipedia.org/wiki/De_Broglie–Bohm_theory
  - eru 3y ago
    Yes. De Broglie Bohm theory is another example of an interpretation that's deterministic. In practice, because of deterministic chaos it doesn't matter too much whether your lowest level theory is deterministic. (It only matters philosophically.)
kaba0 3y ago
It is more than likely that most physical reactions are not deterministic, and in that case the Big Bang definitely didn’t encode anything about all the state we have now. (Also, simply by Occam’s razor it seems much much more likely)

DaiPlusPlus 3y ago

> The site itself says that this is the general idea, proposing that intelligence can actually be defined as the ability to compress information as much as possible I don't understand how anyone can define "intelligence" like that...

GolDDranks 3y ago
It's not an unreasonable definition, if you are aware of Kolmogorov complexity and Solomonoff induction. Intelligence is intimately connected to the ability to predict and model things, and it turns out that data compression is _also_ connected to the ability of predict and model things.
- lIIllIIllIIllII 3y ago
  It makes sense intuitively right? If I tell you that this week it's raining on Monday, raining on Tuesday, raining on Wednesday, etc.. Compressing that information into "it's raining every day this week" requires creating an abstraction around it. Finding a pattern. Producing more order from something more chaotic.
  - vjerancrnjak 3y ago
    Interestingly, compression increases entropy. So the first description is more "ordered", or less random.
  - DaiPlusPlus 3y ago
    > Compressing that information into "it's raining every day this week" requires creating an abstraction around it. Finding a pattern. Producing more order from something more chaotic. Is there any way to determine if that process of abstraction-finding and pattern-finding will always/actually/sometimes/never result in a more compressed output than simpler approaches though?
    - GolDDranks 3y ago
      A hunch of mine is that's undecidable in the general case. In simple cases like this, there's likely to be a way to prove that yes, this is tho most compressed thing out there.
- DaiPlusPlus 3y ago
  > It's not an unreasonable definition, if you are aware of Kolmogorov complexity and Solomonoff induction I was already familiar with Kolmogorov Complexity, but not _Solomonoff's theory of inductive inference_ - I skimmed the Wikipedia article just now and I think/hope I get the general gist of it - so thank you for that. > Intelligence is intimately connected to the ability to predict and model things I agree with that statement - but I am unsure how this relates to other aspects of intelligence - or even other notions of intelligence entirely. If we're redefining exactly what the word "Intelligence" means aren't we just moving the goalposts? ...so I'm unsure if the word "Intelligence" in your post can/should be read as a layperson would understand it, or if it's a specific, well-defined term in this field? (Like how "complexity" in algorithms does not refer to how convoluted and unmaintainable a program is, which is what a layperson would think of when an SWE person says "this program has high computational complexity" to them). > and it turns out that data compression is _also_ connected to the ability of predict and model things. This is where I have difficulty following your argument: As a preface: my academic understanding of data compression only covers things like algorithms in the LZ family, Entropy coding, and (lossy and lossless) domain-specific compression schemes (like JPEG/DCT, MP3/FLAC, etc). I am aware of very interesting results coming from present-day AI approaches, like using LLMs fed only on compressed data - but these are completely outside my scope of understanding (and LLMs are still very spooky to me). Does it matter if a scheme is lossless or lossy? In a lossless system, surely a data compression system needs to be deterministic and mechanical? If so, what room is there for "intelligence" in a rigid, statically-defined, system? Take entropy coding for example - or even simpler probability-based compression schemes like Morse Code (e.g. "E" has a high probability, so it has a very short symbol length). I just can't see the connection from things like entropy-encoding to "modelling": supposing I have a large file (CSV?) of meterological data for a specific region over time - if it's plaintext then I expect I can compress it better using gzip than by using a system that can (somehow!) identify some structure to the weather sensor data (the "model", right?) and then use that as the basis for a better domain-specific compression - but doing this means having to add additional metadata to the source data to describe the structure that the system identified, and then hope that this approach is better than a comparatively "dumb" approach like DEFLATE - and even then, assuming that employing that "model" really does result in smaller compressed output, how is that an example of the system having a general "intelligence"?

eru 3y ago

> Interesting - if you play the universe in reverse (reversing entropy) you wind up at the Big Bang, which is the final boss of compressing everything, into a singularity. Only if you believe that running the universe forward did not add any extra information. Ie everything is deterministic, and no randomness occurs. (That's actually a reasonable stance to take. Quantum mechanics is probabilistic in the Copenhagen interpretation, but completely deterministic in the Many World interpretation.)

vjerancrnjak 3y ago
A non-local formulation of QM is deterministic. [0] An addition to the "many world" interpretation. Still one universe but the underlying reality is fully connected (to go forward you need to take the effects of everything). 0: https://en.wikipedia.org/wiki/De_Broglie–Bohm_theory https://en.wikipedia.org/wiki/De_Broglie–Bohm_theory
- eru 3y ago
  Yes. De Broglie Bohm theory is another example of an interpretation that's deterministic. In practice, because of deterministic chaos it doesn't matter too much whether your lowest level theory is deterministic. (It only matters philosophically.)

kaba0 3y ago

It is more than likely that most physical reactions are not deterministic, and in that case the Big Bang definitely didn’t encode anything about all the state we have now. (Also, simply by Occam’s razor it seems much much more likely)