8 ms·

Learning about Innovation from Half a Century of Conway's Game of Life

111 points by feelthepain 2y ago

kccqzy 2y ago
> if we want to get closer to the study of the pure phenomenon of innovation Innovation in the real world is often driven by the usual incentives of capitalism, like the basic need to out-compete the competitors by improving quality or lowering costs. I do not really think Game of Life serves as a model for innovation in the real world; it might serve as a model of the pure phenomenon of innovation. In the real world, even things like pure math research is motivated by applied math, by monetary factors like NSF grants etc.
- leepyaar 2y ago
  This is a myopic view. How would we have gotten to the point of human history when capitalism became predominant if innovation is driven by capitalism? Also, humanity is hardly an economy of capitalism any more. More similar to oligarchical capital feudalism.
  - enslavedrobot 2y ago
    You're right, it's just a coincidence that the advent of capitalism and an unimaginable increase in global wealth happened at the same time. Thousands of texts written by historians and economists who dedicate their lives to understanding our past are totally wrong.
    - nxobject 2y ago
      I'm not sure what you're saying is inconsistent with parent's comment: you're both talking about different points in time in humanity's history – the snark seems unwarranted.
      - enslavedrobot 2y ago
        Calling the parent myopic while saying "Also, humanity is hardly an economy of capitalism any more. More similar to oligarchical capital feudalism." Is deserving of snark. In the context of 5 centuries of capitalism we live in an age of unprecedented equality, freedom, and opportunity. Further, observed over the last 5000 years of human civilization the stupendous rise of innovation under capitalism is so dramatic that implying it is not responsible for the wealth we enjoy today beggars belief.
  - kccqzy 2y ago
    Well the pace of innovation before the dominance of capitalism was slow. It became much faster afterwards.
- nickpsecurity 2y ago
  I agree. Going back further, we see the innovation drivers were God providing for man (Bible), people providing for their needs/wants at a societal level (most systems), individuals pursuing what they enjoy, seemingly random acts that go against reason, etc. This was enacted via specific processes (human brain) using resources and their environment within their constraints, sometimes surplus. It involved local and global phenomenon that were dependent and independent. In short, it's nothing like cellular automata or most simplistic models of the world. We'd have to model the above within this world's laws to know what drives innovation among humans in this world.
amai 2y ago
Cellular automata are actually useful to learn something about AI, too: https://the-decoder.com/edge-of-chaos-yale-study-finds-sweet-spot-in-data-complexity-helps-ai-learn-better/ https://the-decoder.com/edge-of-chaos-yale-study-finds-sweet...
visarga 2y ago
The cool thing about Conway's Game of Life is that you can't predict it unless you do the full recursion, there is no shortcut. It relates to external undecidability of recurrent processes.
- whatnow37373 2y ago
  "Computation irreducibility"[0] is Mr. Wolfram's word for it and I believe it has some relationship to his CA physics, but I won't pretend I understand. [0] https://en.wikipedia.org/wiki/Computational_irreducibility https://en.wikipedia.org/wiki/Computational_irreducibility
  - jonahbenton 2y ago
    Yes. He relates an interpretation/definition of the second law of thermodynamics- the increasing entropy thing- to his irreducibility. And builds a computational theory about the role and importance of the physics "observer" in these analyses. Computational irreducibility is basically a statement both about the intrinsic requirement for computation to arrive at the future, and also about the computational capability of the "observer"- a model, or our brains- to arrive, or not, at the future more efficiently. I am a computational person, not a scientist, and I think science people find him to be speaking total garbage. That seems a correct assessment to me. His model of the world from physics perspective seems wrong. Nevertheless personally I find his computational lens/bias to be useful.
    - kennysoona 2y ago
      > I am a computational person, not a scientist, and I think science people find him to be speaking total garbage. That seems a correct assessment to me. His model of the world from physics perspective seems wrong. I don't think it's that he is speaking garbage, he is basically talking about digital physics which is a real theory being considered and researched, not pseudoscience. But he doesn't work with the scientific community at all, he just writes his long essays and uses his own terms and ignores anyone else doing similar work. He then gets upset when scientists don't just defer to him.
dvh 2y ago
If you like game of life you gonna love this video about backwards game of life: https://youtube.com/watch?v=g8pjrVbdafY https://youtube.com/watch?v=g8pjrVbdafY
MattGrommes 2y ago
I feel like I'm too dumb to understand whether these CA articles of his are interesting or just numerology deep-dives like people who spend their lives "investigating" the torah.
- leephillips 2y ago
  He’s been at this for decades, without a single prediction of any phenomena in nature. His theories can’t even reproduce known physical phenomena correctly. So you can be dumb and still correctly conclude that it’s basically numerology.
- jerf 2y ago
  From the article, for some context for what I'm about to say: "And indeed, what we’ll often see is that the more optimized a structure is, the less modular it tends to be. If we’re going to construct something “by hand” we usually need to assemble it in parts, because that’s what allows us to “understand what we’re doing”. But if, for example, we just find a structure in a search, there’s no reason for it to be “understandable”, and there’s no reason for it to be particularly modular." If I were going to sum up what I will politely call Wolfram's Conjecture, it is that there is some other way to start with some sort of understanding of cellular automata and derive from that understanding the ability to model systems, and presumably the ability to build systems in cellular automata, using that superior understanding that are not characterized by the modularity shown in human constructions. Something like chaos mathematics, but for cellular automata, and presumably, something that goes beyond statistical characterization into some sort of deeper understanding that is somehow analogous to our ability to deeply understand the modular systems. For the purposes of what I'm saying here, I'm virtually equating "understanding" with "ability to meaningfully manipulate". You or I may not be able to manipulate one of the human constructed systems in Life, but there is a path to learn how that is observably human-comprehensible and capable of being conveyed through human communication, because it has been. By contrast, the article shows many cases of "humans constructed this system with property X, and by random search we found a much more efficient system with that property, but it is not something a human could ever have reasonably designed". What I may less charitably call "Wolfram's Mistake" is that there is at the moment frankly no evidence that I can see that such a thing is true, despite him having been searching for it for a very long time. There's two angles on that, which are: One, is there any such understanding that is accessible to human cognition? And two, is there any such understanding at all, without that limitation, but with some sort of finite level of intelligence significantly smaller than the level necessary to just brute force the problem? Or, to put it another way, it is not hard to hypothesize intelligences that could just glance a description of Life and then simply internally simulate systems directly looking for whatever you like. You may consider them something like "What if a human just had a modern computer built directly into their brain?" They would have vastly, vastly more raw computational power than a human without that for this purpose, but, is there any sort of understanding of cellular automata in general or even Life in particular that is meaningfully more compact than simply running the automata? If you want to dig more into what it means to "understand" something, I refer you to "Why Philosophers Should Care About Computational Complexity" https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf , rather than trying to further elaborate in this post. An "understanding" of this sort of cellular automata, whatever it may be, no matter how alien the system that has this "understanding" may be, should me meaningfully more computationally compact than the exponential complexity of simply having a table of all possibilities in memory. From this perspective "Wolfram's Mistake" is that there is no such understanding; there is fundamentally no way to create a computational model for the phenomena in a cellular automata system that is simpler than the cellular automata itself in a general case. Which means that rather than being the key to understanding, they're actually a really bad lens to try to view the world through for any sort of understanding by us finite beings. As incomplete and simplified as our many other lenses may be, at least they work in some cases. Cellular automata don't seem amenable to compact theories of any utility. (And by "any utility", I mean, any utility; scientific, mathematical, engineering, practical, all of them.) There's a few places of use, but not very many. An interesting aspect is that while all Turing Complete systems ultimately exhibit this behavior, they do not seem to all do so equally. Lambda calculus, for instance, has all the same chaos in some sense, if for no other reason than you can always create the game of Life directly in lambda calculus. Yet lambda calculus also admits human understanding. I select lambda calculus in particular as one of the most human-friendly Turing Complete formalisms. Turning them into actual useful programming languages is on the order of "useful learning exercise". Turing machines are sort of in between; we can work with them, but they do tend to explode into chaos relatively easily. Cellular automata, by contrast, require extreme human effort to work with, just to get them to halfway tame. There's an interesting study here about why that is, which is currently well beyond me. Again, looking at Wolfram's Conjecture another way is that this is somehow not correct and we're just not looking at CAs correctly, and if we did, they might become more useful than Turing machines or perhaps even lambda calculus, and, again, I just don't see the evidence for this.
1ewish 2y ago
Whilst it is not framed as such, there are some interesting high level takeaways in here for AI, particularly around e.g program synthesis/induction and ARC. I think Wolfram could contribute a lot if he shifted his focus towards these domains.
dang 2y ago
[stub for offtopicness]
- r00t- 2y ago
  [flagged]
  - MrMcCall 2y ago
    [flagged]
  - procgen 2y ago
    There's much to be said!
    - seanw444 2y ago
      It's unfortunate that people seem to be prejudiced against Wolfram at this point, when the field has a lot to explore and learn from. Cellular automata are so powerful, yet so conceptually simple, it wouldn't surprise me if it did have revelations for more fundamental concepts of the universe. What if the fundamentals of the universe are so simple it'd shock us, we just haven't looked at it from the right perspective yet, and we're over-complicating it?
      - 9283409232 2y ago
        People should be able to separate the merits of the work from the worker but people didn't just randomly decide to not like Wolfram. He is a narcissist of the highest order.
        seanw444 2y ago
        Perhaps he is, but the field isn't worth discounting nonetheless, and he's one of the most prominent researchers of it.
        2y ago
the__alchemist 2y ago
This article is fascinating! Some of the concepts in it, like _computational irreducibility_ seem to be core concepts that live in a domain so low I'm not sure where to define its bounds. At the same fuzzy level of "evolution".
Isamu 2y ago
Even if you are a hater, this is a very interesting and information-rich overview. Wolfram is relentlessly litigating his place in history and this piece is no exception but I enjoyed the almost manic amount of detail. Long read indeed.

111 points by feelthepain 2y ago

> if we want to get closer to the study of the pure phenomenon of innovation Innovation in the real world is often driven by the usual incentives of capitalism, like the basic need to out-compete the competitors by improving quality or lowering costs. I do not really think Game of Life serves as a model for innovation in the real world; it might serve as a model of the pure phenomenon of innovation. In the real world, even things like pure math research is motivated by applied math, by monetary factors like NSF grants etc.

leepyaar 2y ago
This is a myopic view. How would we have gotten to the point of human history when capitalism became predominant if innovation is driven by capitalism? Also, humanity is hardly an economy of capitalism any more. More similar to oligarchical capital feudalism.
- enslavedrobot 2y ago
  You're right, it's just a coincidence that the advent of capitalism and an unimaginable increase in global wealth happened at the same time. Thousands of texts written by historians and economists who dedicate their lives to understanding our past are totally wrong.
  - nxobject 2y ago
    I'm not sure what you're saying is inconsistent with parent's comment: you're both talking about different points in time in humanity's history – the snark seems unwarranted.
    - enslavedrobot 2y ago
      Calling the parent myopic while saying "Also, humanity is hardly an economy of capitalism any more. More similar to oligarchical capital feudalism." Is deserving of snark. In the context of 5 centuries of capitalism we live in an age of unprecedented equality, freedom, and opportunity. Further, observed over the last 5000 years of human civilization the stupendous rise of innovation under capitalism is so dramatic that implying it is not responsible for the wealth we enjoy today beggars belief.
- kccqzy 2y ago
  Well the pace of innovation before the dominance of capitalism was slow. It became much faster afterwards.
nickpsecurity 2y ago
I agree. Going back further, we see the innovation drivers were God providing for man (Bible), people providing for their needs/wants at a societal level (most systems), individuals pursuing what they enjoy, seemingly random acts that go against reason, etc. This was enacted via specific processes (human brain) using resources and their environment within their constraints, sometimes surplus. It involved local and global phenomenon that were dependent and independent. In short, it's nothing like cellular automata or most simplistic models of the world. We'd have to model the above within this world's laws to know what drives innovation among humans in this world.

amai 2y ago

Cellular automata are actually useful to learn something about AI, too: https://the-decoder.com/edge-of-chaos-yale-study-finds-sweet-spot-in-data-complexity-helps-ai-learn-better/ https://the-decoder.com/edge-of-chaos-yale-study-finds-sweet...

visarga 2y ago

The cool thing about Conway's Game of Life is that you can't predict it unless you do the full recursion, there is no shortcut. It relates to external undecidability of recurrent processes.

whatnow37373 2y ago
"Computation irreducibility"[0] is Mr. Wolfram's word for it and I believe it has some relationship to his CA physics, but I won't pretend I understand. [0] https://en.wikipedia.org/wiki/Computational_irreducibility https://en.wikipedia.org/wiki/Computational_irreducibility
- jonahbenton 2y ago
  Yes. He relates an interpretation/definition of the second law of thermodynamics- the increasing entropy thing- to his irreducibility. And builds a computational theory about the role and importance of the physics "observer" in these analyses. Computational irreducibility is basically a statement both about the intrinsic requirement for computation to arrive at the future, and also about the computational capability of the "observer"- a model, or our brains- to arrive, or not, at the future more efficiently. I am a computational person, not a scientist, and I think science people find him to be speaking total garbage. That seems a correct assessment to me. His model of the world from physics perspective seems wrong. Nevertheless personally I find his computational lens/bias to be useful.
  - kennysoona 2y ago
    > I am a computational person, not a scientist, and I think science people find him to be speaking total garbage. That seems a correct assessment to me. His model of the world from physics perspective seems wrong. I don't think it's that he is speaking garbage, he is basically talking about digital physics which is a real theory being considered and researched, not pseudoscience. But he doesn't work with the scientific community at all, he just writes his long essays and uses his own terms and ignores anyone else doing similar work. He then gets upset when scientists don't just defer to him.

dvh 2y ago

If you like game of life you gonna love this video about backwards game of life: https://youtube.com/watch?v=g8pjrVbdafY https://youtube.com/watch?v=g8pjrVbdafY

MattGrommes 2y ago

I feel like I'm too dumb to understand whether these CA articles of his are interesting or just numerology deep-dives like people who spend their lives "investigating" the torah.

leephillips 2y ago
He’s been at this for decades, without a single prediction of any phenomena in nature. His theories can’t even reproduce known physical phenomena correctly. So you can be dumb and still correctly conclude that it’s basically numerology.
jerf 2y ago
From the article, for some context for what I'm about to say: "And indeed, what we’ll often see is that the more optimized a structure is, the less modular it tends to be. If we’re going to construct something “by hand” we usually need to assemble it in parts, because that’s what allows us to “understand what we’re doing”. But if, for example, we just find a structure in a search, there’s no reason for it to be “understandable”, and there’s no reason for it to be particularly modular." If I were going to sum up what I will politely call Wolfram's Conjecture, it is that there is some other way to start with some sort of understanding of cellular automata and derive from that understanding the ability to model systems, and presumably the ability to build systems in cellular automata, using that superior understanding that are not characterized by the modularity shown in human constructions. Something like chaos mathematics, but for cellular automata, and presumably, something that goes beyond statistical characterization into some sort of deeper understanding that is somehow analogous to our ability to deeply understand the modular systems. For the purposes of what I'm saying here, I'm virtually equating "understanding" with "ability to meaningfully manipulate". You or I may not be able to manipulate one of the human constructed systems in Life, but there is a path to learn how that is observably human-comprehensible and capable of being conveyed through human communication, because it has been. By contrast, the article shows many cases of "humans constructed this system with property X, and by random search we found a much more efficient system with that property, but it is not something a human could ever have reasonably designed". What I may less charitably call "Wolfram's Mistake" is that there is at the moment frankly no evidence that I can see that such a thing is true, despite him having been searching for it for a very long time. There's two angles on that, which are: One, is there any such understanding that is accessible to human cognition? And two, is there any such understanding at all, without that limitation, but with some sort of finite level of intelligence significantly smaller than the level necessary to just brute force the problem? Or, to put it another way, it is not hard to hypothesize intelligences that could just glance a description of Life and then simply internally simulate systems directly looking for whatever you like. You may consider them something like "What if a human just had a modern computer built directly into their brain?" They would have vastly, vastly more raw computational power than a human without that for this purpose, but, is there any sort of understanding of cellular automata in general or even Life in particular that is meaningfully more compact than simply running the automata? If you want to dig more into what it means to "understand" something, I refer you to "Why Philosophers Should Care About Computational Complexity" https://www.scottaaronson.com/papers/philos.pdf https://www.scottaaronson.com/papers/philos.pdf , rather than trying to further elaborate in this post. An "understanding" of this sort of cellular automata, whatever it may be, no matter how alien the system that has this "understanding" may be, should me meaningfully more computationally compact than the exponential complexity of simply having a table of all possibilities in memory. From this perspective "Wolfram's Mistake" is that there is no such understanding; there is fundamentally no way to create a computational model for the phenomena in a cellular automata system that is simpler than the cellular automata itself in a general case. Which means that rather than being the key to understanding, they're actually a really bad lens to try to view the world through for any sort of understanding by us finite beings. As incomplete and simplified as our many other lenses may be, at least they work in some cases. Cellular automata don't seem amenable to compact theories of any utility. (And by "any utility", I mean, any utility; scientific, mathematical, engineering, practical, all of them.) There's a few places of use, but not very many. An interesting aspect is that while all Turing Complete systems ultimately exhibit this behavior, they do not seem to all do so equally. Lambda calculus, for instance, has all the same chaos in some sense, if for no other reason than you can always create the game of Life directly in lambda calculus. Yet lambda calculus also admits human understanding. I select lambda calculus in particular as one of the most human-friendly Turing Complete formalisms. Turning them into actual useful programming languages is on the order of "useful learning exercise". Turing machines are sort of in between; we can work with them, but they do tend to explode into chaos relatively easily. Cellular automata, by contrast, require extreme human effort to work with, just to get them to halfway tame. There's an interesting study here about why that is, which is currently well beyond me. Again, looking at Wolfram's Conjecture another way is that this is somehow not correct and we're just not looking at CAs correctly, and if we did, they might become more useful than Turing machines or perhaps even lambda calculus, and, again, I just don't see the evidence for this.

1ewish 2y ago

Whilst it is not framed as such, there are some interesting high level takeaways in here for AI, particularly around e.g program synthesis/induction and ARC. I think Wolfram could contribute a lot if he shifted his focus towards these domains.

dang 2y ago

[stub for offtopicness]

r00t- 2y ago
[flagged]
- MrMcCall 2y ago
  [flagged]
- procgen 2y ago
  There's much to be said!
  - seanw444 2y ago
    It's unfortunate that people seem to be prejudiced against Wolfram at this point, when the field has a lot to explore and learn from. Cellular automata are so powerful, yet so conceptually simple, it wouldn't surprise me if it did have revelations for more fundamental concepts of the universe. What if the fundamentals of the universe are so simple it'd shock us, we just haven't looked at it from the right perspective yet, and we're over-complicating it?
    - 9283409232 2y ago
      People should be able to separate the merits of the work from the worker but people didn't just randomly decide to not like Wolfram. He is a narcissist of the highest order.
      - seanw444 2y ago
        Perhaps he is, but the field isn't worth discounting nonetheless, and he's one of the most prominent researchers of it.
        2y ago

the__alchemist 2y ago

This article is fascinating! Some of the concepts in it, like _computational irreducibility_ seem to be core concepts that live in a domain so low I'm not sure where to define its bounds. At the same fuzzy level of "evolution".

Isamu 2y ago

Even if you are a hater, this is a very interesting and information-rich overview. Wolfram is relentlessly litigating his place in history and this piece is no exception but I enjoyed the almost manic amount of detail. Long read indeed.