3 ms·

CAs are interesting computational systems but it is really difficult to follow the multiway hypergraph rewriting idea and its connection to physics. It sounds

CAs are interesting computational systems but it is really difficult to follow the multiway hypergraph rewriting idea and its connection to physics. It sounds reasonable but I am missing a few inductive steps along the way. That may be a me thing... but I also don't know the physics to say one way or another that the outputs he finds match the physical result. The mutational stuff for parameter search is known already via genetic algorithms which have been employed across the board in optimization, neural networks, machine learning, and even to CAs. Exploring the space of all 'CAs' (or even the ruliad) by enumerative exhaustion is somewhat interesting but is it similar to defining a FSA by the exposition of all strings accepted by some FSA? If so that seems to be a waste of time. It seems to be a CS version of reading the tea leaves (tasseography). But I may be missing something here as well. I think this is why he's moved on to "evolution" i.e. mutation with selective constraints (evolution). That seems to be the only way to find something somewhat automatically but it also seems that there may be an infinite number of paths... however, the domain of these functions would be limited by the size of the 'genome' (in classical genetic algorithms) or in the functional inputs of the CA. CA are generally restricted to neighbors and all information is propagated via local neighbors. This limits your rule search space and defines your state space but you could easily have CAs that depend on all neighbors (global state). NNs carry global state and early single layer perceptron's didn't scale until they were made deep (and given additional explicit structure like convolution or transformation)... yes they are all computationally equivalent but one set aren't very useful (even for us trying to understand/reason about) and others result in ChatGPT...

bestatsiege_01 2y ago
I have explored trying to make global state CA systems, with no success. If anyone has examples of trainable global state CA systems please share, I'd love to learn more. My idea was that there would be global information but local rules. And the local rules depended on the global information. However I struggled to train such a system and gave up. I would like to go back to it though and some expert perspective might be enough to push me back to it. I find CA's interesting because they are both continuous and could conceptually vary the amount of compute attributed to a particular problem. NN's can't really do either of these things. (I appreciate LLM's vary the amount of compute based on the question. But just running the same model over and over seems like an immature method, there must be a better way.)
- theGnuMe 2y ago
  Thanks for the reply. How would CAs vary the amount of compute? Don't you have to compute everything state-wise every iteration? Right now my understanding is that in neural cellular automata people replace the update rule with a DNN. And this DNN is trained on small inputs. Basically a cell's neighborhood input vs a "pixel" vs token level input... a cellular neighborhood here is basically patches which aligns with DNNs anyway. A good example is: https://distill.pub/2020/growing-ca/ https://distill.pub/2020/growing-ca/ The examples remind of inpainting though in some sense. You can apply transformers to this to get a shared memory (people have done this I believe). Too be honest I feel like neural CAs are a trick but I am probably wrong.
  - bestatsiege_01 2y ago
    My hypothesis is that CA's could have input and output cells, then just run the CA, pass a single or continuous input and then parse the outputs when available. But - I haven't been able to get such a concept to work. I maybe missing some fundamental theory / understand which prevents such a structure (or at least limits is value). One major challenge is how do you train an unconstrained process? Ill take a look at Neural CA's. thanks for sharing. Thanks for sharing Neural CA's. I'll spend some time on them, much easier to train a DNN.
    - theGnuMe 2y ago
      Wolfram talks about mobile automata.. This is where you calculate one cell at a time. https://en.wikipedia.org/wiki/Mobile_automaton https://en.wikipedia.org/wiki/Mobile_automaton and https://www.wolframscience.com/nks/p71--mobile-automata/ https://www.wolframscience.com/nks/p71--mobile-automata/ I'm confused as to how you actually train a cellular automaton? How does one do it? Is it just rule search to match a pattern at time t given input state from time t-1 (and repeated?).. it seems like you'd have a bunch of boolean equations and then look for and extract its solutions.

bestatsiege_01 2y ago

I have explored trying to make global state CA systems, with no success. If anyone has examples of trainable global state CA systems please share, I'd love to learn more. My idea was that there would be global information but local rules. And the local rules depended on the global information. However I struggled to train such a system and gave up. I would like to go back to it though and some expert perspective might be enough to push me back to it. I find CA's interesting because they are both continuous and could conceptually vary the amount of compute attributed to a particular problem. NN's can't really do either of these things. (I appreciate LLM's vary the amount of compute based on the question. But just running the same model over and over seems like an immature method, there must be a better way.)

theGnuMe 2y ago
Thanks for the reply. How would CAs vary the amount of compute? Don't you have to compute everything state-wise every iteration? Right now my understanding is that in neural cellular automata people replace the update rule with a DNN. And this DNN is trained on small inputs. Basically a cell's neighborhood input vs a "pixel" vs token level input... a cellular neighborhood here is basically patches which aligns with DNNs anyway. A good example is: https://distill.pub/2020/growing-ca/ https://distill.pub/2020/growing-ca/ The examples remind of inpainting though in some sense. You can apply transformers to this to get a shared memory (people have done this I believe). Too be honest I feel like neural CAs are a trick but I am probably wrong.
- bestatsiege_01 2y ago
  My hypothesis is that CA's could have input and output cells, then just run the CA, pass a single or continuous input and then parse the outputs when available. But - I haven't been able to get such a concept to work. I maybe missing some fundamental theory / understand which prevents such a structure (or at least limits is value). One major challenge is how do you train an unconstrained process? Ill take a look at Neural CA's. thanks for sharing. Thanks for sharing Neural CA's. I'll spend some time on them, much easier to train a DNN.
  - theGnuMe 2y ago
    Wolfram talks about mobile automata.. This is where you calculate one cell at a time. https://en.wikipedia.org/wiki/Mobile_automaton https://en.wikipedia.org/wiki/Mobile_automaton and https://www.wolframscience.com/nks/p71--mobile-automata/ https://www.wolframscience.com/nks/p71--mobile-automata/ I'm confused as to how you actually train a cellular automaton? How does one do it? Is it just rule search to match a pattern at time t given input state from time t-1 (and repeated?).. it seems like you'd have a bunch of boolean equations and then look for and extract its solutions.