8 ms·

Self-organising textures from cellular automata

418 points by fenomas 6y ago

gchamonlive 6y ago
This is very interesting I observed that the more symmetric the basic structures of the pattern/texture are, the more stable the result is/the faster the automata converges. I wonder what it would take to stabilize the worst case I saw there, the veined leaf texture.
Moosdijk 6y ago
If you want to see similar work by some of the same authors, see [1]. The youtube channel "twominutepapers" has an explanation of this work[2]. [1] https://distill.pub/2020/growing-ca/ https://distill.pub/2020/growing-ca/ [2] https://www.youtube.com/watch?v=bXzauli1TyU https://www.youtube.com/watch?v=bXzauli1TyU
- hans1729 6y ago
  Shoutout to twominutepapers, it's a wonderful channel.
  - deleted 6y ago
    [deleted]
ArtWomb 6y ago
Results resemble common micrographs. But perturbed in such a way as to appear alien. Appears we are on the cusp of "neural synthetic biology" ;) Fast differentiable DNA and protein sequence optimization for molecular design https://arxiv.org/abs/2005.11275 https://arxiv.org/abs/2005.11275 Regenerating Soft Robots through Neural Cellular Automata https://arxiv.org/abs/2102.02579 https://arxiv.org/abs/2102.02579
- enchiridion 6y ago
  Is a micrograph the same a motif from network theory?
willis936 6y ago
Nice approximation of nature. You can see both growth and statistical mechanics in the same demonstration.
jonahrd 6y ago
For some reason I find these quite disgusting to look at. I wonder why!
- SamBam 6y ago
  Besides the organic, slightly bacterial-colony forming nature of them, I wonder if you also have trypophobia. I feel like some of the images were triggering the same kind of feelings I get from that.
- JosephRedfern 6y ago
  The way the the images move during evolulution kinda reminds me of worms or maggots wriggling around in soil.
- choxi 6y ago
  https://en.wikipedia.org/wiki/Trypophobia https://en.wikipedia.org/wiki/Trypophobia was mentioned in another comment
JackFr 6y ago
This is the first I've ever read about neural cellular automata, though I thought I was relatively up to date on both cellular automata and deep learning. I think I was able to pick up the broad strokes from context, but is there a good introductory resource for neural cellular automata?
eyvindn 6y ago
Authors here. If you have any questions we'll do our best to answer them! Glad to see people find our work interesting thus far. We also encourage anyone interested to play with the linked Google Colabs [1][2] and read the other articles in the Distill thread. In the Colabs you'll find a bunch more pre-trained textures as well as a workflow to train on your own images, plus some of the scaffolding to recreate figures. [1] https://colab.sandbox.google.com/github/google-research/self-organising-systems/blob/master/notebooks/texture_nca_tf2.ipynb https://colab.sandbox.google.com/github/google-research/self... [2] https://colab.sandbox.google.com/github/google-research/self-organising-systems/blob/master/notebooks/texture_nca_pytorch.ipynb https://colab.sandbox.google.com/github/google-research/self...
- zitterbewegung 6y ago
  The textures remind me of the beginning of once in a lifetime by talking heads. https://www.youtube.com/watch?v=5IsSpAOD6K8 https://www.youtube.com/watch?v=5IsSpAOD6K8
  - alanbernstein 6y ago
    Looks similar to the Gray-Scott model. https://mrob.com/pub/comp/xmorphia/ https://mrob.com/pub/comp/xmorphia/
- layer8 6y ago
  How large is the state space for each cell? Full 8-bit RGB (= 24 bits)?
  - zzznah 6y ago
    Each cell has 12 8-bit channels, including rgb, so it is 96 bits.
    - yorwba 6y ago
      The article says "our NCA model contains 16 channels. The first three are visible RGB channels and the rest we treat as latent channels which are visible to adjacent cells during update steps, but excluded from loss functions."
Sebastian_09 6y ago
Warning for people suffering from trypophobia, some of the combinations can be quite disturbing! [0] Very interesting how the patterns react to disturbances like rotation, and the animation is very smooth [0] https://en.wikipedia.org/wiki/Trypophobia https://en.wikipedia.org/wiki/Trypophobia
- udp 6y ago
  I usually have a very strong (feeling unbearably itchy) trypophobic response to anything "organic" that has clusters of protrusions or holes e.g. lotus seed pods, but none of these examples have any such effect. I think it's because of the bright colours and low resolution.
  - jointpdf 6y ago
    What about inorganic physical objects, like a shower head?
    - udp 6y ago
      No problem if it's inorganic. I expect there must be some kind of evolutionary reason for it - the fact that it's an itching/crawling feeling strongly suggests it has/had something to do with bugs.
  - hoppla 6y ago
    Ditto, it has to be organic for me. Usually holes with stuff in it.
k2enemy 6y ago
If you haven't heard or seen any presentations about the work coming out of the Levin lab, it is super interesting. I don't really know anything about biology, but the work around modifying organisms via changing electrical circuits rather than genes is fascinating, and to a lay-person such as myself seems like the future of bio. https://ase.tufts.edu/biology/labs/levin/presentations/ https://ase.tufts.edu/biology/labs/levin/presentations/
samwestdev 6y ago
How does it work?
EamonnMR 6y ago
This does a good job of illustrating how patterns in nature formed by cells can self organize. I haven't dug in to see how similar the implementation is, but when you look at the way these textures develop it sure looks like it.
hardmath123 6y ago
See also: using differentiable approximations of cellular automata in PyTorch to reverse Conway's Game of Life; in some cases, you can get striking Turing patterns similar to what's described in this paper! http://hardmath123.github.io/conways-gradient.html http://hardmath123.github.io/conways-gradient.html
taneq 6y ago
This reminds me of a shareware program I had way back in '98 or something. It let you generate (or evolve?) seamlessly tiling textures using a cellular automaton with a bunch of parameters. I remember it being really cool at the time but can't for the life of me remember what it was called.
- cma 6y ago
  Kai's Power Tools texture explorer? https://www.google.com/search?q=kai%27s+texture+explorer https://www.google.com/search?q=kai%27s+texture+explorer
colordrops 6y ago
This is awesome. Not sure why, but I was kinda disappointed to find our that it uses ML.
soheil 6y ago
What is the significance of this, e.g. can we use this approach to build arbitrary material or even living tissue? I can't help but think of this video [0], it seems there may be commonalities between what's happening in life and simple cellular automata. [0] https://www.youtube.com/watch?v=7Q9VyHJ1l2Q https://www.youtube.com/watch?v=7Q9VyHJ1l2Q
adamhp 6y ago
"It reaches out... it reaches out... it reaches out..." This is too cool.
danans 6y ago
> In the same way that cells form eye patterns on the wings of butterflies to excite neurons in the brains of predators, our NCA’s population of cells has learned to collaborate to produce a pattern that excites certain neurons in an external neural network. I know there has been other work on adversarial networks, but this analogy (along with the photo of the butterfly) really communicates the idea well. And although I'm generally skeptical of claims that ANN "x" is the true model of how the human brain works, it makes a lot of sense to me that this is how adversarial self-organizing biological structures interact. Also, it's a powerful example because of just how effective the butterfly wing's "eye" is. Despite understanding that it's a decoy, I still can't look at it and not be unnerved a bit by it.
enricozb 6y ago
This reminds me a lot of the WaveFunctionCollapse texture generation algorithm [0]. It "generates bitmaps that are locally similar to the input bitmap." Very cool! [0]: https://github.com/mxgmn/WaveFunctionCollapse https://github.com/mxgmn/WaveFunctionCollapse
deleted 6y ago
[deleted]
unparadoxed 6y ago
Very nice! The results look similar to my experiments in applying feedback to style transfer networks (https://www.youtube.com/watch?v=fGSXbYDpI9c https://www.youtube.com/watch?v=fGSXbYDpI9c), though the self-healing properties of CA make this more interesting!
March_f6 6y ago
This is amazing. As a complete ML/AI neophyte can someone suggests some books/resources to help me understand the basics of what is going on here?
f430 6y ago
is the animation periodically shaking? I was staring at it and it seem to quiver.
alokdhari 6y ago
Until next trip.
TeeMassive 6y ago
Reminds me of the art style transfer: https://youtu.be/-R9bJGNHltQ https://youtu.be/-R9bJGNHltQ
magicalhippo 6y ago
Very cool, and great demo. This really takes me back as I was really into graphics algorithms, including texture synthesis, in my younger years. The hexagonal grid seems to have some cell-boundary issues though, especially noticeable with the two radial cell alignments.

418 points by fenomas 6y ago

This is very interesting I observed that the more symmetric the basic structures of the pattern/texture are, the more stable the result is/the faster the automata converges. I wonder what it would take to stabilize the worst case I saw there, the veined leaf texture.

Moosdijk 6y ago

If you want to see similar work by some of the same authors, see [1]. The youtube channel "twominutepapers" has an explanation of this work[2]. [1] https://distill.pub/2020/growing-ca/ https://distill.pub/2020/growing-ca/ [2] https://www.youtube.com/watch?v=bXzauli1TyU https://www.youtube.com/watch?v=bXzauli1TyU

hans1729 6y ago
Shoutout to twominutepapers, it's a wonderful channel.
- deleted 6y ago
  [deleted]

ArtWomb 6y ago

Results resemble common micrographs. But perturbed in such a way as to appear alien. Appears we are on the cusp of "neural synthetic biology" ;) Fast differentiable DNA and protein sequence optimization for molecular design https://arxiv.org/abs/2005.11275 https://arxiv.org/abs/2005.11275 Regenerating Soft Robots through Neural Cellular Automata https://arxiv.org/abs/2102.02579 https://arxiv.org/abs/2102.02579

enchiridion 6y ago
Is a micrograph the same a motif from network theory?

willis936 6y ago

Nice approximation of nature. You can see both growth and statistical mechanics in the same demonstration.

jonahrd 6y ago

For some reason I find these quite disgusting to look at. I wonder why!

SamBam 6y ago
Besides the organic, slightly bacterial-colony forming nature of them, I wonder if you also have trypophobia. I feel like some of the images were triggering the same kind of feelings I get from that.
JosephRedfern 6y ago
The way the the images move during evolulution kinda reminds me of worms or maggots wriggling around in soil.
choxi 6y ago
https://en.wikipedia.org/wiki/Trypophobia https://en.wikipedia.org/wiki/Trypophobia was mentioned in another comment

JackFr 6y ago

This is the first I've ever read about neural cellular automata, though I thought I was relatively up to date on both cellular automata and deep learning. I think I was able to pick up the broad strokes from context, but is there a good introductory resource for neural cellular automata?

eyvindn 6y ago

Authors here. If you have any questions we'll do our best to answer them! Glad to see people find our work interesting thus far. We also encourage anyone interested to play with the linked Google Colabs [1][2] and read the other articles in the Distill thread. In the Colabs you'll find a bunch more pre-trained textures as well as a workflow to train on your own images, plus some of the scaffolding to recreate figures. [1] https://colab.sandbox.google.com/github/google-research/self-organising-systems/blob/master/notebooks/texture_nca_tf2.ipynb https://colab.sandbox.google.com/github/google-research/self... [2] https://colab.sandbox.google.com/github/google-research/self-organising-systems/blob/master/notebooks/texture_nca_pytorch.ipynb https://colab.sandbox.google.com/github/google-research/self...

zitterbewegung 6y ago
The textures remind me of the beginning of once in a lifetime by talking heads. https://www.youtube.com/watch?v=5IsSpAOD6K8 https://www.youtube.com/watch?v=5IsSpAOD6K8
- alanbernstein 6y ago
  Looks similar to the Gray-Scott model. https://mrob.com/pub/comp/xmorphia/ https://mrob.com/pub/comp/xmorphia/
layer8 6y ago
How large is the state space for each cell? Full 8-bit RGB (= 24 bits)?
- zzznah 6y ago
  Each cell has 12 8-bit channels, including rgb, so it is 96 bits.
  - yorwba 6y ago
    The article says "our NCA model contains 16 channels. The first three are visible RGB channels and the rest we treat as latent channels which are visible to adjacent cells during update steps, but excluded from loss functions."

Sebastian_09 6y ago

Warning for people suffering from trypophobia, some of the combinations can be quite disturbing! [0] Very interesting how the patterns react to disturbances like rotation, and the animation is very smooth [0] https://en.wikipedia.org/wiki/Trypophobia https://en.wikipedia.org/wiki/Trypophobia

udp 6y ago
I usually have a very strong (feeling unbearably itchy) trypophobic response to anything "organic" that has clusters of protrusions or holes e.g. lotus seed pods, but none of these examples have any such effect. I think it's because of the bright colours and low resolution.
- jointpdf 6y ago
  What about inorganic physical objects, like a shower head?
  - udp 6y ago
    No problem if it's inorganic. I expect there must be some kind of evolutionary reason for it - the fact that it's an itching/crawling feeling strongly suggests it has/had something to do with bugs.
- hoppla 6y ago
  Ditto, it has to be organic for me. Usually holes with stuff in it.

k2enemy 6y ago

If you haven't heard or seen any presentations about the work coming out of the Levin lab, it is super interesting. I don't really know anything about biology, but the work around modifying organisms via changing electrical circuits rather than genes is fascinating, and to a lay-person such as myself seems like the future of bio. https://ase.tufts.edu/biology/labs/levin/presentations/ https://ase.tufts.edu/biology/labs/levin/presentations/

samwestdev 6y ago

How does it work?

EamonnMR 6y ago

This does a good job of illustrating how patterns in nature formed by cells can self organize. I haven't dug in to see how similar the implementation is, but when you look at the way these textures develop it sure looks like it.

hardmath123 6y ago

See also: using differentiable approximations of cellular automata in PyTorch to reverse Conway's Game of Life; in some cases, you can get striking Turing patterns similar to what's described in this paper! http://hardmath123.github.io/conways-gradient.html http://hardmath123.github.io/conways-gradient.html

taneq 6y ago

This reminds me of a shareware program I had way back in '98 or something. It let you generate (or evolve?) seamlessly tiling textures using a cellular automaton with a bunch of parameters. I remember it being really cool at the time but can't for the life of me remember what it was called.

cma 6y ago
Kai's Power Tools texture explorer? https://www.google.com/search?q=kai%27s+texture+explorer https://www.google.com/search?q=kai%27s+texture+explorer

colordrops 6y ago

This is awesome. Not sure why, but I was kinda disappointed to find our that it uses ML.

soheil 6y ago

What is the significance of this, e.g. can we use this approach to build arbitrary material or even living tissue? I can't help but think of this video [0], it seems there may be commonalities between what's happening in life and simple cellular automata. [0] https://www.youtube.com/watch?v=7Q9VyHJ1l2Q https://www.youtube.com/watch?v=7Q9VyHJ1l2Q

adamhp 6y ago

"It reaches out... it reaches out... it reaches out..." This is too cool.

danans 6y ago

> In the same way that cells form eye patterns on the wings of butterflies to excite neurons in the brains of predators, our NCA’s population of cells has learned to collaborate to produce a pattern that excites certain neurons in an external neural network. I know there has been other work on adversarial networks, but this analogy (along with the photo of the butterfly) really communicates the idea well. And although I'm generally skeptical of claims that ANN "x" is the true model of how the human brain works, it makes a lot of sense to me that this is how adversarial self-organizing biological structures interact. Also, it's a powerful example because of just how effective the butterfly wing's "eye" is. Despite understanding that it's a decoy, I still can't look at it and not be unnerved a bit by it.

enricozb 6y ago

This reminds me a lot of the WaveFunctionCollapse texture generation algorithm [0]. It "generates bitmaps that are locally similar to the input bitmap." Very cool! [0]: https://github.com/mxgmn/WaveFunctionCollapse https://github.com/mxgmn/WaveFunctionCollapse

deleted 6y ago

[deleted]

unparadoxed 6y ago

Very nice! The results look similar to my experiments in applying feedback to style transfer networks (https://www.youtube.com/watch?v=fGSXbYDpI9c https://www.youtube.com/watch?v=fGSXbYDpI9c), though the self-healing properties of CA make this more interesting!

March_f6 6y ago

This is amazing. As a complete ML/AI neophyte can someone suggests some books/resources to help me understand the basics of what is going on here?

f430 6y ago

is the animation periodically shaking? I was staring at it and it seem to quiver.

alokdhari 6y ago

Until next trip.

TeeMassive 6y ago

Reminds me of the art style transfer: https://youtu.be/-R9bJGNHltQ https://youtu.be/-R9bJGNHltQ

magicalhippo 6y ago

Very cool, and great demo. This really takes me back as I was really into graphics algorithms, including texture synthesis, in my younger years. The hexagonal grid seems to have some cell-boundary issues though, especially noticeable with the two radial cell alignments.