9 ms·

Generative AI space and the mental imagery of alien minds

254 points by RafelMri 3y ago

Nevermark 3y ago
> I call it the ruliad. Think of it as the entangled limit of everything that is computationally possible: the result of following all possible computational rules in all possible ways. Quote from his link to ruliad. I think maybe what he is talking about is equivalent to what we usually call “mathematics”. Since mathematics actually only instantiates computable things. (It can represent uncomputable things as abstractions, but only at a level that is actually computable. I.e. everything we know about uncomputable reals, or higher order infinities, is a computable statement. So those concept representations are as computationally rule defined as anything else.)
- civilitty 3y ago
  > I think maybe what he is talking about is equivalent to what we usually called “mathematics”. And as we all know, “mathematics” was invented in its entirety by Stephen Wolfram, so sayeth Wolfram Alpha’s new science history DLC. The FTC has required that I disclose that this endorsement is paid for by the Wolfram Supervillain Support Program.
- antonkar 3y ago
  As far as I remember, the ruliad is digital - so it can only approximate with more and more precision analog things like circles. His basic idea is that we are made of and live inside of mathematics but we can only observe the part of it we call physics: https://www.wolframscience.com/metamathematics/ https://www.wolframscience.com/metamathematics/
  - Nevermark 3y ago
    Once you apply arbitrary rules, you get algebras, so natural numbers, integers, rational numbers, algebraic numbers, continuous fractions, geometry, topology, ... The view that the physics we experience is a subset of mathematics is obviously not proven, but not original or controversial. Wolfram has a lot of good ideas. Many of which already existed, but he obscures, renames or ignores that. He is a genuinely brilliant individual, and if he clearly demarcated what he did that was genuinely original, I think he would both get more respect, and be likelier to not distract himself from doing better work with his own grandiose abstractions. A lot of great work starts out by solving something small but highly original, and following up every loose end, so that you end up somewhere interesting that neither you or anyone else might have expected. Maybe somewhere profound. Starting out with grandiose publicly disclosed project definitions creates enormous pressure to confabulate.
yieldcrv 3y ago
I always find it interesting how a hero dose of LSD gives similar visuals to what these image AI's do to achieve a coherent image. on top of that the associated Hallucinogen Persisting Perception Disorder (HPPD) clinical condition seems to be more like an image processing issue, somewhere between the rods and cones (sensor) and memory (persistent storage). like an unwanted configuration change in a program or development environment. I feel like the more we get AI to act like humans, and the more those engineers and others use LSD, the more convergence we are going to have with curiosity and breakthroughs about how we function. More akin to biological machines without such unique mental faculties, just the illusion or hallucination of such.
- robblbobbl 3y ago
  Similar to a DMT Breakthrough: https://www.youtube.com/watch?v=RONAjJYB5bE https://www.youtube.com/watch?v=RONAjJYB5bE This is how our mind perceive or forms reality, ourselves and the objects around us in a huge floating chaos of elements. That's what we call order. Pretty fascinating and overwhelming.
deleted 3y ago
[deleted]
Der_Einzige 3y ago
There are a bunch of extensions in the Automatic1111 eco-system which can be used for doing this sort of analysis, or related types of analysis. One of my favorites is this: https://github.com/ljleb/prompt-fusion-extension https://github.com/ljleb/prompt-fusion-extension
avereveard 3y ago
There is a logic jump in the concept of the ruliad. First it goes: > the entangled limit of everything that is computationally possible: the result of following all possible computational rules in all possible ways And then > it encapsulates not only all formal possibilities but also everything about our physical universe The assumption that everything about our physical universe is described by computational rules is not a given, and is never explored nor justified. Even the assumption that our perception follows computational rules is not really explored, just given. There is this old idea that with an universal computer that knew everything, one could predict anything. But that has been challenged by uncertainty principles, observer effect and all that. So it seems this castle has sand foundations.
- Legend2440 3y ago
  This is a well-known conjecture called the Physical Church-Turing Thesis. It is just a conjecture, but emperically it has held true to the limits of our knowledge of physics. It also still works even if the universe is probabilistic.
  - p-e-w 3y ago
    > but emperically it has held true to the limits of our knowledge of physics Considering that our entire knowledge of physics is encoded as computational rules, because that's all we really know how to do, this statement is basically a tautology.
    - Legend2440 3y ago
      We do physics this way exactly because it works so well - it is "unreasonably effective", as the saying goes.
      - p-e-w 3y ago
        No, we do physics this way because we don't know any other way. When this way fails (e.g. quantum gravity, turbulence), we assume that the problem is that we haven't tried hard enough yet. Which may be true, but it's also possible that the computational approach simply doesn't work for those phenomena.
jacknews 3y ago
Always fascinating from Stephen Wolfram. In fact I didn't realise it to start with, but after I'd scrolled for half an eternity, and saw there was still an eternity of content to go (I guess that's how eternity works), I thought 'huh, it's really long, is this a Stephen Wolfram piece?'. I don't know how he does it, he must type in his sleep.
- teruakohatu 3y ago
  > I don't know how he does it, he must type in his sleep. You are close. He writes while he goes on walks and has had a number of writing on the go hardware setups over the years. https://writings.stephenwolfram.com/2019/02/seeking-the-productive-life-some-details-of-my-personal-infrastructure/ https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
  - mensetmanusman 3y ago
    I like to imagine him wandering in the Sahara desert typing away madly not realizing where he has gone while I sip my coffee.
Legend2440 3y ago
Keep in mind that this is all about diffusion models. The latent space of GANs is known to be structured differently - much smoother and more disentanglement of concepts. Very likely each type of model has a unique latent space.
- gwern 3y ago
  Yes, you wouldn't get anything like what he shows in all these pictures if you were walking a BigGAN or StyleGAN latent space rather than the 'latent space' of Stable Diffusion 1 (which isn't really a latent space to begin with). So that rather undermines any connections he might want to draw to grander things than just SD1.
tracnlf 3y ago
Is this "Ruliad" concept accepted anywhere else? I have a hard time reading Wolfram's expositions, which seem like GEB mysticism (everything is recursion), except here everything is a "Ruliad" (whatever that may be).
- tanepiper 3y ago
  "accepted" is maybe hard to push here, but the Ruliad isn't that different to the Vedic concept of Indra's Net - https://en.wikipedia.org/wiki/Indra%27s_net https://en.wikipedia.org/wiki/Indra%27s_net
tomohelix 3y ago
The first time I saw an AI creating images in like 2018 or so, I remembered the scientists called it "dreams" and I thought it was so appropriate. The constantly morphing landscape and how things blend together to form new things and how almost everything is wrong when look at closely. It was an interesting description, and I didn't care much more than that. Then the AI got better, and still the dreamlike things never go away. The constant problem with the hands, words never properly formed, clocks always look wrong, etc. But now it is more cohesive, more "solid" but still malleable. Like a lucid dream. Still straddling the border between consciousness and unconsciousness but there is a hint of control and direction now. As someone who lucid dream, I think the AI images and the LLMs are just like a person dreaming right now. They sort of have control, but actually not really. Kinda hard to explain but even when I lucid dream, I know it is a dream and I can bend it to my will but at the same time, it isn't possible to control my thoughts. Trying too hard to assert control and I wake up. So it is still a state of unconsciousness for me and not at all comparable to the "me" when I am fully awake. Of course, the AIs can't wake up if we use that analogy. They are not capable of anything more than this state right now. But to me, lucid dreaming is already a step above the total unconsciousness of just dreaming, or just nothing at all. And wakefulness always follows shortly after I lucid dream.
- arketyp 3y ago
  I feel like lucid dreaming is in a way the baseline and that when we're awake, reality constantly knocks us back into groundedness. In a lucid dream there is no error correcting signal to keep our fabulations in check. Things tend to drift away like the silhouettes of an infinity mirror. Reality is like the flash of solid pixels when switching away from the hysteresis stream in a shared window Zoom call.
  - anonylizard 3y ago
    Wow, absence of error correction explains why dreams are the way they are... No wonder my dreams always start out normal and mundane, but end up with extremely exotic events in the end (even though I don't realize it during the dream).
  - 3y ago
dclowd9901 3y ago
Ohhhh I fear this one has lost the plot. Our models have the be the furthest thing from an alien mind. Almost mathematically. A trained model with no hallucination is essentially an average of all training data. Which comes from us. Who are not aliens.
- 3cats-in-a-coat 3y ago
  Do not focus so much on the world alien.
- Sosh101 3y ago
  Alien as in "other".
deleted 3y ago
[deleted]
3cats-in-a-coat 3y ago
He says coherent spots are tiny specs, surrounded by bizarre patterns and eventually vast space of noise and dots. Reminds me of the Universe itself. So much empty space full of fine quantum vacuum noise, and planets and stars… small coherent dots in giant space. I pay attention to these coincidences, for good or bad.
ttctciyf 3y ago
> we can progressively increase the numerical values of the weights—eventually in some sense “blowing the mind” of the network (and going a bit “psychedelic” in the process) I wonder if there's a more exact analog of the action of psychedelics on the brain that could be performed on generative models? Andrew Gallimore's online course on psychedelic neurochemistry[1] argues that psychedelics disrupt something like a constant fine-tuning cycle taking place in normal perceptual activity, causing new activations between cortical structures where connections had previously been dormant (see [2] for an in-at-the-deep-end account), causing the generative model in the higher layers of the cortex[3] to adjust in an effort to regain predictive efficacy. I wonder if some such kind of "dynamic" generative mechanism might be needed to perturb an artificial model analogously? 1: https://www.youtube.com/playlist?list=PLbqdD4EM-aEfmLvbWu8GQDhUII236MZf- https://www.youtube.com/playlist?list=PLbqdD4EM-aEfmLvbWu8GQ... 2: https://www.youtube.com/watch?v=L45A6XlPRM0&list=PLbqdD4EM-aEfmLvbWu8GQDhUII236MZf-&index=36 https://www.youtube.com/watch?v=L45A6XlPRM0&list=PLbqdD4EM-a... 3: https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding
jojobas 3y ago
That's weird. A cat is a cat is a cat, any alien with a visual organ and a perfect ability to paint, would reproduce the same thing. Inside his mind he might have completely different associations with the image, and this weight manipulation trickery doesn't get us closer to those hypothetical associations.
- YurgenJurgensen 3y ago
  An infrared image of a cat an an X-ray image of a cat are drastically different to an optical one, so it's not a given that they'd all draw the same thing.
  - jojobas 3y ago
    Fair enough, still it's not what the authors impute at all.
  - NoMoreNicksLeft 3y ago
    If the alien is blasting the cat with soft x-rays constantly to perceive it, then it's mostly different because it killed the cat with radiation poisoning. Shapes and proportions remain the same regardless. And even human artists can be... "inventive", with coloration or emphasis of some features over others. It seems improbable that they'd draw anything we haven't drawn ourselves.
- twobitshifter 3y ago
  If you asked me to draw a cat in a party hat, I would get the number if ears legs and eyes correct, but my fur would be non-existent and there’s no way my cat eyes would glow in a reflection. The AI is thinking differently than humans.
Thorentis 3y ago
Meh? I don't really see this as anything special, or anything close to being an "alien mind". An alien mind would have been trained on its own environment with perhaps wildly different inputs. This is nothing more than a broken human mind. A mind trained on the same inputs as humans, but then altered haphazardly. This teaches us nothing and appears to be purely a gimmick.
Mizza 3y ago
I'm a Wolfram-skeptic, but I liked this article. I'd never considered the islands-surrounded-by-fuzz before, but it makes me wonder a few questions - is there a programmatic way to identify the "islands" (without using a separate NN classifier)? Is trimming the "fuzz" a potential method of compression for models?
- DougBTX 3y ago
  The dimensionality is a challenge here - these are 2D islands in a, what, 2000D space? Imagine a cross section of a cylinder, that would look like an island if it went through the cylinder, but not along it. I wonder what it would look like if the cross-section plane was aligned with a line between two recognisable points.
  - fnordpiglet 3y ago
    When discussing “n parameters,” such as 6B parameters or in the case of GPT4 around 1.7T is the rumored parameters size, each parameter is a dimension. So, yeah, that’s a lot more than 2d or even 2000d. More like 1,700,000,000,000D.
- m12k 3y ago
  Any attempt at visibility into the inner workings of ML models should be welcomed IMO. It's going to be essential in the coming years, if we're going to reason about or regulate them. E.g. how would we hardcode Asimov's laws of robotics into some future deep learning AGI, if it's still just one big black box for us?
  - kridsdale3 3y ago
    Thinking about law (As Asimov's are), I perceive essentially an impenetrable barrier in this possibility space. We create order from the chaos of potential behavior by defining limits. Currently, in this 2000D StableDiffusion landscape, there are no boundaries for allowable "travel" and you quickly end up in the "sea". So if we want AI to behave, we should research how to define the perimeter (in thousands / billions of dimensions) and "wall it off" so we can ensure the potentials inside the no-no space are inaccessible.
Attummm 3y ago
Is generative AI a alien mind when it's build synthetic neurons and the images and videos look akin to dreams?
cmilton 3y ago
This is such a fascinating evolution of technology for me. It reminds me a lot of my experience with psychedelics. The ability to see things in a much different way. To visualize the many possible ways an image may truly present to our brains without our learned constraints. Adjusting our model weights. Its beautiful. :)
nuclearsugar 3y ago
On the topic of exploring the innards of machine learning models, here are some visualizations I created of the hidden internal representation layers within the StyleGAN3 FFHQ1024 model. https://youtu.be/2aDeS_RFqHs https://youtu.be/2aDeS_RFqHs https://www.jasonfletcher.info/vjloops/index.html#internal-rep https://www.jasonfletcher.info/vjloops/index.html#internal-r...
- ShamelessC 3y ago
  All of your posts are fantastic. I’m amazed you have the patience for the data cleanup required for the plants finetune. I remember wanting to do a Pokemon finetune (before such things were commonplace) and the dataset cleanup was the worst part. Stability issues and lack of decent multi-GPU setup being a close second.
  - nuclearsugar 3y ago
    Thanks! Indeed the cleanup required for the 'Nature Artificial' dataset was intense and forced me to consider other techniques. It was around that time that I realized that I could use Stable Diffusion to output 10,000 to 50,000 images, which has proved to be a useful dataset creation tool for training StyleGAN2 or StyleGAN3. Yet nailing down a SD text prompt that will continuously output images in a specific style and context can be challenging. StyleGAN2/3 is quite sensitive to fine-tuning models with a dataset that contains lots of variation, such as the Pokemon dataset. But it will converge much better with a dataset of just Pikachu images in tons of different poses. It seemingly wants to find the most common pattern in the dataset and interpolate within that space. From there nailing down the ideal value for the gamma attribute is difficult, but luckily can be reduced every 1000 to 2000kimg until the model no longer trains favorably. Currently my typical gamma strategy is: 80 > 10 > 5 > 2
- sdwr 3y ago
  Incredible! You should make music videos
Applejinx 3y ago
Trust Stephen Wolfram to get the idea behind this. I agree completely. My own work in Stable Diffusion is way less about prompt engineering or approximating 'human art' and much more in exploring its capacity for vivid and plausible hallucination, likewise the beginnings of my experiments with LLAMA. There is a kind of meaning to these spaces when you can compellingly imply something with purpose and integrity, but it's not prescriptive, or it's not a human thought. I'm looking for the evocative, but I'm willing to be a lot more collaborative about what it evokes… people's explorations into negative prompting are fascinating to me, even though I think it's something of a false path. Telling AIs to replace human artists and follow instructions while aping their styles is not useful behavior (except economically, and even then it's degenerative). Using AI to do what you'd have to inflict brain damage on an artist to do, is more interesting and ethical :)
- fnordpiglet 3y ago
  I think these tools are much more like a calculator for what is the mechanical part of the human arts - writing, visualization, and others (beyond that I think those use cases are not that interesting, there’s a whole non linear optimization angle I think gets lost in the LLM noise). The “art” they produce on their own with minimal ingenuity is boring. But they also democratize the mechanical skill of rendering images and language, and the intent and manipulation of the semantic space through language, inpainting, LORA, etc - that’s all human induced and human creativity. I think we will find in our future we will be awash in brilliant human arts, assisted by machines, being used to amplify the creativity of people who were before hindered by their skill at the craft of art or the mechanics of writing, much as many brilliant mathematicians and physicists are poor at arithmetic and are greatly assisted by machines in their day to day work.
keepamovin 3y ago
Interesting how some of these is very similar to 20th C art movements like Cubism, Impressionism, Pointillism and Constructivism. Picasso et al Many of which were very much actively practicing altering, expanding or engaging with different aspects or modes of perception. (edit: also funny how art has its own type of elaborate legalese-type language to (deliberately im-)precisely capture its diverse and expansive meanings)
netbioserror 3y ago
Unsure why he needs to adopt the unnecessary axiom of calling existing models "human-aligned". If they were trained from random noise, fresh deep neural nets are the closest things we have to blank slates, which are utterly alien to us and the animal world. Every neural net in nature comes with a bevy of pre-baked patterns that form in the womb and, for humans, continue forming outside of it.
- prometheus76 3y ago
  I think the reason Stephen Wolfram is using "human-aligned" is because that is the only feedback loop that is training the models. If you haven't read "What is it like to be a bat?" by Thomas Nagel, I highly recommend it.
sdwr 3y ago
Can see Wolfram's approach here - interesting how he's coming to generative AI with a beginner's mind, from first principles. He makes a lot of interesting points, but I can't help and think that he would save a lot of time talking to someone who knows the field better, instead of figuring it all out on his own. The stuff about attractors and dimensions in model space is missing the architectural understanding (layers representing features, backprop)
- thrashh 3y ago
  Sometimes it's nice starting from scratch because once you learn something, your thinking is "corrupted" and your take isn't very fresh anymore.
joe_the_user 3y ago
I don't think it's impossible for these images to have some resonance with human dreams and altered states of consciousness. However, the phrase "mental imagery of alien minds" seems to carry unjustified conceptual baggage. The usual implication of "mind" is something like a process that includes intentions, meta-understandings and etc (varying depending on one's conceptions but usually having these). The processes that generate things could be described as purely algorithmic - or they might have further implications. But the thing there's no proof or even strong indicator of the further implications.
mk_stjames 3y ago
> There’s also a strange preponderance of torso-only pictures—presumably the result of “fashion shots” in the training data (and, yes, with some rather wild “fashion statements”): Actually I think this is due to the staggered cropping stage on the input data- the images were all cropped to square, and if an image, like of a person, was taller than wide, the image would get cropped into a few square crops, with one of those being just the center of the image, which, for people in a tall portrait shaped photo standing up, will leave just the torso. There was a blurb in the SDXL paper about changing the cropping preprocessing to alleviate this potential biasing.
- gwern 3y ago
  Correct. Very well known artifact, you see it in all the GANs and diffusion models depending on the choice to do center vs random crop. Often changes results by several FID points (random crop adds a lot of variations and is much harder to model, confusing the NNs), which is why I suggested the cropping conditioning idea that SDXL uses with excellent results. (This is another example of how many of OP's observations are essentially minor, contingent aspects of the data or arch, and don't generalize beyond SD1.)

254 points by RafelMri 3y ago

> I call it the ruliad. Think of it as the entangled limit of everything that is computationally possible: the result of following all possible computational rules in all possible ways. Quote from his link to ruliad. I think maybe what he is talking about is equivalent to what we usually call “mathematics”. Since mathematics actually only instantiates computable things. (It can represent uncomputable things as abstractions, but only at a level that is actually computable. I.e. everything we know about uncomputable reals, or higher order infinities, is a computable statement. So those concept representations are as computationally rule defined as anything else.)

civilitty 3y ago
> I think maybe what he is talking about is equivalent to what we usually called “mathematics”. And as we all know, “mathematics” was invented in its entirety by Stephen Wolfram, so sayeth Wolfram Alpha’s new science history DLC. The FTC has required that I disclose that this endorsement is paid for by the Wolfram Supervillain Support Program.
antonkar 3y ago
As far as I remember, the ruliad is digital - so it can only approximate with more and more precision analog things like circles. His basic idea is that we are made of and live inside of mathematics but we can only observe the part of it we call physics: https://www.wolframscience.com/metamathematics/ https://www.wolframscience.com/metamathematics/
- Nevermark 3y ago
  Once you apply arbitrary rules, you get algebras, so natural numbers, integers, rational numbers, algebraic numbers, continuous fractions, geometry, topology, ... The view that the physics we experience is a subset of mathematics is obviously not proven, but not original or controversial. Wolfram has a lot of good ideas. Many of which already existed, but he obscures, renames or ignores that. He is a genuinely brilliant individual, and if he clearly demarcated what he did that was genuinely original, I think he would both get more respect, and be likelier to not distract himself from doing better work with his own grandiose abstractions. A lot of great work starts out by solving something small but highly original, and following up every loose end, so that you end up somewhere interesting that neither you or anyone else might have expected. Maybe somewhere profound. Starting out with grandiose publicly disclosed project definitions creates enormous pressure to confabulate.

yieldcrv 3y ago

I always find it interesting how a hero dose of LSD gives similar visuals to what these image AI's do to achieve a coherent image. on top of that the associated Hallucinogen Persisting Perception Disorder (HPPD) clinical condition seems to be more like an image processing issue, somewhere between the rods and cones (sensor) and memory (persistent storage). like an unwanted configuration change in a program or development environment. I feel like the more we get AI to act like humans, and the more those engineers and others use LSD, the more convergence we are going to have with curiosity and breakthroughs about how we function. More akin to biological machines without such unique mental faculties, just the illusion or hallucination of such.

robblbobbl 3y ago
Similar to a DMT Breakthrough: https://www.youtube.com/watch?v=RONAjJYB5bE https://www.youtube.com/watch?v=RONAjJYB5bE This is how our mind perceive or forms reality, ourselves and the objects around us in a huge floating chaos of elements. That's what we call order. Pretty fascinating and overwhelming.

deleted 3y ago

[deleted]

Der_Einzige 3y ago

There are a bunch of extensions in the Automatic1111 eco-system which can be used for doing this sort of analysis, or related types of analysis. One of my favorites is this: https://github.com/ljleb/prompt-fusion-extension https://github.com/ljleb/prompt-fusion-extension

avereveard 3y ago

There is a logic jump in the concept of the ruliad. First it goes: > the entangled limit of everything that is computationally possible: the result of following all possible computational rules in all possible ways And then > it encapsulates not only all formal possibilities but also everything about our physical universe The assumption that everything about our physical universe is described by computational rules is not a given, and is never explored nor justified. Even the assumption that our perception follows computational rules is not really explored, just given. There is this old idea that with an universal computer that knew everything, one could predict anything. But that has been challenged by uncertainty principles, observer effect and all that. So it seems this castle has sand foundations.

Legend2440 3y ago
This is a well-known conjecture called the Physical Church-Turing Thesis. It is just a conjecture, but emperically it has held true to the limits of our knowledge of physics. It also still works even if the universe is probabilistic.
- p-e-w 3y ago
  > but emperically it has held true to the limits of our knowledge of physics Considering that our entire knowledge of physics is encoded as computational rules, because that's all we really know how to do, this statement is basically a tautology.
  - Legend2440 3y ago
    We do physics this way exactly because it works so well - it is "unreasonably effective", as the saying goes.
    - p-e-w 3y ago
      No, we do physics this way because we don't know any other way. When this way fails (e.g. quantum gravity, turbulence), we assume that the problem is that we haven't tried hard enough yet. Which may be true, but it's also possible that the computational approach simply doesn't work for those phenomena.

jacknews 3y ago

Always fascinating from Stephen Wolfram. In fact I didn't realise it to start with, but after I'd scrolled for half an eternity, and saw there was still an eternity of content to go (I guess that's how eternity works), I thought 'huh, it's really long, is this a Stephen Wolfram piece?'. I don't know how he does it, he must type in his sleep.

teruakohatu 3y ago
> I don't know how he does it, he must type in his sleep. You are close. He writes while he goes on walks and has had a number of writing on the go hardware setups over the years. https://writings.stephenwolfram.com/2019/02/seeking-the-productive-life-some-details-of-my-personal-infrastructure/ https://writings.stephenwolfram.com/2019/02/seeking-the-prod...
- mensetmanusman 3y ago
  I like to imagine him wandering in the Sahara desert typing away madly not realizing where he has gone while I sip my coffee.

Legend2440 3y ago

Keep in mind that this is all about diffusion models. The latent space of GANs is known to be structured differently - much smoother and more disentanglement of concepts. Very likely each type of model has a unique latent space.

gwern 3y ago
Yes, you wouldn't get anything like what he shows in all these pictures if you were walking a BigGAN or StyleGAN latent space rather than the 'latent space' of Stable Diffusion 1 (which isn't really a latent space to begin with). So that rather undermines any connections he might want to draw to grander things than just SD1.

tracnlf 3y ago

Is this "Ruliad" concept accepted anywhere else? I have a hard time reading Wolfram's expositions, which seem like GEB mysticism (everything is recursion), except here everything is a "Ruliad" (whatever that may be).

tanepiper 3y ago
"accepted" is maybe hard to push here, but the Ruliad isn't that different to the Vedic concept of Indra's Net - https://en.wikipedia.org/wiki/Indra%27s_net https://en.wikipedia.org/wiki/Indra%27s_net

tomohelix 3y ago

The first time I saw an AI creating images in like 2018 or so, I remembered the scientists called it "dreams" and I thought it was so appropriate. The constantly morphing landscape and how things blend together to form new things and how almost everything is wrong when look at closely. It was an interesting description, and I didn't care much more than that. Then the AI got better, and still the dreamlike things never go away. The constant problem with the hands, words never properly formed, clocks always look wrong, etc. But now it is more cohesive, more "solid" but still malleable. Like a lucid dream. Still straddling the border between consciousness and unconsciousness but there is a hint of control and direction now. As someone who lucid dream, I think the AI images and the LLMs are just like a person dreaming right now. They sort of have control, but actually not really. Kinda hard to explain but even when I lucid dream, I know it is a dream and I can bend it to my will but at the same time, it isn't possible to control my thoughts. Trying too hard to assert control and I wake up. So it is still a state of unconsciousness for me and not at all comparable to the "me" when I am fully awake. Of course, the AIs can't wake up if we use that analogy. They are not capable of anything more than this state right now. But to me, lucid dreaming is already a step above the total unconsciousness of just dreaming, or just nothing at all. And wakefulness always follows shortly after I lucid dream.

arketyp 3y ago
I feel like lucid dreaming is in a way the baseline and that when we're awake, reality constantly knocks us back into groundedness. In a lucid dream there is no error correcting signal to keep our fabulations in check. Things tend to drift away like the silhouettes of an infinity mirror. Reality is like the flash of solid pixels when switching away from the hysteresis stream in a shared window Zoom call.
- anonylizard 3y ago
  Wow, absence of error correction explains why dreams are the way they are... No wonder my dreams always start out normal and mundane, but end up with extremely exotic events in the end (even though I don't realize it during the dream).
- 3y ago

dclowd9901 3y ago

Ohhhh I fear this one has lost the plot. Our models have the be the furthest thing from an alien mind. Almost mathematically. A trained model with no hallucination is essentially an average of all training data. Which comes from us. Who are not aliens.

3cats-in-a-coat 3y ago
Do not focus so much on the world alien.
Sosh101 3y ago
Alien as in "other".

deleted 3y ago

[deleted]

3cats-in-a-coat 3y ago

He says coherent spots are tiny specs, surrounded by bizarre patterns and eventually vast space of noise and dots. Reminds me of the Universe itself. So much empty space full of fine quantum vacuum noise, and planets and stars… small coherent dots in giant space. I pay attention to these coincidences, for good or bad.

ttctciyf 3y ago

> we can progressively increase the numerical values of the weights—eventually in some sense “blowing the mind” of the network (and going a bit “psychedelic” in the process) I wonder if there's a more exact analog of the action of psychedelics on the brain that could be performed on generative models? Andrew Gallimore's online course on psychedelic neurochemistry[1] argues that psychedelics disrupt something like a constant fine-tuning cycle taking place in normal perceptual activity, causing new activations between cortical structures where connections had previously been dormant (see [2] for an in-at-the-deep-end account), causing the generative model in the higher layers of the cortex[3] to adjust in an effort to regain predictive efficacy. I wonder if some such kind of "dynamic" generative mechanism might be needed to perturb an artificial model analogously? 1: https://www.youtube.com/playlist?list=PLbqdD4EM-aEfmLvbWu8GQDhUII236MZf- https://www.youtube.com/playlist?list=PLbqdD4EM-aEfmLvbWu8GQ... 2: https://www.youtube.com/watch?v=L45A6XlPRM0&list=PLbqdD4EM-aEfmLvbWu8GQDhUII236MZf-&index=36 https://www.youtube.com/watch?v=L45A6XlPRM0&list=PLbqdD4EM-a... 3: https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding

jojobas 3y ago

That's weird. A cat is a cat is a cat, any alien with a visual organ and a perfect ability to paint, would reproduce the same thing. Inside his mind he might have completely different associations with the image, and this weight manipulation trickery doesn't get us closer to those hypothetical associations.

YurgenJurgensen 3y ago
An infrared image of a cat an an X-ray image of a cat are drastically different to an optical one, so it's not a given that they'd all draw the same thing.
- jojobas 3y ago
  Fair enough, still it's not what the authors impute at all.
- NoMoreNicksLeft 3y ago
  If the alien is blasting the cat with soft x-rays constantly to perceive it, then it's mostly different because it killed the cat with radiation poisoning. Shapes and proportions remain the same regardless. And even human artists can be... "inventive", with coloration or emphasis of some features over others. It seems improbable that they'd draw anything we haven't drawn ourselves.
twobitshifter 3y ago
If you asked me to draw a cat in a party hat, I would get the number if ears legs and eyes correct, but my fur would be non-existent and there’s no way my cat eyes would glow in a reflection. The AI is thinking differently than humans.

Thorentis 3y ago

Meh? I don't really see this as anything special, or anything close to being an "alien mind". An alien mind would have been trained on its own environment with perhaps wildly different inputs. This is nothing more than a broken human mind. A mind trained on the same inputs as humans, but then altered haphazardly. This teaches us nothing and appears to be purely a gimmick.

Mizza 3y ago

I'm a Wolfram-skeptic, but I liked this article. I'd never considered the islands-surrounded-by-fuzz before, but it makes me wonder a few questions - is there a programmatic way to identify the "islands" (without using a separate NN classifier)? Is trimming the "fuzz" a potential method of compression for models?

DougBTX 3y ago
The dimensionality is a challenge here - these are 2D islands in a, what, 2000D space? Imagine a cross section of a cylinder, that would look like an island if it went through the cylinder, but not along it. I wonder what it would look like if the cross-section plane was aligned with a line between two recognisable points.
- fnordpiglet 3y ago
  When discussing “n parameters,” such as 6B parameters or in the case of GPT4 around 1.7T is the rumored parameters size, each parameter is a dimension. So, yeah, that’s a lot more than 2d or even 2000d. More like 1,700,000,000,000D.
m12k 3y ago
Any attempt at visibility into the inner workings of ML models should be welcomed IMO. It's going to be essential in the coming years, if we're going to reason about or regulate them. E.g. how would we hardcode Asimov's laws of robotics into some future deep learning AGI, if it's still just one big black box for us?
- kridsdale3 3y ago
  Thinking about law (As Asimov's are), I perceive essentially an impenetrable barrier in this possibility space. We create order from the chaos of potential behavior by defining limits. Currently, in this 2000D StableDiffusion landscape, there are no boundaries for allowable "travel" and you quickly end up in the "sea". So if we want AI to behave, we should research how to define the perimeter (in thousands / billions of dimensions) and "wall it off" so we can ensure the potentials inside the no-no space are inaccessible.

Attummm 3y ago

Is generative AI a alien mind when it's build synthetic neurons and the images and videos look akin to dreams?

cmilton 3y ago

This is such a fascinating evolution of technology for me. It reminds me a lot of my experience with psychedelics. The ability to see things in a much different way. To visualize the many possible ways an image may truly present to our brains without our learned constraints. Adjusting our model weights. Its beautiful. :)

nuclearsugar 3y ago

On the topic of exploring the innards of machine learning models, here are some visualizations I created of the hidden internal representation layers within the StyleGAN3 FFHQ1024 model. https://youtu.be/2aDeS_RFqHs https://youtu.be/2aDeS_RFqHs https://www.jasonfletcher.info/vjloops/index.html#internal-rep https://www.jasonfletcher.info/vjloops/index.html#internal-r...

ShamelessC 3y ago
All of your posts are fantastic. I’m amazed you have the patience for the data cleanup required for the plants finetune. I remember wanting to do a Pokemon finetune (before such things were commonplace) and the dataset cleanup was the worst part. Stability issues and lack of decent multi-GPU setup being a close second.
- nuclearsugar 3y ago
  Thanks! Indeed the cleanup required for the 'Nature Artificial' dataset was intense and forced me to consider other techniques. It was around that time that I realized that I could use Stable Diffusion to output 10,000 to 50,000 images, which has proved to be a useful dataset creation tool for training StyleGAN2 or StyleGAN3. Yet nailing down a SD text prompt that will continuously output images in a specific style and context can be challenging. StyleGAN2/3 is quite sensitive to fine-tuning models with a dataset that contains lots of variation, such as the Pokemon dataset. But it will converge much better with a dataset of just Pikachu images in tons of different poses. It seemingly wants to find the most common pattern in the dataset and interpolate within that space. From there nailing down the ideal value for the gamma attribute is difficult, but luckily can be reduced every 1000 to 2000kimg until the model no longer trains favorably. Currently my typical gamma strategy is: 80 > 10 > 5 > 2
sdwr 3y ago
Incredible! You should make music videos

Applejinx 3y ago

Trust Stephen Wolfram to get the idea behind this. I agree completely. My own work in Stable Diffusion is way less about prompt engineering or approximating 'human art' and much more in exploring its capacity for vivid and plausible hallucination, likewise the beginnings of my experiments with LLAMA. There is a kind of meaning to these spaces when you can compellingly imply something with purpose and integrity, but it's not prescriptive, or it's not a human thought. I'm looking for the evocative, but I'm willing to be a lot more collaborative about what it evokes… people's explorations into negative prompting are fascinating to me, even though I think it's something of a false path. Telling AIs to replace human artists and follow instructions while aping their styles is not useful behavior (except economically, and even then it's degenerative). Using AI to do what you'd have to inflict brain damage on an artist to do, is more interesting and ethical :)

fnordpiglet 3y ago
I think these tools are much more like a calculator for what is the mechanical part of the human arts - writing, visualization, and others (beyond that I think those use cases are not that interesting, there’s a whole non linear optimization angle I think gets lost in the LLM noise). The “art” they produce on their own with minimal ingenuity is boring. But they also democratize the mechanical skill of rendering images and language, and the intent and manipulation of the semantic space through language, inpainting, LORA, etc - that’s all human induced and human creativity. I think we will find in our future we will be awash in brilliant human arts, assisted by machines, being used to amplify the creativity of people who were before hindered by their skill at the craft of art or the mechanics of writing, much as many brilliant mathematicians and physicists are poor at arithmetic and are greatly assisted by machines in their day to day work.

keepamovin 3y ago

Interesting how some of these is very similar to 20th C art movements like Cubism, Impressionism, Pointillism and Constructivism. Picasso et al Many of which were very much actively practicing altering, expanding or engaging with different aspects or modes of perception. (edit: also funny how art has its own type of elaborate legalese-type language to (deliberately im-)precisely capture its diverse and expansive meanings)

netbioserror 3y ago

Unsure why he needs to adopt the unnecessary axiom of calling existing models "human-aligned". If they were trained from random noise, fresh deep neural nets are the closest things we have to blank slates, which are utterly alien to us and the animal world. Every neural net in nature comes with a bevy of pre-baked patterns that form in the womb and, for humans, continue forming outside of it.

prometheus76 3y ago
I think the reason Stephen Wolfram is using "human-aligned" is because that is the only feedback loop that is training the models. If you haven't read "What is it like to be a bat?" by Thomas Nagel, I highly recommend it.

sdwr 3y ago

Can see Wolfram's approach here - interesting how he's coming to generative AI with a beginner's mind, from first principles. He makes a lot of interesting points, but I can't help and think that he would save a lot of time talking to someone who knows the field better, instead of figuring it all out on his own. The stuff about attractors and dimensions in model space is missing the architectural understanding (layers representing features, backprop)

thrashh 3y ago
Sometimes it's nice starting from scratch because once you learn something, your thinking is "corrupted" and your take isn't very fresh anymore.

joe_the_user 3y ago

I don't think it's impossible for these images to have some resonance with human dreams and altered states of consciousness. However, the phrase "mental imagery of alien minds" seems to carry unjustified conceptual baggage. The usual implication of "mind" is something like a process that includes intentions, meta-understandings and etc (varying depending on one's conceptions but usually having these). The processes that generate things could be described as purely algorithmic - or they might have further implications. But the thing there's no proof or even strong indicator of the further implications.

mk_stjames 3y ago

> There’s also a strange preponderance of torso-only pictures—presumably the result of “fashion shots” in the training data (and, yes, with some rather wild “fashion statements”): Actually I think this is due to the staggered cropping stage on the input data- the images were all cropped to square, and if an image, like of a person, was taller than wide, the image would get cropped into a few square crops, with one of those being just the center of the image, which, for people in a tall portrait shaped photo standing up, will leave just the torso. There was a blurb in the SDXL paper about changing the cropping preprocessing to alleviate this potential biasing.

gwern 3y ago
Correct. Very well known artifact, you see it in all the GANs and diffusion models depending on the choice to do center vs random crop. Often changes results by several FID points (random crop adds a lot of variations and is much harder to model, confusing the NNs), which is why I suggested the cropping conditioning idea that SDXL uses with excellent results. (This is another example of how many of OP's observations are essentially minor, contingent aspects of the data or arch, and don't generalize beyond SD1.)