5 ms·

The amount of creativity here is astounding. Just imagine all the decisions the AI made in incorporating Kermit into the movies: the clothing it's chosen, how t

The amount of creativity here is astounding. Just imagine all the decisions the AI made in incorporating Kermit into the movies: the clothing it's chosen, how the character wears the clothing, the facial expressions, how to make Kermit himself look similar to the other movies characters. Should he be lanky? pudgy? Even simple decisions like obliviously Kermit in Wall-E is going to be a robot, it has to figure out what he looks like as a robot, what his mouth looks like, that his eyes should be enlarged. It gets a lot of things wrong, like I'm not sure why kermit has a plastic texture in many of the pictures. If you showed me ten pictures of Kermit and ten frames of total recall, and for some reason 8 of your pictures had a plastic Kermit, and asked me to combine them in my head, I'd probably imagine something on-par or worse than what Dalle has managed to do. But I wouldn't be able to show anyone what I'd made!

tdehnel 4y ago
Sorry to be this guy but that is not creativity. It’s using what already exists, not conjecturing something new. Contrast with real creativity (what people can do but machines currently cannot) where you conjecture something completely new. For example, Copernicus conjecturing the idea that the Earth revolves around the Sun. No machine learning model would have gotten there because it would have been trained on a bunch of data that said the Earth was the center of the universe.
- dougmwne 4y ago
  These are fun discussions because words like "artistic creativity" have a colloquial meaning that could only apply to humans since the dawn of humanity. Now you have an image of Kermit in Wall-E. I have never seen or conceived of an image of Kermit in Wall-E. Let's assume that adorable robot Kermits do not exist in the training data to be spit out like a search algorithm. The image is new, it did not previously exist. It is a creation, a very vague idea of a few words that was created in full realization. So it sees like the only difference between the "Not creativity" that Dall-E is doing and "Real Creativity" that humans do is tht humans are the ones doing it? I agree there's this concept of expanding the frontiers of human aesthetic capability that has slow-marched from cave paintings till post-modernism. That there are a very few artists that invent completely new styles that the rest of us copy and remix. It's questionable that Dall-E can do that, but I'm also not sure that it can't do that.
  - orbital-decay 4y ago
    >So it sees like the only difference between the "Not creativity" that Dall-E is doing and "Real Creativity" that humans do is tht humans are the ones doing it? The differentiator is whether the result is worthy to look at for humans, that's all. In case of the OP, you forget that the human had to predict that the combination of two would be interesting for other humans, and then construct the prompt, possibly selecting the best pictures. That's who did most of the work here, and it was effortless for the neural network of a human. Could DALL-E analyze the world autonomously without human intervention at all? No, it's an open loop system. Novelty hinges on the ability to conceptualize things, not the execution. Sure, DALL-E 2 shows a glimpse of conceptualization internally as it works with compressed descriptions (concepts, abstractions) of things it draws. But it's super limited and not flexible enough to create new ideas, it doesn't have either short-term or long-term memory, it doesn't change, it has all knowledge about Kermit and Blade Runner pre-baked, and so on. You have to re-train it from scratch every time you want it to remember something truly new, there's no feedback loop to do that. Human ability is still much more powerful. DALL-E 2 is almost at the point where it can supplement the human conceptualization with AI execution, though. Possibly in a couple iterations it will be there, with more believable results. In very limited cases, of course - as it's temporally unstable (it makes a totally new image each time), cannot correct the output from new details provided by a human (like a professional concept artist could), etc.
    - treesprite82 4y ago
      > you forget that the human had to predict that the combination of two would be interesting for other humans, and then construct the prompt, possibly selecting the best pictures. That's who did most of the work here If the Twitter user claimed that the text prompts themselves were generated by asking GPT-3 for "An interesting sequence of text prompts to feed an image generation AI" or something like that, I would have believed them. Presumptively, I imagine it's harder to create a model that generates images matching a certain human-language input prompt than to create an image generation model with no language component and have it pick its own scenarios internally. I don't think the former is done to palm off "most of the work" to humans, but rather because people want an easy way to see create own ideas so there's more demand for it. > You have to re-train it from scratch every time you want it to remember something truly new, there's no feedback loop to do that. As far as I'm aware, this isn't true. Deep learning is perfectly compatible with fine-tuning an existing model using new data. OpenAI/MS have been doing this with Codex to improve it based on Copilot telemetry and code from new languages/libraries.
- andrewstuart2 4y ago
  The scientific and creative methods are both entirely centered around evaluating data or existing creative elements and combining it in novel ways. Science adds the goal of making and testing hypotheses but nothing about Copernicus's conjecture would have been possible without observing existing information. I'm including e.g. "new" astronomical observations in the realm of existing information because even a human is simply observing and measuring it. I fail to see what the difference is between 99% of existing "creativity," which is essentially arranging existing ideas into novel combinations, and what DALLE2 does.
- gwern 4y ago
  > No machine learning model https://www.nature.com/articles/d41586-019-03332-7 https://www.nature.com/articles/d41586-019-03332-7
- pphysch 4y ago
  Humans don't generate new ideas from nothing. Your "real creativity" doesn't exist. Everything is dependent on what came before, and therefore derivative to some extent. Copernicus got his idea after gathering a lot of data, explicitly and implicitly, training his internal model of the world.
  - shrimp_emoji 4y ago
    Even when it may appear that a human brain "generated something from nothing" (that is: you completely fail to account from where it could have possibly derived its output), you can always fall back to "genetic memory". Even a newborn has a lot going on that was "pre-programmed" by genetics, like a form of hardcoded training. :p
- adamsmith143 4y ago
  >Sorry to be this guy but that is not creativity. It’s using what already exists, not conjecturing something new. Such a cute point of view, completely wrong but cute. Please go find the original images of Kermit in Blade Runner and WallE that were just copied here. >For example, Copernicus conjecturing the idea that the Earth revolves around the Sun. No machine learning model would have gotten there because it would have been trained on a bunch of data that said the Earth was the center of the universe. If the model were trained on actual observations of planetary trajectories it would trivially recreate keplers laws, newtons laws etc.
  - soperj 4y ago
    >If the model were trained on actual observations of planetary trajectories it would trivially recreate keplers laws, newtons laws etc. Should be easy to prove.
    - ben_w 4y ago
      Indeed: • https://news.ycombinator.com/item?id=31609839 https://news.ycombinator.com/item?id=31609839 • https://arxiv.org/pdf/1810.10525.pdf https://arxiv.org/pdf/1810.10525.pdf • https://arxiv.org/pdf/1905.11481.pdf https://arxiv.org/pdf/1905.11481.pdf
      - soperj 4y ago
        from your link: "Renner stresses that although the algorithm derived the formulae, a human eye is needed to interpret the equations and understand how they relate to the movement of planets around the Sun."
        ben_w 4y ago
        Even if that had been the only example rather than just one among them, it would still demonstrate the claim that an AI can discover various laws of physics just from observing data.
  - tdehnel 4y ago
- klik99 4y ago
  "Good artists borrow, great artists steal" Creativity is a very vague word, I'm sure we can come up with definitions of it that let humans keep sole domain over it. But breakthroughs often come from combining domains and concepts, very very rarely do we ever jump out of one local maxima into another, and I'm not even convinced that Copernicus counts as that. There's a reason why there are so many examples of the same breakthrough happening in multiple places in the world independently - innovation is a slow gradual collaborative process and not plateaus waiting for men of genius to have a spark of inspiration. Also I'm not convinced that a computer couldn't have discovered the earth revolves around the sun - it's hard to make machine learning jump out of local maxima, but it does happen, and I can see some hidden layers becoming far more efficient at predicting outcomes by stumbling across a model that centered the sun. That being said - there likely are examples of things that computers couldn't have theoretically figured out the model for, but I'm hard pressed to think of one.
  - soperj 4y ago
    That is actually a misquote from people who stole.
    - kelseyfrog 4y ago
      That's great.
- vouwfietsman 4y ago
  This is a common fallacy. Call it moving goalposts, no true Scottsman, the AI effect, whatever. The behavior is as follows: an argument over whether an ill-defined attribute is possessed by a computer is defended or attacked with useless semantics since nobody can agree on what any of the words mean anyway. Creativity, intelligence, consciousness. It doesn't matter what you say, you cannot define these concepts with the same clarity you use to defend that the concept is missing. Saying there is no creativity because its just a neural net extrapolating from data is like saying there's no god because its all just atoms: what is god and why would the existence of atoms have anything to do with it. Learn from Wittgenstein: Worüber man nicht sprechen kann, darüber muss man schweigen.
  - tdehnel 4y ago
    This single quote from Wittgenstein might just be the dumbest statement ever made. How would we ever make progress if we did not discuss what we don't understand, and critique and debate new creative ideas?
    - vouwfietsman 4y ago
      By not taking the quote literally, but trying to understand it in its context. The quote is a judgement about what can be logically achieved through discourse, and what cannot (and should not). He draws this boundary to separate the things we can reason about and the things that fall beyond reason and so cannot be reasoned about at all. If you cannot reason about something, you should not bring it up in (philosophical) discourse. He does this to settle philosophical debates around, for instance, mystical subjects that were fought over to exist or not exist. The conclusion for him is: we do not have the logical tools and concepts to reason about the problem, so we should stop wasting time talking about it. It's not about lack of knowledge or intelligence, its about acknowledging the limits of reason and discourse, and forgoing any conclusions made beyond those limits. It's a pursuit of truth. At least, this is how I understand it, I'm not a philosopher.
      - tdehnel 4y ago
        It’s not controversial to say we have to stick to reason and logic in the pursuit of truth. But there is nothing about understanding the difference between human and machine intelligence that is forbidden by any reason or logic that I am aware of. Certainly there is a lot we don’t know about that topic. But unless that knowledge is specifically blocked by logic or the laws of physics, it is knowable. The only thing missing is the knowledge required to understand it. The only source of knowledge is conjecture and criticism, and discussion with others is a great tool for that process. The more the better. So unless you believe we know all there is to know about a topic right now, there is always the possibility that some new knowledge could help us get closer to the objective truth about it. And it’s guaranteed that new knowledge will come from creative new ideas and criticism of existing ideas. Saying something is unknowable is akin to believing in a mysterious god creator, or fairies. But more importantly, it is a way to stop or limit the progress of our understanding of the universe. It’s like when parents say “because I said so”. It shuts down discourse and understanding and leads to closed societies.
- deleted 4y ago
  [deleted]
- SergeAx 4y ago
  Artist is using ideas already existed and creates something new out of them. It has always been this way since cave wall art. I am 100% sure Copernicus was not the first to suggest a heliocentric system, but he was the one who put enough energy into proving it and defend that theory.
gwern 4y ago
> like I'm not sure why kermit has a plastic texture in many of the pictures That might be the known bug with low-resolution textures: the DALL-E 2 paper notes that the details in very complex scenes can be bad, and thinks it's because you start with a 64px image which is necessarily bad for details (64px is really small!) and upscale with dumber models from there https://cdn.openai.com/papers/dall-e-2.pdf#page=17 https://cdn.openai.com/papers/dall-e-2.pdf#page=17 I think this explains the issues with images where the 'skin' or 'fur' looks really creepy (eg. all the semi-nude bears).

tdehnel 4y ago

Sorry to be this guy but that is not creativity. It’s using what already exists, not conjecturing something new. Contrast with real creativity (what people can do but machines currently cannot) where you conjecture something completely new. For example, Copernicus conjecturing the idea that the Earth revolves around the Sun. No machine learning model would have gotten there because it would have been trained on a bunch of data that said the Earth was the center of the universe.

dougmwne 4y ago
These are fun discussions because words like "artistic creativity" have a colloquial meaning that could only apply to humans since the dawn of humanity. Now you have an image of Kermit in Wall-E. I have never seen or conceived of an image of Kermit in Wall-E. Let's assume that adorable robot Kermits do not exist in the training data to be spit out like a search algorithm. The image is new, it did not previously exist. It is a creation, a very vague idea of a few words that was created in full realization. So it sees like the only difference between the "Not creativity" that Dall-E is doing and "Real Creativity" that humans do is tht humans are the ones doing it? I agree there's this concept of expanding the frontiers of human aesthetic capability that has slow-marched from cave paintings till post-modernism. That there are a very few artists that invent completely new styles that the rest of us copy and remix. It's questionable that Dall-E can do that, but I'm also not sure that it can't do that.
- orbital-decay 4y ago
  >So it sees like the only difference between the "Not creativity" that Dall-E is doing and "Real Creativity" that humans do is tht humans are the ones doing it? The differentiator is whether the result is worthy to look at for humans, that's all. In case of the OP, you forget that the human had to predict that the combination of two would be interesting for other humans, and then construct the prompt, possibly selecting the best pictures. That's who did most of the work here, and it was effortless for the neural network of a human. Could DALL-E analyze the world autonomously without human intervention at all? No, it's an open loop system. Novelty hinges on the ability to conceptualize things, not the execution. Sure, DALL-E 2 shows a glimpse of conceptualization internally as it works with compressed descriptions (concepts, abstractions) of things it draws. But it's super limited and not flexible enough to create new ideas, it doesn't have either short-term or long-term memory, it doesn't change, it has all knowledge about Kermit and Blade Runner pre-baked, and so on. You have to re-train it from scratch every time you want it to remember something truly new, there's no feedback loop to do that. Human ability is still much more powerful. DALL-E 2 is almost at the point where it can supplement the human conceptualization with AI execution, though. Possibly in a couple iterations it will be there, with more believable results. In very limited cases, of course - as it's temporally unstable (it makes a totally new image each time), cannot correct the output from new details provided by a human (like a professional concept artist could), etc.
  - treesprite82 4y ago
    > you forget that the human had to predict that the combination of two would be interesting for other humans, and then construct the prompt, possibly selecting the best pictures. That's who did most of the work here If the Twitter user claimed that the text prompts themselves were generated by asking GPT-3 for "An interesting sequence of text prompts to feed an image generation AI" or something like that, I would have believed them. Presumptively, I imagine it's harder to create a model that generates images matching a certain human-language input prompt than to create an image generation model with no language component and have it pick its own scenarios internally. I don't think the former is done to palm off "most of the work" to humans, but rather because people want an easy way to see create own ideas so there's more demand for it. > You have to re-train it from scratch every time you want it to remember something truly new, there's no feedback loop to do that. As far as I'm aware, this isn't true. Deep learning is perfectly compatible with fine-tuning an existing model using new data. OpenAI/MS have been doing this with Codex to improve it based on Copilot telemetry and code from new languages/libraries.
andrewstuart2 4y ago
The scientific and creative methods are both entirely centered around evaluating data or existing creative elements and combining it in novel ways. Science adds the goal of making and testing hypotheses but nothing about Copernicus's conjecture would have been possible without observing existing information. I'm including e.g. "new" astronomical observations in the realm of existing information because even a human is simply observing and measuring it. I fail to see what the difference is between 99% of existing "creativity," which is essentially arranging existing ideas into novel combinations, and what DALLE2 does.
gwern 4y ago
> No machine learning model https://www.nature.com/articles/d41586-019-03332-7 https://www.nature.com/articles/d41586-019-03332-7
pphysch 4y ago
Humans don't generate new ideas from nothing. Your "real creativity" doesn't exist. Everything is dependent on what came before, and therefore derivative to some extent. Copernicus got his idea after gathering a lot of data, explicitly and implicitly, training his internal model of the world.
- shrimp_emoji 4y ago
  Even when it may appear that a human brain "generated something from nothing" (that is: you completely fail to account from where it could have possibly derived its output), you can always fall back to "genetic memory". Even a newborn has a lot going on that was "pre-programmed" by genetics, like a form of hardcoded training. :p
adamsmith143 4y ago
>Sorry to be this guy but that is not creativity. It’s using what already exists, not conjecturing something new. Such a cute point of view, completely wrong but cute. Please go find the original images of Kermit in Blade Runner and WallE that were just copied here. >For example, Copernicus conjecturing the idea that the Earth revolves around the Sun. No machine learning model would have gotten there because it would have been trained on a bunch of data that said the Earth was the center of the universe. If the model were trained on actual observations of planetary trajectories it would trivially recreate keplers laws, newtons laws etc.
- soperj 4y ago
  >If the model were trained on actual observations of planetary trajectories it would trivially recreate keplers laws, newtons laws etc. Should be easy to prove.
  - ben_w 4y ago
    Indeed: • https://news.ycombinator.com/item?id=31609839 https://news.ycombinator.com/item?id=31609839 • https://arxiv.org/pdf/1810.10525.pdf https://arxiv.org/pdf/1810.10525.pdf • https://arxiv.org/pdf/1905.11481.pdf https://arxiv.org/pdf/1905.11481.pdf
    - soperj 4y ago
      from your link: "Renner stresses that although the algorithm derived the formulae, a human eye is needed to interpret the equations and understand how they relate to the movement of planets around the Sun."
      - ben_w 4y ago
        Even if that had been the only example rather than just one among them, it would still demonstrate the claim that an AI can discover various laws of physics just from observing data.
- tdehnel 4y ago
klik99 4y ago
"Good artists borrow, great artists steal" Creativity is a very vague word, I'm sure we can come up with definitions of it that let humans keep sole domain over it. But breakthroughs often come from combining domains and concepts, very very rarely do we ever jump out of one local maxima into another, and I'm not even convinced that Copernicus counts as that. There's a reason why there are so many examples of the same breakthrough happening in multiple places in the world independently - innovation is a slow gradual collaborative process and not plateaus waiting for men of genius to have a spark of inspiration. Also I'm not convinced that a computer couldn't have discovered the earth revolves around the sun - it's hard to make machine learning jump out of local maxima, but it does happen, and I can see some hidden layers becoming far more efficient at predicting outcomes by stumbling across a model that centered the sun. That being said - there likely are examples of things that computers couldn't have theoretically figured out the model for, but I'm hard pressed to think of one.
- soperj 4y ago
  That is actually a misquote from people who stole.
  - kelseyfrog 4y ago
    That's great.
vouwfietsman 4y ago
This is a common fallacy. Call it moving goalposts, no true Scottsman, the AI effect, whatever. The behavior is as follows: an argument over whether an ill-defined attribute is possessed by a computer is defended or attacked with useless semantics since nobody can agree on what any of the words mean anyway. Creativity, intelligence, consciousness. It doesn't matter what you say, you cannot define these concepts with the same clarity you use to defend that the concept is missing. Saying there is no creativity because its just a neural net extrapolating from data is like saying there's no god because its all just atoms: what is god and why would the existence of atoms have anything to do with it. Learn from Wittgenstein: Worüber man nicht sprechen kann, darüber muss man schweigen.
- tdehnel 4y ago
  This single quote from Wittgenstein might just be the dumbest statement ever made. How would we ever make progress if we did not discuss what we don't understand, and critique and debate new creative ideas?
  - vouwfietsman 4y ago
    By not taking the quote literally, but trying to understand it in its context. The quote is a judgement about what can be logically achieved through discourse, and what cannot (and should not). He draws this boundary to separate the things we can reason about and the things that fall beyond reason and so cannot be reasoned about at all. If you cannot reason about something, you should not bring it up in (philosophical) discourse. He does this to settle philosophical debates around, for instance, mystical subjects that were fought over to exist or not exist. The conclusion for him is: we do not have the logical tools and concepts to reason about the problem, so we should stop wasting time talking about it. It's not about lack of knowledge or intelligence, its about acknowledging the limits of reason and discourse, and forgoing any conclusions made beyond those limits. It's a pursuit of truth. At least, this is how I understand it, I'm not a philosopher.
    - tdehnel 4y ago
      It’s not controversial to say we have to stick to reason and logic in the pursuit of truth. But there is nothing about understanding the difference between human and machine intelligence that is forbidden by any reason or logic that I am aware of. Certainly there is a lot we don’t know about that topic. But unless that knowledge is specifically blocked by logic or the laws of physics, it is knowable. The only thing missing is the knowledge required to understand it. The only source of knowledge is conjecture and criticism, and discussion with others is a great tool for that process. The more the better. So unless you believe we know all there is to know about a topic right now, there is always the possibility that some new knowledge could help us get closer to the objective truth about it. And it’s guaranteed that new knowledge will come from creative new ideas and criticism of existing ideas. Saying something is unknowable is akin to believing in a mysterious god creator, or fairies. But more importantly, it is a way to stop or limit the progress of our understanding of the universe. It’s like when parents say “because I said so”. It shuts down discourse and understanding and leads to closed societies.
deleted 4y ago
[deleted]
SergeAx 4y ago
Artist is using ideas already existed and creates something new out of them. It has always been this way since cave wall art. I am 100% sure Copernicus was not the first to suggest a heliocentric system, but he was the one who put enough energy into proving it and defend that theory.

gwern 4y ago

> like I'm not sure why kermit has a plastic texture in many of the pictures That might be the known bug with low-resolution textures: the DALL-E 2 paper notes that the details in very complex scenes can be bad, and thinks it's because you start with a 64px image which is necessarily bad for details (64px is really small!) and upscale with dumber models from there https://cdn.openai.com/papers/dall-e-2.pdf#page=17 https://cdn.openai.com/papers/dall-e-2.pdf#page=17 I think this explains the issues with images where the 'skin' or 'fur' looks really creepy (eg. all the semi-nude bears).