5 ms·

What makes you think human creativity is anything more than statistical modelling? What else can it be, based as it is in a physical substrate? Objections like

What makes you think human creativity is anything more than statistical modelling? What else can it be, based as it is in a physical substrate? Objections like yours are usually a sneaky way of implying that human consciousness breaks free of the physical world, cartesian dualism hidden within people's arguments. "It cannot be creative because it's only bits/cogs/linear algebra/etc/etc." Well describe to me the way it is different to the processes of the human brain? "There is some magic sauce in the human brain we do not yet understand!", well then how do you know that this magic sauce does not exist within the statistical models inside the computer? I find it very irritating that such shallow reasoning prevails amongst intelligent people.

omnicognate 5y ago
> What else can it be, based as it is in a physical substrate? This is no less shallow reasoning. The question of whether the academic field of statistical modelling already contains the necessary ideas to produce strong AI is not decided, and won't be unless/until somebody makes a strong AI. People have different intuitions about what the answer will be and until it can be determined empirically I suggest treating them as what they are: intuitions.
- FeepingCreature 5y ago
  Sure, but I see no reason to not treat things like GPT-3 and TWDNE as evidence. If it looks like a creativity, and quacks like a creativity, then we should at least entertain the notion that we have a small artistic process on our hands.
  - omnicognate 5y ago
    Of course we can entertain the notion. We just shouldn't pretend the question is settled one way or the other like the two comments above and pointlessly argue about it on hn. Personally I'm inclined to see things like GPT-3 as qualitatively different to human creativity, but I don't claim to know this for a fact.
    - FeepingCreature 5y ago
      Well, but also, if you think that GPT or TWDNE are different from creativity, when they look similar to creativity on the face of it, I do think it's not enough to say "well I think it's a different thing", you should actually put out there where you think that human creativity materially diverges from the output of an ANN. I just think there's a lot of people arguing "well this can't be creativity" out of some sort of anti-AI signalling sentiment, where they've become so used to disagreeing with unwarranted AI optimism that they undervalue AI progress routinely. In other words, "this can't be creativity, because we don't understand creativity" or "this can't be creativity because AI is very far from human-level" or such. (Personally, I believe GPT-3 and CLIP are already superhuman; that is, superior to the circuitry in the brain doing the equivalent job. They only look weak because we're trying to make them do all of the job in one step, when we're comparing them to the brain's highly heterogenous self-correcting approach.)
      - omnicognate 5y ago
        You say I'm required to "put out there where [I] think that human creativity materially diverges from the output of an ANN" but you state your opinion with as little support as I did. Both are just opinions. My views on the matter have nothing to do with being "anti-AI" or "signalling" to anyone, and the latter suggestion is mildly offensive. Like many here, while being no sort of expert I've thought quite a lot about these questions and have fairly detailed views about them. I suspect strong AI is possible and will not require new fundamental physics or anything like that. However, I suspect that if it is achieved it will be of a very different nature to GPT-3 etc and to ML in general. I don't think the behaviour demonstrated by these systems so far resembles human creativity, understanding or reasoning at all, although they are capable of impressive and fascinating things. I'd go so far as to say the differences seem, to me at least, rather obvious, if difficult to define. However, I'm open to alternative views and to the possibility I may be wrong. I greatly enjoy discussing these sorts of things, but adversarial exchanges on hn filled with ad-hominems aren't the way I like to do it.
  - plutonorm 5y ago
    Exactly. There's some definite substrate-ism (like sex-ism) going on here!
- plutonorm 5y ago
  I didn't mean to imply it was a settled question. I had in my mind the notion that all this is rather unproven at the moment. I quite strongly suspect that there is a small but significant amount of whatever "intelligence" might be within GPT3 et al. But I completely agree, it is just a suspicion.
capableweb 5y ago
> What makes you think human creativity is anything more than statistical modelling? Our ability to make the decision between following the rules and breaking the rules, when suitable. A computer could also break the rules, but in most cases it wouldn't make sense or look like good, while a human could make judgement about when to break the rule. Sure, we all learn by copying, but after a while, we start getting a feel for when to break the rule, and that's when unique art appears. Computers seems to not have learned this fact yet (or rather, haven't been taught that yet). Using the tool that this submission offers, all the results will look similar and can be traced back to the training set you give it. Do something similar with a human (over similar amount of time that the machine got, in terms of human time) and eventually the results will look way different than the training set, as what we see with artists in real life.
YeGoblynQueenne 5y ago
>> What makes you think human creativity is anything more than statistical modelling? For example, humans don't need to see millions of examples of waifu before they can draw their own. Also, humans can draw in different styles, including novel styles that look nothing like styles they have seen before. Statistical models like GANs can only draw in styles similar to the ones in their training sets. Statistical modelling can only represent the data in a training dataset and is incapable of novelty. Humans are capable of novelty. >> Well describe to me the way it is different to the processes of the human brain? We haven't created the human brain and it's very unlikely it uses a technology we understand, like linear algebra.
- nybble41 5y ago
  > For example, humans don't need to see millions of examples of waifu before they can draw their own. Humans have the advantage of being trained for years on a far larger and more generalized dataset before they're asked to draw anything. > Statistical models like GANs can only draw in styles similar to the ones in their training sets. The intermediate states in the article's example seemed to contain a number of novel styles to me. The final results are filtered for conformance to a specific style due to the nature of the project, but that doesn't mean the GAN is incapable of drawing in other styles. (Of course, most of these alternate styles are not particularly appealing to humans, but the same is true for many of the experimental styles invented by human artists.) > Statistical modelling can only represent the data in a training dataset and is incapable of novelty. These AI models are generating images which never existed before, and which were not in their datasets. How is that not novelty?
  - YeGoblynQueenne 5y ago
    >> Humans have the advantage of being trained for years on a far larger and more generalized dataset before they're asked to draw anything. That's a big assumption wrapped up in an over-wrought analogy. Humans don't "train" in the sene that statistical models, or neural nets, are trained. We don't have any clear supervision for example, no ground truth. And we don't need examples of exactly the things we learn, to learn them. For instance, nobody ever saw an example of a manga character before the first manga character was drawn. And yet, someone drew it. >> These AI models are generating images which never existed before, and which were not in their datasets. How is that not novelty? How I like to think of it, which is a bit of a fudge, is that neural nets learn to convert each of their input images into a connect-the-dot puzzle (the "dots" are the data points in a very high-dimensional space that encompasses the pixels of all their training images; like I say, it's a bit of a fudge). Every new training image gets its own connect-the-dot puzzle superimposed on those of all previous images. Once training is done, you can ask the trained model to generate new images and it basically puts its pen down on a dot, and starts drawing a line. What dot comes next depends on timey-wimey model-probabilities. Obviously, in that way, it can't draw a line to a dot outside the big network of superimposed connect-and-dot puzzles it has put together. Such outside-context dots don't exist for the model, in any real sense. So it can only create images that exist within that puzzle. In truth, the puzzle, i.e. the trained model, is a dense region of cartesian space (a manifold). What comes out of the model must already exist in that manifold, so it must be a variation, or combination, of the training images used to construct the manifold. Which means, it can't innovate. So for instance, you can't expect to train it on images of manga characters and find that it now draws you in the style of Michelangelo. That's what I mean. Of course you'll see images that are not exactly the images you put in, but you won't see images that are very different from the ones you put in. It is, in a very concrete sense, a very limited ability to generate new images.
bmitc 5y ago
You imply a bunch of things I didn't say. We don't have a definition or model of creativity or intelligence, even within humans and other animals. Most humans struggle to view other things as intelligent when they are at a close inspection. We have some mild understanding at a sort of system level, but it's usually enough to generally bin things up into not intelligent or not. For something like the original post, we do know what these things are. They're statistical models. Full stop. They show no indication of what we see in creative and intelligent behavior, that is the ability to self-adapt to both internal and external initiatives. This GAN in the post has no ability to step outside of the statistics in the training set unless the model is updated to prod it to do so. The model can be changed, but it is a forceful change. If you show me tens of thousands of images, I am not, at an emergent, top-level, system level, etc., bounded to the statistics of that image set. Is this GAN asked something or given a goal aside from an implicit "draw something like what we've given you"? Even if I do draw something like or akin to the given image set, I have full creative control over the image (assuming some drawing skill). If the human brain (and really body) can be modeled via a statistical model (which is not yet known but is surmised as you imply), that doesn't necessarily explain high-level behaviors. More is different. You call it magic sauce, but others call it emergence. Our understanding of emergent behavior and complex systems at large is still in work. In my view, metaphorical thinking, of which analogical thinking is a subset, is a likely kernel of human intelligence. While these statistical models are copying, which is similar in a way to analogy building, it's not quite there. The reason things it generates looks like other things is because it searched a parameter space for matching statistics. However, it cannot even explain that's why it generated what it did. We explain for it. These things are no more artificially intelligent than things like thermodynamics are naturally intelligent. Lastly, as I pointed out in my original comment, if this is indeed creative as someone like you implies, the article fails to make a convincing argument and bounces around a lot of buzzwords. > I find it very irritating that such shallow reasoning prevails amongst intelligent people. I was offended, but I suppose I agree. ;)

omnicognate 5y ago

> What else can it be, based as it is in a physical substrate? This is no less shallow reasoning. The question of whether the academic field of statistical modelling already contains the necessary ideas to produce strong AI is not decided, and won't be unless/until somebody makes a strong AI. People have different intuitions about what the answer will be and until it can be determined empirically I suggest treating them as what they are: intuitions.

FeepingCreature 5y ago
Sure, but I see no reason to not treat things like GPT-3 and TWDNE as evidence. If it looks like a creativity, and quacks like a creativity, then we should at least entertain the notion that we have a small artistic process on our hands.
- omnicognate 5y ago
  Of course we can entertain the notion. We just shouldn't pretend the question is settled one way or the other like the two comments above and pointlessly argue about it on hn. Personally I'm inclined to see things like GPT-3 as qualitatively different to human creativity, but I don't claim to know this for a fact.
  - FeepingCreature 5y ago
    Well, but also, if you think that GPT or TWDNE are different from creativity, when they look similar to creativity on the face of it, I do think it's not enough to say "well I think it's a different thing", you should actually put out there where you think that human creativity materially diverges from the output of an ANN. I just think there's a lot of people arguing "well this can't be creativity" out of some sort of anti-AI signalling sentiment, where they've become so used to disagreeing with unwarranted AI optimism that they undervalue AI progress routinely. In other words, "this can't be creativity, because we don't understand creativity" or "this can't be creativity because AI is very far from human-level" or such. (Personally, I believe GPT-3 and CLIP are already superhuman; that is, superior to the circuitry in the brain doing the equivalent job. They only look weak because we're trying to make them do all of the job in one step, when we're comparing them to the brain's highly heterogenous self-correcting approach.)
    - omnicognate 5y ago
      You say I'm required to "put out there where [I] think that human creativity materially diverges from the output of an ANN" but you state your opinion with as little support as I did. Both are just opinions. My views on the matter have nothing to do with being "anti-AI" or "signalling" to anyone, and the latter suggestion is mildly offensive. Like many here, while being no sort of expert I've thought quite a lot about these questions and have fairly detailed views about them. I suspect strong AI is possible and will not require new fundamental physics or anything like that. However, I suspect that if it is achieved it will be of a very different nature to GPT-3 etc and to ML in general. I don't think the behaviour demonstrated by these systems so far resembles human creativity, understanding or reasoning at all, although they are capable of impressive and fascinating things. I'd go so far as to say the differences seem, to me at least, rather obvious, if difficult to define. However, I'm open to alternative views and to the possibility I may be wrong. I greatly enjoy discussing these sorts of things, but adversarial exchanges on hn filled with ad-hominems aren't the way I like to do it.
- plutonorm 5y ago
  Exactly. There's some definite substrate-ism (like sex-ism) going on here!
plutonorm 5y ago
I didn't mean to imply it was a settled question. I had in my mind the notion that all this is rather unproven at the moment. I quite strongly suspect that there is a small but significant amount of whatever "intelligence" might be within GPT3 et al. But I completely agree, it is just a suspicion.

capableweb 5y ago

> What makes you think human creativity is anything more than statistical modelling? Our ability to make the decision between following the rules and breaking the rules, when suitable. A computer could also break the rules, but in most cases it wouldn't make sense or look like good, while a human could make judgement about when to break the rule. Sure, we all learn by copying, but after a while, we start getting a feel for when to break the rule, and that's when unique art appears. Computers seems to not have learned this fact yet (or rather, haven't been taught that yet). Using the tool that this submission offers, all the results will look similar and can be traced back to the training set you give it. Do something similar with a human (over similar amount of time that the machine got, in terms of human time) and eventually the results will look way different than the training set, as what we see with artists in real life.

YeGoblynQueenne 5y ago

>> What makes you think human creativity is anything more than statistical modelling? For example, humans don't need to see millions of examples of waifu before they can draw their own. Also, humans can draw in different styles, including novel styles that look nothing like styles they have seen before. Statistical models like GANs can only draw in styles similar to the ones in their training sets. Statistical modelling can only represent the data in a training dataset and is incapable of novelty. Humans are capable of novelty. >> Well describe to me the way it is different to the processes of the human brain? We haven't created the human brain and it's very unlikely it uses a technology we understand, like linear algebra.

nybble41 5y ago
> For example, humans don't need to see millions of examples of waifu before they can draw their own. Humans have the advantage of being trained for years on a far larger and more generalized dataset before they're asked to draw anything. > Statistical models like GANs can only draw in styles similar to the ones in their training sets. The intermediate states in the article's example seemed to contain a number of novel styles to me. The final results are filtered for conformance to a specific style due to the nature of the project, but that doesn't mean the GAN is incapable of drawing in other styles. (Of course, most of these alternate styles are not particularly appealing to humans, but the same is true for many of the experimental styles invented by human artists.) > Statistical modelling can only represent the data in a training dataset and is incapable of novelty. These AI models are generating images which never existed before, and which were not in their datasets. How is that not novelty?
- YeGoblynQueenne 5y ago
  >> Humans have the advantage of being trained for years on a far larger and more generalized dataset before they're asked to draw anything. That's a big assumption wrapped up in an over-wrought analogy. Humans don't "train" in the sene that statistical models, or neural nets, are trained. We don't have any clear supervision for example, no ground truth. And we don't need examples of exactly the things we learn, to learn them. For instance, nobody ever saw an example of a manga character before the first manga character was drawn. And yet, someone drew it. >> These AI models are generating images which never existed before, and which were not in their datasets. How is that not novelty? How I like to think of it, which is a bit of a fudge, is that neural nets learn to convert each of their input images into a connect-the-dot puzzle (the "dots" are the data points in a very high-dimensional space that encompasses the pixels of all their training images; like I say, it's a bit of a fudge). Every new training image gets its own connect-the-dot puzzle superimposed on those of all previous images. Once training is done, you can ask the trained model to generate new images and it basically puts its pen down on a dot, and starts drawing a line. What dot comes next depends on timey-wimey model-probabilities. Obviously, in that way, it can't draw a line to a dot outside the big network of superimposed connect-and-dot puzzles it has put together. Such outside-context dots don't exist for the model, in any real sense. So it can only create images that exist within that puzzle. In truth, the puzzle, i.e. the trained model, is a dense region of cartesian space (a manifold). What comes out of the model must already exist in that manifold, so it must be a variation, or combination, of the training images used to construct the manifold. Which means, it can't innovate. So for instance, you can't expect to train it on images of manga characters and find that it now draws you in the style of Michelangelo. That's what I mean. Of course you'll see images that are not exactly the images you put in, but you won't see images that are very different from the ones you put in. It is, in a very concrete sense, a very limited ability to generate new images.

bmitc 5y ago

You imply a bunch of things I didn't say. We don't have a definition or model of creativity or intelligence, even within humans and other animals. Most humans struggle to view other things as intelligent when they are at a close inspection. We have some mild understanding at a sort of system level, but it's usually enough to generally bin things up into not intelligent or not. For something like the original post, we do know what these things are. They're statistical models. Full stop. They show no indication of what we see in creative and intelligent behavior, that is the ability to self-adapt to both internal and external initiatives. This GAN in the post has no ability to step outside of the statistics in the training set unless the model is updated to prod it to do so. The model can be changed, but it is a forceful change. If you show me tens of thousands of images, I am not, at an emergent, top-level, system level, etc., bounded to the statistics of that image set. Is this GAN asked something or given a goal aside from an implicit "draw something like what we've given you"? Even if I do draw something like or akin to the given image set, I have full creative control over the image (assuming some drawing skill). If the human brain (and really body) can be modeled via a statistical model (which is not yet known but is surmised as you imply), that doesn't necessarily explain high-level behaviors. More is different. You call it magic sauce, but others call it emergence. Our understanding of emergent behavior and complex systems at large is still in work. In my view, metaphorical thinking, of which analogical thinking is a subset, is a likely kernel of human intelligence. While these statistical models are copying, which is similar in a way to analogy building, it's not quite there. The reason things it generates looks like other things is because it searched a parameter space for matching statistics. However, it cannot even explain that's why it generated what it did. We explain for it. These things are no more artificially intelligent than things like thermodynamics are naturally intelligent. Lastly, as I pointed out in my original comment, if this is indeed creative as someone like you implies, the article fails to make a convincing argument and bounces around a lot of buzzwords. > I find it very irritating that such shallow reasoning prevails amongst intelligent people. I was offended, but I suppose I agree. ;)