7 ms·

I think you hit the nail on the head, with the salient point here being that in the near future "creative" things will be automated first (see Image GPT, Jukebo

I think you hit the nail on the head, with the salient point here being that in the near future "creative" things will be automated first (see Image GPT, Jukebox, etc. Google has 100 billion dollars cash and countless TPUs, best engineers, infra, etc - they could probably replicate results far better than each of these OpenAI projects within a few years). One of the things that got me into ML research was the notion that we could automate a lot of the hard work humans do every day (agriculture, cooking, desk jobs, etc) so that humans could do things that were uniquely theirs & interesting, that were human, that were beautiful... Unfortunately it turns out that classical music and waxing poetic are easily generative in an enjoyable way. In the most ironic fashion possible, it turns out that the very thing we do when we conduct ML research, what you call the "logical domain", is one of the only things that stays human-only in the foreseeable future. GPT-3 and other projects seem to drive hype cycles in the tech community and convince people like Elon Musk that the AGI revolution is near. But I think recent progress is just another example of machine learning models being able to generalize on super large datasets, even if it's the biggest model so far. It's not clear to me that larger models will solve this in the limit; take the way GPT3 fails on addition past a certain number, and the fundamental inability for transformers to learn certain algorithms. It is certainly still possible for this type of large dataset, large model style of ML to make human life better in many ways - like Tesla is trying to do with self driving cars, or Covariant with automating Amazon-like jobs. But I think when it comes to tackling the hard problems of true intelligence, we're missing a dimension somewhere.

qppo 6y ago
This is going to sound very dismissive and condescending: "meh." Generative music has been around for half a century, or longer depending on how you want to interpret things. Mimicry as a mechanism for composition has been around for as long as humans have made music. It is wholly uninteresting to discover that we can design generative systems for music that excel at mimicry, because we've already perfected that mechanism in analog. The interesting bit is that the genesis of new musical ideas is driven by manual interaction and direction of the generative system, and at that point it's the guiding hand of the engineer turned artist that we can respect and appreciate, not the mimicry of a machine.
- visarga 6y ago
  > Generative music has been around for half a century, If you start by referring to results from 50 years ago, have you tried listening to state of the art generative music systems lately? They can probably compose music better than 99% of humans.
  - chrisco255 6y ago
    Have you been moved by any of that music though? Am I missing something?
    - TomMarius 6y ago
      Yeah, it's very moving to see a human-made machine do such wonders. Fills me with awe, appreciation, and hope.
      - dahfizz 6y ago
        That's exactly it, though. This stuff is interesting because of the novelty of AI. The works themselves are not independently relevant (not yet, at least).
        TomMarius 6y ago
        Elsewhere someone replied that art is interesting in a large part because of the personal story. How is this differemt?
        FartyMcFarter 6y ago
        99% of the time, I don't listen to music for the personal story of the artists involved. In fact, a lot of the music I listen to is made by artists that I know very little about.
    - woko 6y ago
      Listen to some samples: - https://openai.com/blog/jukebox/ https://openai.com/blog/jukebox/ (2020, quite good, but no classical music) - https://openai.com/blog/musenet/ https://openai.com/blog/musenet/ (2019 so not as good as the 2020 one, but showcases classical music) There is no reason to assume that one cannot be moved by AI-generated music, as the AI has learnt from human-generated music and tries to mimick the styles.
andykx 6y ago
But does the fact that machines can also create works of music and art make it any less enjoyable for humans to create them? Will we suddenly stop writing or drawing for pleasure?
- chrisco255 6y ago
  Probably not. Humans are still playing Jeopardy and chess despite losing dominance in those games a long time ago.
- skor 6y ago
  There is nothing like the feeling of performing music for a crowd. There is also nothing like hitting a chord in a big empty space and listening while the sound slowly fades away. Related to instruments themselves, the trial and error is one very important aspects I can think of right now that's enjoyable: playing something off beat or out of tune and correcting yourself. The feeling of correction and improvement. It is a real pity the actual algorithm itself has no way to enjoy what it is creating.
duckerude 6y ago
GPT-3 can write working React components. But we can't expect it to scale up to complete useful programs soon. GPT-3 can write hauntingly beautiful snippets of prose. Can we expect it to scale up to coherent novels? It's easier to see the limitations in the areas you know best. It's significant that it's this good at creative tasks, but I'm not convinced that creative tasks are the most at risk.
spacechild1 6y ago
Disclaimer: I'm a composer > Unfortunately it turns out that classical music and waxing poetic are easily generative in an enjoyable way On the contrary, I would say that generating convincing and original classical is an incredibly hard (if not impossible) task. All the current music AI projects give results which may sound “good“ to a casual listener, but they sound horribly wrong to any educated listener. The reason is that AI can only imitate the surface, but completely misses to recognize/synthesize larger structures. This might be ok for some background noodling in a TV drama, but not for the concert stage. Finally, we rarely perceive art works in isolation. We know and appreciate the fact that a certain work has been created by a certain person in a certain time.
- ukj 6y ago
  This is the same argument people made against MP3 compression. Lossy is bad. Humans will never stand for it. Perfections will not stand for it. Pragmatists won’t notice. This isn’t a bad thing. We need perfectionists to drag us across the “good enough” line. Despite our childish kicking&screaming.
  - Nimitz14 6y ago
    Absolutely terrible comparison, completely not relevant.
- sriku 6y ago
  The reality is likely neither here nor there - i.e. computing may have more to offer to the creative endeavor than creators would like to admit, but still leave an obvious gap which technologists might be loathe to admit. It may be instructive to look at David Cope's [1] work (what he calls "recombinant music" [2]). Cope's been writing algorithms to compose in the styles of the masters (Mozart/Chopin/et al) for about 3 decades now, well before the recent surge in "AI". His techniques are much less sexy for the "deep learning" enthusiasts, and yet he managed to outrage an audience of connoisseurs who assembled to listen to a "lost Chopin piece" only to be told, after they shared their applause, that it was composed by a computer taught to mimic Chopin's style (the composition was performed by a musician). The response, in my opinion, also points to music as a social constructed experience and not purely attributable to the sound signal itself. i.e. if I give you a romantic background story for a lost composition of a master, you may be inclined to experience the piece in a more favorable light than if I told you it was generated by an algorithm (or the converse). You're absolutely right that the musical output of the current crop of "AI" projects (especially the ones using deep learning / neural networks) are crappy to even a modestly trained listener .. or even a lay untrained listener for that matter. However, more involved modeling (such as Cope's) has produced some very compelling results decades ago, so it would be a mistake to assume that the current crop won't get close enough [3]. The fact that DL systems don't need to be instructed in the way Cope has had to encode his musical understanding is also something to be considered in the evaluation as well as in scoping their capabilities going forward. [1]: https://en.wikipedia.org/wiki/David_Cope https://en.wikipedia.org/wiki/David_Cope [2]: https://www.recombinantinc.com https://www.recombinantinc.com [3]: https://deepmind.com/blog/article/wavenet-generative-model-raw-audio https://deepmind.com/blog/article/wavenet-generative-model-r... (see "Making Music" section and examples there)
bambax 6y ago
But in the arts, can AI come up with something truly new? This should be testable: train AI on all the music ever written before Bach, and see if it ever produces something ressembling Bach. Maybe that kind of test has alretbeen done; it would be interesting to know what comes out of it.
- spyder 6y ago
  The GPT-2 based Musenet music generator is already interesting but far from perfect. You can try it in the middle of this article: https://openai.com/blog/musenet/ https://openai.com/blog/musenet/ (you can even upload custom prompts in the advanced mode) Would be interesting to see it with the updated GPT-3. There is also AIVA with more production ready results: https://www.youtube.com/watch?v=gzGkC_o9hXI&list=PLv7BOfa4CxsHAMHQj0ScPXSbgBlLglRPo&index=1 https://www.youtube.com/watch?v=gzGkC_o9hXI&list=PLv7BOfa4Cx... Not sure how it works, but it has better results maybe because it's using more predefined components and less AI so it's also less "creative". More AI music projects here: https://magenta.tensorflow.org/ https://magenta.tensorflow.org/
- an_opabinia 6y ago
  Obviously no. But there’s so much classical music out there, that an average person would never be able to tell the difference between something that is generated anew and something just really obscure. Have you ever tried copying and pasting sections of GPT output into Google?
- p1esk 6y ago
  This should be testable There have been music resembling Bach written before Bach (e.g. https://www.youtube.com/watch?v=VUcdBz3LIuU https://www.youtube.com/watch?v=VUcdBz3LIuU). How much more of resemblance you hope for?
Isinlor 6y ago
> It's not clear to me that larger models will solve this in the limit; take the way GPT3 fails on addition past a certain number, and the fundamental inability for transformers to learn certain algorithms. GPT-3 was OpenAI exercise in how far pure scaling can get you. They have used some 2 years old method. Already at the point when they started training GPT-3 there were readily available remedies to many of GPT-3 issues. Given how they energized the wider community I'm sure even more focus will be given to improving language models in the following years. Some rough ideas right now: - People think that cherry-picking the best GPT-3 examples is cheating - why? Train a model that will be selecting the best examples for you. My proposition is to train a model that guesses whether some text was GPT-3 generated or human made - select samples that look the most human like. - Use a good search method to look for the best samples. Monte Carlo Tree Search? AlphaZero? MuZero? If MuZero can play a games of Chess, Shogi, Go and all of Atari then way should it not be able to play a game of what word will come next? - Hook up the language model to a search engine. Instead of writing a whole program yourself, why not to copy-paste some stuff from StackOverflow with some slight modifications? Etc. It doesn't address the issues with agency, grounding and multi-modality, but it's a good road map for the next 2-3 years.
- p1esk 6y ago
  train a model that guesses whether some text was GPT-3 generated or human made - select samples that look the most human like. What you said is essentially: "Train a better GPT model". Humans have trouble distinguishing between (some of) GPT-3 and human writing. The only way to build a classifier that can do this is to build a model that is better than GPT-3 at understanding text. It would need to have features currently absent in GPT-3, such as common sense and understanding the world (e.g. causality, physics, psychology, history, etc). If what you say could be done, GPT-3 would have been designed as a GAN.
  - Isinlor 6y ago
    It's a lot easier to notice logical mistakes in already written text, than it is to avoid making them in the first place. When you write text do you write it in one pass or do you read yourself and fix mistakes, reformulate sentences etc.? I have reformulated this piece of text at least once in order to make my argument clear. That's the difference between GPT and BERT. GPT can only attend to the past outputs, while BERT one can attend also to the future outputs. Now imagine that what you are going to say is not actually determined by you, but it is sampled randomly from what seems like a reasonable thing to say. This is how GPT-3 works. If somebody ask you some kind of question you can guess 70% yes or 30% no, then roll a 10 side dice to pick one, but once you pick there is no way back. And I already mentioned that it does not address agency, grounding and multi-modality, but it could improve GPT ability to formulate coherent arguments, follow instructions, write mathematical proofs and computer programs or play games. BTW - I actually have implemented it and it works quite reasonably. Here are samples from GPT-2 small and GPT-2 small + RoBERTa adversarial decoder. https://github.com/Isinlor/AdvDecoder/tree/master/outputs https://github.com/Isinlor/AdvDecoder/tree/master/outputs
karmakaze 6y ago
A better or hopeful projection is that "creative" things will split into casually consumed which is largely automated and more active/deeply experienced content which will be human made or directed. The first already exists in formulaic content generated by humans with little consideration for a cohesive story without self contradiction. I don't know which way things will go. Will newer and later generations be accustomed to and accept lower fidelity art? the uncanny valley be bridged from both sides? Or will there be attention being drawn to what is 'real' vs 'synthetic'. Good art is pain. Labelling these things distinctly will probably reveal that I consume some 'real', annoyed by some 'synthetic' while enjoying as much. This will get challenging as machine generated can seem more 'real' than much human made content: 'real' is/was a subset of human made, machine made is/was a subset of 'synthetic'. This line of reasoning leads me to believe that premium content will be interactive. This means that the content has to either have a human connection or be closer and closer to passing a Turing test. The current examples of machine made static content wont cut it.

qppo 6y ago

This is going to sound very dismissive and condescending: "meh." Generative music has been around for half a century, or longer depending on how you want to interpret things. Mimicry as a mechanism for composition has been around for as long as humans have made music. It is wholly uninteresting to discover that we can design generative systems for music that excel at mimicry, because we've already perfected that mechanism in analog. The interesting bit is that the genesis of new musical ideas is driven by manual interaction and direction of the generative system, and at that point it's the guiding hand of the engineer turned artist that we can respect and appreciate, not the mimicry of a machine.

visarga 6y ago
> Generative music has been around for half a century, If you start by referring to results from 50 years ago, have you tried listening to state of the art generative music systems lately? They can probably compose music better than 99% of humans.
- chrisco255 6y ago
  Have you been moved by any of that music though? Am I missing something?
  - TomMarius 6y ago
    Yeah, it's very moving to see a human-made machine do such wonders. Fills me with awe, appreciation, and hope.
    - dahfizz 6y ago
      That's exactly it, though. This stuff is interesting because of the novelty of AI. The works themselves are not independently relevant (not yet, at least).
      - TomMarius 6y ago
        Elsewhere someone replied that art is interesting in a large part because of the personal story. How is this differemt?
        FartyMcFarter 6y ago
        99% of the time, I don't listen to music for the personal story of the artists involved. In fact, a lot of the music I listen to is made by artists that I know very little about.
  - woko 6y ago
    Listen to some samples: - https://openai.com/blog/jukebox/ https://openai.com/blog/jukebox/ (2020, quite good, but no classical music) - https://openai.com/blog/musenet/ https://openai.com/blog/musenet/ (2019 so not as good as the 2020 one, but showcases classical music) There is no reason to assume that one cannot be moved by AI-generated music, as the AI has learnt from human-generated music and tries to mimick the styles.

andykx 6y ago

But does the fact that machines can also create works of music and art make it any less enjoyable for humans to create them? Will we suddenly stop writing or drawing for pleasure?

chrisco255 6y ago
Probably not. Humans are still playing Jeopardy and chess despite losing dominance in those games a long time ago.
skor 6y ago
There is nothing like the feeling of performing music for a crowd. There is also nothing like hitting a chord in a big empty space and listening while the sound slowly fades away. Related to instruments themselves, the trial and error is one very important aspects I can think of right now that's enjoyable: playing something off beat or out of tune and correcting yourself. The feeling of correction and improvement. It is a real pity the actual algorithm itself has no way to enjoy what it is creating.

duckerude 6y ago

GPT-3 can write working React components. But we can't expect it to scale up to complete useful programs soon. GPT-3 can write hauntingly beautiful snippets of prose. Can we expect it to scale up to coherent novels? It's easier to see the limitations in the areas you know best. It's significant that it's this good at creative tasks, but I'm not convinced that creative tasks are the most at risk.

spacechild1 6y ago

Disclaimer: I'm a composer > Unfortunately it turns out that classical music and waxing poetic are easily generative in an enjoyable way On the contrary, I would say that generating convincing and original classical is an incredibly hard (if not impossible) task. All the current music AI projects give results which may sound “good“ to a casual listener, but they sound horribly wrong to any educated listener. The reason is that AI can only imitate the surface, but completely misses to recognize/synthesize larger structures. This might be ok for some background noodling in a TV drama, but not for the concert stage. Finally, we rarely perceive art works in isolation. We know and appreciate the fact that a certain work has been created by a certain person in a certain time.

ukj 6y ago
This is the same argument people made against MP3 compression. Lossy is bad. Humans will never stand for it. Perfections will not stand for it. Pragmatists won’t notice. This isn’t a bad thing. We need perfectionists to drag us across the “good enough” line. Despite our childish kicking&screaming.
- Nimitz14 6y ago
  Absolutely terrible comparison, completely not relevant.
sriku 6y ago
The reality is likely neither here nor there - i.e. computing may have more to offer to the creative endeavor than creators would like to admit, but still leave an obvious gap which technologists might be loathe to admit. It may be instructive to look at David Cope's [1] work (what he calls "recombinant music" [2]). Cope's been writing algorithms to compose in the styles of the masters (Mozart/Chopin/et al) for about 3 decades now, well before the recent surge in "AI". His techniques are much less sexy for the "deep learning" enthusiasts, and yet he managed to outrage an audience of connoisseurs who assembled to listen to a "lost Chopin piece" only to be told, after they shared their applause, that it was composed by a computer taught to mimic Chopin's style (the composition was performed by a musician). The response, in my opinion, also points to music as a social constructed experience and not purely attributable to the sound signal itself. i.e. if I give you a romantic background story for a lost composition of a master, you may be inclined to experience the piece in a more favorable light than if I told you it was generated by an algorithm (or the converse). You're absolutely right that the musical output of the current crop of "AI" projects (especially the ones using deep learning / neural networks) are crappy to even a modestly trained listener .. or even a lay untrained listener for that matter. However, more involved modeling (such as Cope's) has produced some very compelling results decades ago, so it would be a mistake to assume that the current crop won't get close enough [3]. The fact that DL systems don't need to be instructed in the way Cope has had to encode his musical understanding is also something to be considered in the evaluation as well as in scoping their capabilities going forward. [1]: https://en.wikipedia.org/wiki/David_Cope https://en.wikipedia.org/wiki/David_Cope [2]: https://www.recombinantinc.com https://www.recombinantinc.com [3]: https://deepmind.com/blog/article/wavenet-generative-model-raw-audio https://deepmind.com/blog/article/wavenet-generative-model-r... (see "Making Music" section and examples there)

bambax 6y ago

But in the arts, can AI come up with something truly new? This should be testable: train AI on all the music ever written before Bach, and see if it ever produces something ressembling Bach. Maybe that kind of test has alretbeen done; it would be interesting to know what comes out of it.

spyder 6y ago
The GPT-2 based Musenet music generator is already interesting but far from perfect. You can try it in the middle of this article: https://openai.com/blog/musenet/ https://openai.com/blog/musenet/ (you can even upload custom prompts in the advanced mode) Would be interesting to see it with the updated GPT-3. There is also AIVA with more production ready results: https://www.youtube.com/watch?v=gzGkC_o9hXI&list=PLv7BOfa4CxsHAMHQj0ScPXSbgBlLglRPo&index=1 https://www.youtube.com/watch?v=gzGkC_o9hXI&list=PLv7BOfa4Cx... Not sure how it works, but it has better results maybe because it's using more predefined components and less AI so it's also less "creative". More AI music projects here: https://magenta.tensorflow.org/ https://magenta.tensorflow.org/
an_opabinia 6y ago
Obviously no. But there’s so much classical music out there, that an average person would never be able to tell the difference between something that is generated anew and something just really obscure. Have you ever tried copying and pasting sections of GPT output into Google?
p1esk 6y ago
This should be testable There have been music resembling Bach written before Bach (e.g. https://www.youtube.com/watch?v=VUcdBz3LIuU https://www.youtube.com/watch?v=VUcdBz3LIuU). How much more of resemblance you hope for?

Isinlor 6y ago

> It's not clear to me that larger models will solve this in the limit; take the way GPT3 fails on addition past a certain number, and the fundamental inability for transformers to learn certain algorithms. GPT-3 was OpenAI exercise in how far pure scaling can get you. They have used some 2 years old method. Already at the point when they started training GPT-3 there were readily available remedies to many of GPT-3 issues. Given how they energized the wider community I'm sure even more focus will be given to improving language models in the following years. Some rough ideas right now: - People think that cherry-picking the best GPT-3 examples is cheating - why? Train a model that will be selecting the best examples for you. My proposition is to train a model that guesses whether some text was GPT-3 generated or human made - select samples that look the most human like. - Use a good search method to look for the best samples. Monte Carlo Tree Search? AlphaZero? MuZero? If MuZero can play a games of Chess, Shogi, Go and all of Atari then way should it not be able to play a game of what word will come next? - Hook up the language model to a search engine. Instead of writing a whole program yourself, why not to copy-paste some stuff from StackOverflow with some slight modifications? Etc. It doesn't address the issues with agency, grounding and multi-modality, but it's a good road map for the next 2-3 years.

p1esk 6y ago
train a model that guesses whether some text was GPT-3 generated or human made - select samples that look the most human like. What you said is essentially: "Train a better GPT model". Humans have trouble distinguishing between (some of) GPT-3 and human writing. The only way to build a classifier that can do this is to build a model that is better than GPT-3 at understanding text. It would need to have features currently absent in GPT-3, such as common sense and understanding the world (e.g. causality, physics, psychology, history, etc). If what you say could be done, GPT-3 would have been designed as a GAN.
- Isinlor 6y ago
  It's a lot easier to notice logical mistakes in already written text, than it is to avoid making them in the first place. When you write text do you write it in one pass or do you read yourself and fix mistakes, reformulate sentences etc.? I have reformulated this piece of text at least once in order to make my argument clear. That's the difference between GPT and BERT. GPT can only attend to the past outputs, while BERT one can attend also to the future outputs. Now imagine that what you are going to say is not actually determined by you, but it is sampled randomly from what seems like a reasonable thing to say. This is how GPT-3 works. If somebody ask you some kind of question you can guess 70% yes or 30% no, then roll a 10 side dice to pick one, but once you pick there is no way back. And I already mentioned that it does not address agency, grounding and multi-modality, but it could improve GPT ability to formulate coherent arguments, follow instructions, write mathematical proofs and computer programs or play games. BTW - I actually have implemented it and it works quite reasonably. Here are samples from GPT-2 small and GPT-2 small + RoBERTa adversarial decoder. https://github.com/Isinlor/AdvDecoder/tree/master/outputs https://github.com/Isinlor/AdvDecoder/tree/master/outputs

karmakaze 6y ago

A better or hopeful projection is that "creative" things will split into casually consumed which is largely automated and more active/deeply experienced content which will be human made or directed. The first already exists in formulaic content generated by humans with little consideration for a cohesive story without self contradiction. I don't know which way things will go. Will newer and later generations be accustomed to and accept lower fidelity art? the uncanny valley be bridged from both sides? Or will there be attention being drawn to what is 'real' vs 'synthetic'. Good art is pain. Labelling these things distinctly will probably reveal that I consume some 'real', annoyed by some 'synthetic' while enjoying as much. This will get challenging as machine generated can seem more 'real' than much human made content: 'real' is/was a subset of human made, machine made is/was a subset of 'synthetic'. This line of reasoning leads me to believe that premium content will be interactive. This means that the content has to either have a human connection or be closer and closer to passing a Turing test. The current examples of machine made static content wont cut it.