4 ms·

Deepfake version of Travis Scott

52 points by saadalem 7y ago

rogerkirkness 7y ago
I feel like it probably took a lot of tweaking and overlaying the more cogent things it generated to make this (?). Very impressive though.
- zitterbewegung 7y ago
  If it takes enough tweaking then how do you know it was by AI? If you don’t see the model or any of the source code or any other process then aren’t you just cherry picking input ?
  - rogerkirkness 7y ago
    I guess there's an argument that the creative process is automated but the production is not.
bdcravens 7y ago
Since all of his music is R&Bish auto-tuned, not that hard for a computer to generate that sound. (I have to admit, I've hit the "get off my lawn" phase, missing the "good ole days" of "real music" ie 90's hip hop)
- monkeyfacebag 7y ago
  I'm not clear what you mean. How does auto-tune make it easier for the computer to generate?
  - PeterisP 7y ago
    Computer generated voice has certain artifacts that makes it a bit distinctive. Auto-tuned natural voice has similar artifacts, so if an artist uses auto-tune, then there's less noticeable difference between their real (computer-altered) voice and a fake computer-generated voice. A more exaggerated example would be artists that use a vocoder (e.g. Daft Punk or something like that), which would be even easier to synthesise convincingly because the real vocals are also mostly synthesised.
    - mirimir 7y ago
      Or VibeSquaD. (not UK Vibe Squad)
  - bdcravens 7y ago
    Less "uncanny valley" affect, because in both cases, you're hearing a synthesized voice.
- sdinsn 7y ago
  They generated the lyrics and melodies using AI. Everything else was done by a person, including the vocals.
  - wwright 7y ago
    Do you have a link on that? I can’t find it in the HN link.
    - sdinsn 7y ago
      See the adweek article that was linked in OP's article, it has more details. The AI part was only lyrics, melodies, and percussion arrangements; not actually the production itself.
    - squeaky-clean 7y ago
      The Adweek source linked goes into a little more about it, but not much. Specifically mentioning a model was used for lyrics first, then the AI created a melodic and percussive arrangement. It doesn't say much more about how it was done. I found another article that states they had a real human voice sing the lyrics (then auto-tuned it to Scott's style), but there's no actual sources here, or on any other articles I can find. https://musebycl.io/digital-data/agency-used-ai-make-travis-bott-bogus-travis-scott https://musebycl.io/digital-data/agency-used-ai-make-travis-...
      - wwright 7y ago
        I did follow through to Adweek, but given that it is Adweek and not NME, I wasn’t sure how literally to take the word “arrangement” there :-) Thanks!
  - bdcravens 7y ago
    I didn't get that from the article.
    - sdinsn 7y ago
      See the adweek article that was linked in OP's article, it has more details. The AI part was only lyrics, melodies, and percussion arrangements; not actually the production itself.
- dropoutcoder 7y ago
  The irony being that the 90’s hip hop sound was largely influenced by MPC60/3000 quantization, amongst other drum machines.
  - djmips 7y ago
    Except for J Dilla. https://www.npr.org/sections/ablogsupreme/2013/02/07/171349007/why-j-dilla-may-be-jazzs-latest-great-innovator https://www.npr.org/sections/ablogsupreme/2013/02/07/1713490...
- scollet 7y ago
  I can't parse this comment. What is "R&Bish"? Why does that matter that it's easy? What makes 90's hip-hop "real" or implicitly superior to the last two decades?
  - chestervonwinch 7y ago
    R&B: https://en.wikipedia.org/wiki/Rhythm_and_blues https://en.wikipedia.org/wiki/Rhythm_and_blues ish: https://www.dictionary.com/browse/ish https://www.dictionary.com/browse/ish So, having an aesthetic similar to rhythm and blues.
RcouF1uZ4gsC 7y ago
If we take created by AI to mean: machine learning generates a crazy amount of some things and then humans pick and release the most interesting one, then AI can create some pretty amazing stuff.
tcbawo 7y ago
I'm certain that as the famous musicians die off or retire from performing, there will be a huge business of recreating their sound and generating new material.
- RcouF1uZ4gsC 7y ago
  This will be kind of sad since this will limit the market for new musicians and new styles. We will get so caught up with preserving the past, that we will ignore opportunities of the present.
  - 52-6F-62 7y ago
    I like to give humanity the benefit of the doubt in these cases. I’m not sure if that’s right, but I’d certainly rather not let it taint my appreciation of music. I don’t think it will stop people from creating either. But you might, unfortunately, be onto something with regard to the market. Maybe less because of what people appreciate, and more because the suits will pursue the lowest common denominator with the highest revenue potential.
  - birdyrooster 7y ago
    Everything is a remix, everything contemporary has history.
- a_bonobo 7y ago
  I've posted this on HN before, NHK 'raised' one dead singer who sang a new song for this year's New Years special using similar tech: https://www.japantimes.co.jp/culture/2020/01/10/films/nhk-raises-dead-kohaku-mixed-reviews/?hootPostID=56dfc4a008194c25483ea66ed5978aaf https://www.japantimes.co.jp/culture/2020/01/10/films/nhk-ra... It's starting!
- krapp 7y ago
  Using famous actors for promotions after they die is already a business[0]. But why bother with living musicians or their estates at all? Use deepfake technology to generate a physical persona, and AI to generate a voice and style, all unique enough to be legally distinct and copyrightable, and fully owned by a company. Live performances can even be done via hologram. I can easily imagine a future where most creative media, including movies, is entirely or almost entirely AI generated. Certainly, actors and performers won't be real people. [0]https://www.vice.com/en_us/article/539a5z/hollywoods-post-biological-future-where-actors-can-perform-after-death https://www.vice.com/en_us/article/539a5z/hollywoods-post-bi...
etaioinshrdlu 7y ago
Imagine a GPT2-like algorithm trained on all the world's music. It could then generate practically any style and probably quite creatively. Then fine-tune it on a particular artist's style. It will then mimic that artist. Depending how strongly you fine tune it, you can bias more strongly towards the target artist or towards general music. You could then generate music in the same general style as an artist but taking any amount of inspiration from the rest of the world of music. I imagine with enough data and the right algorithms it would work very well and sound fantastic. The same applies to visual works which I'm sure the reader is familiar with. Just remember the algorithms we have today are the starting point not the ending point.
- emptybits 7y ago
  > It could then generate practically any style and probably quite creatively. Mimicking a style... easy. Delivering a coherent message or story... harder. Emotional impact... even harder. IMO. But I think these are great goals for an Artistic Turing Test.
  - krapp 7y ago
    Most popular music doesn't deliver a coherent message or emotional impact. Some genres (i'm looking at you mumble rap) don't even require coherent language.
    - Swizec 7y ago
      > Most popular music doesn't deliver a coherent message or emotional impact. It does via the Forer Effect. The goal is to make generic statements and tell the listener that you’re talking about them (or someone just like them). Listeners then subconsciously fill in the blanks and create the most relatable impactful art imaginable. That’s why beauty is in the eye of the beholder. You need the audience to impart the meaning ;) https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect
    - meowface 7y ago
      You don't need interpretable language for emotional impact or even narrative. Have you ever felt moved by a song in a language you don't know a word of, or a purely instrumental song? The same's true of paintings, movies without dialogue, etc. Mumble rap is a totally legitimate art form. I just tend to not be a huge fan of most of the song structures and compositions, personally.
d1zzy 7y ago
Check out AI made black metal, it's not fantastic but also well above garbage. https://dadabots.bandcamp.com/album/coditany-of-timeness https://dadabots.bandcamp.com/album/coditany-of-timeness
farouqaldori 7y ago
There isn't any resemblance to Travis Scott besides the obvious "It's lit"-adlibs to be honest. I loved the parts where they showed some PHP echoing some HTML to give the video that hacker vibe.
basilamer 7y ago
The voice was re-recorded and mixed to sound close to Travis's real vocals; no digital voice reconstruction. However, the reconstruction of his cadence/melodic style is cool to hear!
- wyxuan 7y ago
  Yeah, if it was like that it would be truly impressive. There project is still cool nonetheless.

52 points by saadalem 7y ago

I feel like it probably took a lot of tweaking and overlaying the more cogent things it generated to make this (?). Very impressive though.

zitterbewegung 7y ago
If it takes enough tweaking then how do you know it was by AI? If you don’t see the model or any of the source code or any other process then aren’t you just cherry picking input ?
- rogerkirkness 7y ago
  I guess there's an argument that the creative process is automated but the production is not.

bdcravens 7y ago

Since all of his music is R&Bish auto-tuned, not that hard for a computer to generate that sound. (I have to admit, I've hit the "get off my lawn" phase, missing the "good ole days" of "real music" ie 90's hip hop)

monkeyfacebag 7y ago
I'm not clear what you mean. How does auto-tune make it easier for the computer to generate?
- PeterisP 7y ago
  Computer generated voice has certain artifacts that makes it a bit distinctive. Auto-tuned natural voice has similar artifacts, so if an artist uses auto-tune, then there's less noticeable difference between their real (computer-altered) voice and a fake computer-generated voice. A more exaggerated example would be artists that use a vocoder (e.g. Daft Punk or something like that), which would be even easier to synthesise convincingly because the real vocals are also mostly synthesised.
  - mirimir 7y ago
    Or VibeSquaD. (not UK Vibe Squad)
- bdcravens 7y ago
  Less "uncanny valley" affect, because in both cases, you're hearing a synthesized voice.
sdinsn 7y ago
They generated the lyrics and melodies using AI. Everything else was done by a person, including the vocals.
- wwright 7y ago
  Do you have a link on that? I can’t find it in the HN link.
  - sdinsn 7y ago
    See the adweek article that was linked in OP's article, it has more details. The AI part was only lyrics, melodies, and percussion arrangements; not actually the production itself.
  - squeaky-clean 7y ago
    The Adweek source linked goes into a little more about it, but not much. Specifically mentioning a model was used for lyrics first, then the AI created a melodic and percussive arrangement. It doesn't say much more about how it was done. I found another article that states they had a real human voice sing the lyrics (then auto-tuned it to Scott's style), but there's no actual sources here, or on any other articles I can find. https://musebycl.io/digital-data/agency-used-ai-make-travis-bott-bogus-travis-scott https://musebycl.io/digital-data/agency-used-ai-make-travis-...
    - wwright 7y ago
      I did follow through to Adweek, but given that it is Adweek and not NME, I wasn’t sure how literally to take the word “arrangement” there :-) Thanks!
- bdcravens 7y ago
  I didn't get that from the article.
  - sdinsn 7y ago
    See the adweek article that was linked in OP's article, it has more details. The AI part was only lyrics, melodies, and percussion arrangements; not actually the production itself.
dropoutcoder 7y ago
The irony being that the 90’s hip hop sound was largely influenced by MPC60/3000 quantization, amongst other drum machines.
- djmips 7y ago
  Except for J Dilla. https://www.npr.org/sections/ablogsupreme/2013/02/07/171349007/why-j-dilla-may-be-jazzs-latest-great-innovator https://www.npr.org/sections/ablogsupreme/2013/02/07/1713490...
scollet 7y ago
I can't parse this comment. What is "R&Bish"? Why does that matter that it's easy? What makes 90's hip-hop "real" or implicitly superior to the last two decades?
- chestervonwinch 7y ago
  R&B: https://en.wikipedia.org/wiki/Rhythm_and_blues https://en.wikipedia.org/wiki/Rhythm_and_blues ish: https://www.dictionary.com/browse/ish https://www.dictionary.com/browse/ish So, having an aesthetic similar to rhythm and blues.

RcouF1uZ4gsC 7y ago

If we take created by AI to mean: machine learning generates a crazy amount of some things and then humans pick and release the most interesting one, then AI can create some pretty amazing stuff.

tcbawo 7y ago

I'm certain that as the famous musicians die off or retire from performing, there will be a huge business of recreating their sound and generating new material.

RcouF1uZ4gsC 7y ago
This will be kind of sad since this will limit the market for new musicians and new styles. We will get so caught up with preserving the past, that we will ignore opportunities of the present.
- 52-6F-62 7y ago
  I like to give humanity the benefit of the doubt in these cases. I’m not sure if that’s right, but I’d certainly rather not let it taint my appreciation of music. I don’t think it will stop people from creating either. But you might, unfortunately, be onto something with regard to the market. Maybe less because of what people appreciate, and more because the suits will pursue the lowest common denominator with the highest revenue potential.
- birdyrooster 7y ago
  Everything is a remix, everything contemporary has history.
a_bonobo 7y ago
I've posted this on HN before, NHK 'raised' one dead singer who sang a new song for this year's New Years special using similar tech: https://www.japantimes.co.jp/culture/2020/01/10/films/nhk-raises-dead-kohaku-mixed-reviews/?hootPostID=56dfc4a008194c25483ea66ed5978aaf https://www.japantimes.co.jp/culture/2020/01/10/films/nhk-ra... It's starting!
krapp 7y ago
Using famous actors for promotions after they die is already a business[0]. But why bother with living musicians or their estates at all? Use deepfake technology to generate a physical persona, and AI to generate a voice and style, all unique enough to be legally distinct and copyrightable, and fully owned by a company. Live performances can even be done via hologram. I can easily imagine a future where most creative media, including movies, is entirely or almost entirely AI generated. Certainly, actors and performers won't be real people. [0]https://www.vice.com/en_us/article/539a5z/hollywoods-post-biological-future-where-actors-can-perform-after-death https://www.vice.com/en_us/article/539a5z/hollywoods-post-bi...

etaioinshrdlu 7y ago

Imagine a GPT2-like algorithm trained on all the world's music. It could then generate practically any style and probably quite creatively. Then fine-tune it on a particular artist's style. It will then mimic that artist. Depending how strongly you fine tune it, you can bias more strongly towards the target artist or towards general music. You could then generate music in the same general style as an artist but taking any amount of inspiration from the rest of the world of music. I imagine with enough data and the right algorithms it would work very well and sound fantastic. The same applies to visual works which I'm sure the reader is familiar with. Just remember the algorithms we have today are the starting point not the ending point.

emptybits 7y ago
> It could then generate practically any style and probably quite creatively. Mimicking a style... easy. Delivering a coherent message or story... harder. Emotional impact... even harder. IMO. But I think these are great goals for an Artistic Turing Test.
- krapp 7y ago
  Most popular music doesn't deliver a coherent message or emotional impact. Some genres (i'm looking at you mumble rap) don't even require coherent language.
  - Swizec 7y ago
    > Most popular music doesn't deliver a coherent message or emotional impact. It does via the Forer Effect. The goal is to make generic statements and tell the listener that you’re talking about them (or someone just like them). Listeners then subconsciously fill in the blanks and create the most relatable impactful art imaginable. That’s why beauty is in the eye of the beholder. You need the audience to impart the meaning ;) https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect
  - meowface 7y ago
    You don't need interpretable language for emotional impact or even narrative. Have you ever felt moved by a song in a language you don't know a word of, or a purely instrumental song? The same's true of paintings, movies without dialogue, etc. Mumble rap is a totally legitimate art form. I just tend to not be a huge fan of most of the song structures and compositions, personally.

d1zzy 7y ago

Check out AI made black metal, it's not fantastic but also well above garbage. https://dadabots.bandcamp.com/album/coditany-of-timeness https://dadabots.bandcamp.com/album/coditany-of-timeness

farouqaldori 7y ago

There isn't any resemblance to Travis Scott besides the obvious "It's lit"-adlibs to be honest. I loved the parts where they showed some PHP echoing some HTML to give the video that hacker vibe.

basilamer 7y ago

The voice was re-recorded and mixed to sound close to Travis's real vocals; no digital voice reconstruction. However, the reconstruction of his cadence/melodic style is cool to hear!

wyxuan 7y ago
Yeah, if it was like that it would be truly impressive. There project is still cool nonetheless.