4 ms·

This is something I've been working on https://www.youtube.com/watch?v=52qWiLoOeIQ https://www.youtube.com/watch?v=52qWiLoOeIQ you should clearly see the connec

by McIceT 7y ago

This is something I've been working on https://www.youtube.com/watch?v=52qWiLoOeIQ https://www.youtube.com/watch?v=52qWiLoOeIQ you should clearly see the connection between the morphs and the music

jressey 7y ago
I'm blown away. So the glasses are always present when the bass/808s are hitting, so is there something that maps the sound to the images? What is it about the algorithms that make the images 'dance' so quickly between the 3.5 beat and the 1? Is it because there are static risers that move so quickly through the wave spectrum? Wait... is light skin mapped to when highs dominate and dark skin to lows?
- McIceT 7y ago
  I'm glad you like it! Actually compared to the linked post I don't do any manual latent-space representation selection. It's just a bit of "smart" signal processing. I've written a framework that makes it really simple to do these visualizations (not open-source yet). Here's one more example: https://www.youtube.com/watch?v=X4r4njUjE2M https://www.youtube.com/watch?v=X4r4njUjE2M
  - isoprophlex 7y ago
    Jesus holy fuck that's SUPER IMPRESSIVE I'd be delighted to learn how you translate musical structure to a point in latent space!
  - JonathanFly 7y ago
    Looking forward to when you release this, would love to try this with other models!
  - jcims 7y ago
    It could just be my brain, but it seems like there is a loose correlation between the mouths in the video and the lyrics. In Phantom Part II they mostly have their mouths closed. In La La Land it varies but the mouths are mostly open. If you focus on the mouth you'll get little mental radar blips where the mouth could be tracking what is being said. "Pretty girl and you let go" - https://youtu.be/52qWiLoOeIQ?t=18 https://youtu.be/52qWiLoOeIQ?t=18 "If you wanna waste time baby" - https://youtu.be/52qWiLoOeIQ?t=44 https://youtu.be/52qWiLoOeIQ?t=44 "Yeah i met her at a one oak" - https://youtu.be/52qWiLoOeIQ?t=84 https://youtu.be/52qWiLoOeIQ?t=84 (esp the one oak part) Anyone that actually watches these will probably just think I'm low on sleep, but it's kind of interesting.
  - eufouria 7y ago
    Cool stuff! Would be interested in knowing if and when this library becomes open source. Where could I follow you?
    - McIceT 7y ago
      You could follow me on twitter @tsmcalister. I'll post there once it's released. Depends a lot on how much time I have to work on it. Hopefully by the end of the month!
jcims 7y ago
For some reason this made me yearn for a GAN that generates motorcycles riding through landscapes. Love the storytelling potential of your work, great stuff!
TeMPOraL 7y ago
Absolutely. Amazing. I was waiting for someone to try this, and you did, and the results are way more impressive than I imagined they would be. (Try this with different kinds of songs if you can!)
- McIceT 7y ago
  Any specific song request? :)

by McIceT 7y ago

This is something I've been working on https://www.youtube.com/watch?v=52qWiLoOeIQ https://www.youtube.com/watch?v=52qWiLoOeIQ you should clearly see the connection between the morphs and the music

jressey 7y ago

I'm blown away. So the glasses are always present when the bass/808s are hitting, so is there something that maps the sound to the images? What is it about the algorithms that make the images 'dance' so quickly between the 3.5 beat and the 1? Is it because there are static risers that move so quickly through the wave spectrum? Wait... is light skin mapped to when highs dominate and dark skin to lows?

McIceT 7y ago
I'm glad you like it! Actually compared to the linked post I don't do any manual latent-space representation selection. It's just a bit of "smart" signal processing. I've written a framework that makes it really simple to do these visualizations (not open-source yet). Here's one more example: https://www.youtube.com/watch?v=X4r4njUjE2M https://www.youtube.com/watch?v=X4r4njUjE2M
- isoprophlex 7y ago
  Jesus holy fuck that's SUPER IMPRESSIVE I'd be delighted to learn how you translate musical structure to a point in latent space!
- JonathanFly 7y ago
  Looking forward to when you release this, would love to try this with other models!
- jcims 7y ago
  It could just be my brain, but it seems like there is a loose correlation between the mouths in the video and the lyrics. In Phantom Part II they mostly have their mouths closed. In La La Land it varies but the mouths are mostly open. If you focus on the mouth you'll get little mental radar blips where the mouth could be tracking what is being said. "Pretty girl and you let go" - https://youtu.be/52qWiLoOeIQ?t=18 https://youtu.be/52qWiLoOeIQ?t=18 "If you wanna waste time baby" - https://youtu.be/52qWiLoOeIQ?t=44 https://youtu.be/52qWiLoOeIQ?t=44 "Yeah i met her at a one oak" - https://youtu.be/52qWiLoOeIQ?t=84 https://youtu.be/52qWiLoOeIQ?t=84 (esp the one oak part) Anyone that actually watches these will probably just think I'm low on sleep, but it's kind of interesting.
- eufouria 7y ago
  Cool stuff! Would be interested in knowing if and when this library becomes open source. Where could I follow you?
  - McIceT 7y ago
    You could follow me on twitter @tsmcalister. I'll post there once it's released. Depends a lot on how much time I have to work on it. Hopefully by the end of the month!

jcims 7y ago

For some reason this made me yearn for a GAN that generates motorcycles riding through landscapes. Love the storytelling potential of your work, great stuff!

TeMPOraL 7y ago

Absolutely. Amazing. I was waiting for someone to try this, and you did, and the results are way more impressive than I imagined they would be. (Try this with different kinds of songs if you can!)

McIceT 7y ago
Any specific song request? :)