4 ms·

I know that image generation with the latest models (DALL-E2, Stable Diffusion) has really captured collective attention as being mind-blowing. But as someone

I know that image generation with the latest models (DALL-E2, Stable Diffusion) has really captured collective attention as being mind-blowing. But as someone who has been working with ML-based audio/music generation [1]... continue to think that what Jukebox did with making songs is another level altogether. Yes its not as user-friendly or approachable (hence harder to dissipate to general use)... but I feel like that space is where image-generation/mods were 5+ years ago with generic style-transfer models... can't wait for Jukebox 2 (or similar) to come out with higher fidelity. I absolutely dont see Jukebox as "taking over/replacing" music... like all ML/AI models... it is yet another tool. "Learning to use Jukebox" (or DALL-E or StableDiffusion) is like "learning to play the guitar/learning to paint"... no need to be afraid. [1] https://songshtr.github.io https://songshtr.github.io

p1esk 4y ago
How do you explain zero progress in raw audio based music generation since Jukebox paper? In the last two years we've seen huge progress in both NLP and image generation, but not in music.
- songeater 4y ago
  in my opinion (again I'm a user / not a developer) feel like the data/computing/training requirements are MUCH higher. Think about the RGB values / pixel in an image vs raw audio waveform.... just takes more $$$. Let me see if I can find a note where someone calculated the cost to train Jukebox... was definitely in tens of millions.
  - p1esk 4y ago
    According to the paper, Jukebox was trained for approximately 7 weeks on 512 V100 GPUs. That's ~300k per training run assuming $0.5/hr/GPU. Obviously you need several runs to experiment with, so I'm guessing 1-2M to develop something similar. But sure, for a much larger Jukebox-2 it could be 10x more. Still, OpenAI has plenty of funding, so it's either they don't see much value in developing it further, or they can't improve it. I'm surprised that no one else does either. For example, with GPT models every big AI lab started to compete trying to outdo each other. Same is happening now with DALL-E. But no one has tried to train a Jukebox successor.
hazmazlaz 4y ago
I am terribly sorry to point this out, but reading that URL I cannot hear anything in my head except "Song-Shitter". Perhaps that's apropos though? (no offense intended, it's a super cool project!)
- songeater 4y ago
  yes it IS a song-shitter... it eats music on the way in / shits out more ;) (software eats the world... hardware shits it out) = thats the tag line haha.

p1esk 4y ago

How do you explain zero progress in raw audio based music generation since Jukebox paper? In the last two years we've seen huge progress in both NLP and image generation, but not in music.

songeater 4y ago
in my opinion (again I'm a user / not a developer) feel like the data/computing/training requirements are MUCH higher. Think about the RGB values / pixel in an image vs raw audio waveform.... just takes more $$$. Let me see if I can find a note where someone calculated the cost to train Jukebox... was definitely in tens of millions.
- p1esk 4y ago
  According to the paper, Jukebox was trained for approximately 7 weeks on 512 V100 GPUs. That's ~300k per training run assuming $0.5/hr/GPU. Obviously you need several runs to experiment with, so I'm guessing 1-2M to develop something similar. But sure, for a much larger Jukebox-2 it could be 10x more. Still, OpenAI has plenty of funding, so it's either they don't see much value in developing it further, or they can't improve it. I'm surprised that no one else does either. For example, with GPT models every big AI lab started to compete trying to outdo each other. Same is happening now with DALL-E. But no one has tried to train a Jukebox successor.

hazmazlaz 4y ago

I am terribly sorry to point this out, but reading that URL I cannot hear anything in my head except "Song-Shitter". Perhaps that's apropos though? (no offense intended, it's a super cool project!)

songeater 4y ago
yes it IS a song-shitter... it eats music on the way in / shits out more ;) (software eats the world... hardware shits it out) = thats the tag line haha.