4 ms·

Sorry, where did we all get the idea that we want to use text to generate music? That thought really has never entered my head, despite many people trying to sh

Sorry, where did we all get the idea that we want to use text to generate music? That thought really has never entered my head, despite many people trying to shove this UX down my throat. Is it just that we don't have creative thinking about what a transformative music creation UX should be? "Writing about music is like dancing about architecture."

cqqxo4zV46cp 3y ago
I highly doubt that anyone cares about you, specifically, enough, to be ‘shoving this down your throat’. Plenty of people have been enjoying this category of software. Sorry to hear that you don’t like it. That doesn’t mean that the world agrees with you.
- bravura 3y ago
  Look, I know I am coming across a bit of a prick. "Plenty of people have been enjoying this category of software." Really? I ask sincerely. Because it seems to me that this category of software is mostly a curiosity / novelty. Are there actual publishing musicians who have used text-to-music as part of their workflow? My perception is that no one actually uses these tools for real music creation. Hence my point that they are novelty and not actually impactful.
  - bravura 3y ago
    And just to contribute to the discussion in a meaningful way: The problem with text-to-music isn't really that text is a poor UX for music description, but also that it's almost definitely a one-shot process. And any sort of turn-taking with the user isn't really co-collaborative, it's really just being forced to accept or reject what the AI has generated, with little feedback or control possible. That's why I think rethinking the entire UX from a musician centric perspective (rather than: "what's the easiest thing for me to specify as a tool-builder? I know, text to music") is such an overlooked endeavor. I'm much more bullish about things like the anticipatory music transformer, where there is novel contribution on the UX and not just the ML: https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.html https://crfm.stanford.edu/2023/06/16/anticipatory-music-tran...
  - bongodongobob 3y ago
    Oh they are definitely using it, but no one is going to admit it. You misunderstand the workflow. I'm not trying to create a text prompt that generates hits. It's a way to get ideas and find interesting stuff rather than doing it manually with a guitar or on the piano. Then you have to walk away for some time to clean your sonic pallette and try again. As a lifelong musician I am absolutely loving all the conversation around this stuff. Musicians steal everything, all the time. The idea of some genius that sits down and pops out a hit in 30 minutes that comes deep from within their soul and has all the meaning attached to it is a myth. We're trying to make things people can connect with. Yes, some musicians are doing it for the sake of art, but not most, and even with a lot of those, it's just an image they project. When you hear a song, it has probably gone through half a dozen different versions and was worked on for months. These new AI music generation tools will churn out 100 different ideas from you to borrow from and cut the revisions down quite a bit I think.
  - glimshe 3y ago
    I would love to use it if I can add enough variation to the song and it can generate a good backing for the lyrics. I'm not a "publishing musician", but the music market is much bigger than music performers - for instance, music for advertisements, presentation, video backing tracks etc.
  - earslap 3y ago
    Music is not constrained to... publishing musicians, and "real music" is not defined by them. The fact that novelists are not depending on LLMs to write their next book does not mean that they are not (or will not be) impactful. From a product point of view, you have an audience of billions of non-musicians to begin with. You surely can't pretend to have the final say on people's relationship with how they like to enjoy their music or other sound content. Besides, even a novelty can evolve to something not imaginable today. The fact that we can go that far with text only at this early stage is a sign of things to come. When creating something, there is a place for expressing your intent with language. Maybe not all of it all the time, but we'll see how things will evolve.
  - deleted 3y ago
    [deleted]
subpixel 3y ago
Nobody ever said “this store could use some soul-sucking Muzak” but that didn’t stop progress!
- birracerveza 3y ago
  Near my house there is a supermarket with a in-store radio that every single day plays the exact same playlist. If you go at the same hour in two different days, you'll hear the same song. And it's not good songs either. It's bottom of the barrel, a penny a dozen kind of music if you know what I'm saying. They probably do this because it's cheaper than paying rights to play an actual radio. Now, how the staff survives this kind of psychological torture, I don't know. What I do know is that both staff and the customers would be very glad to listen to anything else, including AI generated music. It's already miles better than some "music" we are subjected to... especially pop music, which is somehow more artificial than actual AI generated music. There are a lot of places where I'd rather might want actual music, even if AI generated, instead of repeating the same sample over and over because you do not have enough budget to spend on it for something that is at least not completely irritating.
mitthrowaway2 3y ago
I'm waiting for the "listen to me hum a tune and tap a beat, and turn it into a {fully orchestrated / heavy metal / pop / chiptune / etc} clip, then let me string a few of those together, blend them smoothly, and add lyrics" app.
- Terretta 3y ago
  With image prompts for text-to-image generation, why not whistle or hum prompts to music generation? It's baffling this isn't a thing.
- pksebben 3y ago
  We'd have to orchestrate an effort to compile the dataset. Have people listen to their favorite songs, then play dashboard drums and hum about it. Record these tagged with the actual song. You'd need a lot, though. Unless you could somehow fuzz the data to create synthetic pairings, it'd take a lot of people recording themselves sounding awful (which could be fun just by itself now that I think about it).
  - HeatrayEnjoyer 3y ago
    If you believe the rumors OpenAI has already done this, but not published it because the RIAA/music industry has more copyright case law firepower than text or visual domains.
- voltaireodactyl 3y ago
  Apple Music Memos was pretty much this and I miss it.
- throwaway290 3y ago
  What would be your contribution, a hum and a tempo? The rest is regurgitated music from preexisting musicians? Hopefully this app never gets good:)
- cooper_ganglia 3y ago
  Yeah, to me this is the inevitable holy grail. I think you have to iterate and optimize your way there, and the current clunky methods of using text to generate an entire song is just the first step, but I can’t imagine this product won’t exist sometime within the next 1-3 years.
Archelaos 3y ago
> "Writing about music is like dancing about architecture." Here is a short essay about "The Similarity between architecture and dance": https://www.re-thinkingthefuture.com/rtf-fresh-perspectives/a4080-the-similarity-between-architecture-and-dance/ https://www.re-thinkingthefuture.com/rtf-fresh-perspectives/...
- jachee 3y ago
  “Dancing is like architecture.” is not dancing about architecture.
  - Archelaos 3y ago
    Sure. With my comment I want to expant on the thought, not necessarily contradict it.
blueboo 3y ago
Natural language is how we talk about music and we’ve been talking about music since time immemorial. What’s your favourite music? Why?
plastic3169 3y ago
I think we have the same problems with text-to-image. These UX will not be the final interfaces, but amazing first proof of concepts. It is so striking that computer can comprehend text and generate something based on these abstract meanings, everybodys mind is blown. Of course when you try to do anything with them you notice what you said. The UX is so bad and inaccurate. There is no need to transfer across domain like this. Best way to discuss music is by using mostly sound and less words. Same with images. And the right level is somewhere below this. You really want a tool that you can use to help you not a generator that tries to go from nothing to finished in one go.
addandsubtract 3y ago
Even Beethoven has been known for using strings.
kristianbrigman 3y ago
‘Siri, create me a song that mixes the styles of Lettuce, Solarstone, and Bollywood, and make the vocals Tuvan throat singers.’
pfannkuchen 3y ago
It’s a tech demo that is accessible to non producers, not a real music creation tool.
c1sc0 3y ago
Because to the cerebral and analytical mind text rules everything, it’s the fountain of all knowledge
bravura 3y ago
UPDATE: I tried Udio and it is very cool, I apologize for being snarky.

cqqxo4zV46cp 3y ago

I highly doubt that anyone cares about you, specifically, enough, to be ‘shoving this down your throat’. Plenty of people have been enjoying this category of software. Sorry to hear that you don’t like it. That doesn’t mean that the world agrees with you.

bravura 3y ago
Look, I know I am coming across a bit of a prick. "Plenty of people have been enjoying this category of software." Really? I ask sincerely. Because it seems to me that this category of software is mostly a curiosity / novelty. Are there actual publishing musicians who have used text-to-music as part of their workflow? My perception is that no one actually uses these tools for real music creation. Hence my point that they are novelty and not actually impactful.
- bravura 3y ago
  And just to contribute to the discussion in a meaningful way: The problem with text-to-music isn't really that text is a poor UX for music description, but also that it's almost definitely a one-shot process. And any sort of turn-taking with the user isn't really co-collaborative, it's really just being forced to accept or reject what the AI has generated, with little feedback or control possible. That's why I think rethinking the entire UX from a musician centric perspective (rather than: "what's the easiest thing for me to specify as a tool-builder? I know, text to music") is such an overlooked endeavor. I'm much more bullish about things like the anticipatory music transformer, where there is novel contribution on the UX and not just the ML: https://crfm.stanford.edu/2023/06/16/anticipatory-music-transformer.html https://crfm.stanford.edu/2023/06/16/anticipatory-music-tran...
- bongodongobob 3y ago
  Oh they are definitely using it, but no one is going to admit it. You misunderstand the workflow. I'm not trying to create a text prompt that generates hits. It's a way to get ideas and find interesting stuff rather than doing it manually with a guitar or on the piano. Then you have to walk away for some time to clean your sonic pallette and try again. As a lifelong musician I am absolutely loving all the conversation around this stuff. Musicians steal everything, all the time. The idea of some genius that sits down and pops out a hit in 30 minutes that comes deep from within their soul and has all the meaning attached to it is a myth. We're trying to make things people can connect with. Yes, some musicians are doing it for the sake of art, but not most, and even with a lot of those, it's just an image they project. When you hear a song, it has probably gone through half a dozen different versions and was worked on for months. These new AI music generation tools will churn out 100 different ideas from you to borrow from and cut the revisions down quite a bit I think.
- glimshe 3y ago
  I would love to use it if I can add enough variation to the song and it can generate a good backing for the lyrics. I'm not a "publishing musician", but the music market is much bigger than music performers - for instance, music for advertisements, presentation, video backing tracks etc.
- earslap 3y ago
  Music is not constrained to... publishing musicians, and "real music" is not defined by them. The fact that novelists are not depending on LLMs to write their next book does not mean that they are not (or will not be) impactful. From a product point of view, you have an audience of billions of non-musicians to begin with. You surely can't pretend to have the final say on people's relationship with how they like to enjoy their music or other sound content. Besides, even a novelty can evolve to something not imaginable today. The fact that we can go that far with text only at this early stage is a sign of things to come. When creating something, there is a place for expressing your intent with language. Maybe not all of it all the time, but we'll see how things will evolve.
- deleted 3y ago
  [deleted]

subpixel 3y ago

Nobody ever said “this store could use some soul-sucking Muzak” but that didn’t stop progress!

birracerveza 3y ago
Near my house there is a supermarket with a in-store radio that every single day plays the exact same playlist. If you go at the same hour in two different days, you'll hear the same song. And it's not good songs either. It's bottom of the barrel, a penny a dozen kind of music if you know what I'm saying. They probably do this because it's cheaper than paying rights to play an actual radio. Now, how the staff survives this kind of psychological torture, I don't know. What I do know is that both staff and the customers would be very glad to listen to anything else, including AI generated music. It's already miles better than some "music" we are subjected to... especially pop music, which is somehow more artificial than actual AI generated music. There are a lot of places where I'd rather might want actual music, even if AI generated, instead of repeating the same sample over and over because you do not have enough budget to spend on it for something that is at least not completely irritating.

mitthrowaway2 3y ago

I'm waiting for the "listen to me hum a tune and tap a beat, and turn it into a {fully orchestrated / heavy metal / pop / chiptune / etc} clip, then let me string a few of those together, blend them smoothly, and add lyrics" app.

Terretta 3y ago
With image prompts for text-to-image generation, why not whistle or hum prompts to music generation? It's baffling this isn't a thing.
pksebben 3y ago
We'd have to orchestrate an effort to compile the dataset. Have people listen to their favorite songs, then play dashboard drums and hum about it. Record these tagged with the actual song. You'd need a lot, though. Unless you could somehow fuzz the data to create synthetic pairings, it'd take a lot of people recording themselves sounding awful (which could be fun just by itself now that I think about it).
- HeatrayEnjoyer 3y ago
  If you believe the rumors OpenAI has already done this, but not published it because the RIAA/music industry has more copyright case law firepower than text or visual domains.
voltaireodactyl 3y ago
Apple Music Memos was pretty much this and I miss it.
throwaway290 3y ago
What would be your contribution, a hum and a tempo? The rest is regurgitated music from preexisting musicians? Hopefully this app never gets good:)
cooper_ganglia 3y ago
Yeah, to me this is the inevitable holy grail. I think you have to iterate and optimize your way there, and the current clunky methods of using text to generate an entire song is just the first step, but I can’t imagine this product won’t exist sometime within the next 1-3 years.

Archelaos 3y ago

> "Writing about music is like dancing about architecture." Here is a short essay about "The Similarity between architecture and dance": https://www.re-thinkingthefuture.com/rtf-fresh-perspectives/a4080-the-similarity-between-architecture-and-dance/ https://www.re-thinkingthefuture.com/rtf-fresh-perspectives/...

jachee 3y ago
“Dancing is like architecture.” is not dancing about architecture.
- Archelaos 3y ago
  Sure. With my comment I want to expant on the thought, not necessarily contradict it.

blueboo 3y ago

Natural language is how we talk about music and we’ve been talking about music since time immemorial. What’s your favourite music? Why?

plastic3169 3y ago

I think we have the same problems with text-to-image. These UX will not be the final interfaces, but amazing first proof of concepts. It is so striking that computer can comprehend text and generate something based on these abstract meanings, everybodys mind is blown. Of course when you try to do anything with them you notice what you said. The UX is so bad and inaccurate. There is no need to transfer across domain like this. Best way to discuss music is by using mostly sound and less words. Same with images. And the right level is somewhere below this. You really want a tool that you can use to help you not a generator that tries to go from nothing to finished in one go.

addandsubtract 3y ago

Even Beethoven has been known for using strings.

kristianbrigman 3y ago

‘Siri, create me a song that mixes the styles of Lettuce, Solarstone, and Bollywood, and make the vocals Tuvan throat singers.’

pfannkuchen 3y ago

It’s a tech demo that is accessible to non producers, not a real music creation tool.

c1sc0 3y ago

Because to the cerebral and analytical mind text rules everything, it’s the fountain of all knowledge

bravura 3y ago

UPDATE: I tried Udio and it is very cool, I apologize for being snarky.