4 ms·

Ah yes, the performance side of it. That's interesting. And I can see why learned timing and articulation can be used to greatly improve the "realism" of that.

Ah yes, the performance side of it. That's interesting. And I can see why learned timing and articulation can be used to greatly improve the "realism" of that. You could even have "celebrity" performer profiles that learn Eric Clapton guitar performances the same way that synthesised voice style can be extracted from a speaker corpus. A tool like that would be really fun and something I'd like to play with. It would take my composition and _perform_ the parts: Drums: Ringo Starr, Solo Guitar: Steve Vai, Bass: Sting In music it's quite usual though to separate the composition, arrangement and performance. TBH, what I found with _all_ ML/AI tools in the composition and orchestration/arrangement stages is badly lacking. Not because the "technology is bad" - massive leaps have been made with gradient descent and transformation, in scale, algorithms and hardware, but because the use paradigm doesn't appeal. It's actually one of the biggest, painful learning experiences in my tech career, and I have taken it to heart - for example with the RJDJ startup we pioneered "reactive music" and had a ton of big artists on board, Chemical Brothers, Hans Zimmer, Massive Attack, .. and we made really credible, innovative music that pretty much defined a new genre and market that didn't exist. But it failed [1]. Why? Because people don't want their music that way. We made the classic mistake of thinking we knew what people wanted because _we_ thought it was cool (and it was, and investors poured money into it). I used to excuse the fail by saying "it was ahead of its time". But as I get older I see that hubris alone was the sword we fell on, by thinking we understood how people want to relate to music. There is such a concept as "fit" - I don't know what else to call it - that tech entrepreneurs often seem to miss. That's why I say that a clunky old thing like MMA is actually a "better" tool, for me personally as a pro/am (once working) musician. I hope that makes sense. [1] AFAIK (I am out of the scene now) one of the people still doing this successfully is Matt from Coldcut with their "Ninja Jam" setup.

Rochus 5y ago
> In music it's quite usual though to separate the composition, arrangement and performance. Well, that's mostly what classic musicology does (and some Jazz musicologists inspired by classics). At the latest when you only have a lead sheet and the "composer" expects the musicians to add all the rest, the interpretation of the piece is also an essential part of the musical content/"composition". > the use paradigm doesn't appeal It makes sense to first make some proofs of concept (as e.g. done with MuseNet) concerning compositional/performance quality and then adapt the technology to specific use cases. From my humble point of view there was no convincing result in that respect before e.g. BachBot or MuseNet - maybe with the exception of Cope's work, but his algorithms actually require a human selection and optimization step for useful results - it's not fully algorithmic. > Why? Because people don't want their music that way. Having the right ideas is just a small fraction of success; most is pure luck (from my humble experience/observations). But solving an important/interesting problem is not identical with pleasing a mass market. Concerning music composition and performance there are still big unsolved representation problems, i.e. how to model this n-dimensional problem so that all relevant patterns along each subset of dimensions can be properly described and related to other patterns/features. MuseNet has an interesting approach, but it's mostly a spin-off from linguistics and from my point of view still far from what we should have. There is no satisfactory solution since the beginning of music theory and especially since the beginning of the AI era. All we have are only rudimentary projections, makeshifts to have anything at all. That's why a "a clunky old thing like MMA" will never pass a trained musicians "Turing test".
- PaulDavisThe1st 5y ago
  > That's why a "a clunky old thing like MMA" will never pass a trained musicians "Turing test". Most good musicians will have distinct assessments of other musicians. "Good" here is a metric that involves a combination of time playing, time performing, time spent collaborating and possibly some innate abilities, should such a thing exist. "I really enjoy playing with them, their sense of timing is so great" "They are really good technically but their development ideas just don't go where I go" etc. etc. There's no reason to expect that any computationally based accompaniment system will do any better than the human ones: if it's actually good, it will have a "personality" that won't work for some people; if it doesn't have a "personality", it almost certainly isn't good. There are many exceptionally talented musicians out there, measured by their technical abilities on their instruments. There are far, far fewer musicians that lots of other musicians want to play with.
  - TheOtherHobbes 5y ago
    "Good" is context dependent, so any credible backing generation system would have to be aware of context - ideally while playing with others. Which is a whole other level of difficulty compared to pumping out canned MIDI files. In any case, studio session musicians have a very different set of skills to pop headliners. Arrangers have different skills to composers. Producers have some of all of the above, but are there to bridge the differing creative pulls of the artist and the record company and manage the project. Label executives and A&R set goals and make high level editorial choices. When the public hears a finished track, it will have been worked on by tens or sometimes hundreds of people. Some of them - image and marketing consultants, PR people, social media managers - have no musical input of any kind, but still shape the subjective experience of the product. As you say, all of the above are paid because they bring a personality and a unique creative flavour to a project. For a culturally significant and/or successful project, each flavour has to be distinctive but also has to blend successfully. Music is always a composite product. Headline talent only gets you so far. The rest comes from creative tensions between individuals with different goals working in different domains. For most styles - including classical - the public wants a narrative/image/hook fantasy that locates the product in a credible cultural space. Generative music has almost none of this. Even if you create a perfectly polished pop single it will be culturally two dimensional because all of the surrounding context is missing. It will be flavourless - not just musically (most likely) but also culturally. In fact there are spaces where this either doesn't matter much or (rarely) doesn't matter at all. But they're small, specialised, and not particularly interesting to work in. Influential, interesting, highly paid projects demand strong and identifiable personal flavours. And they're far harder to synthesise than musically credible tracks/pieces.
bambax 5y ago
> Because people don't want their music that way. We made the classic mistake of thinking we knew what people wanted because _we_ thought it was cool This is a really great discussion. Can you elaborate a little about what you thought the public wanted, and what you found they actually want? (I make some stochastic / algorithic music myself and am very interested in this.)
- nonrandomstring 5y ago
  > Can you elaborate a little about what you thought the public wanted, and what you found they actually want? Sure I'll give it a shot, its over a decade ago now and I'm getting old at this. It was circa 2010 IIRC, and we had a a bunch of trailblazers, one Last.fm founder, a Second Life veteran, a top girl from Eve Online, me (procedural audio), and a vision for the delivery of "interactive" pop music - at the point of delivery, using loads of synthesis, but crucially using all the sensors to shape the music. So each "Reactive music" (coined by Michael Breidenbruecker) piece basically used the gyro, temperature, time of day, microphone, light levels and all that jazz... to create parameters for the music. We pioneered all kinds of stuff, like music that adapts to your running speed, driving, location (through GPS or other context awareness), sounds in the environment etcetera. I put a lot into the DSP and philosophy of "deferred form" as an artistic concept. What we thought people wanted was basically "co-creation". We moved legal mountains over licensing stems, derivative works, sharing of UGC by audiences... and working with artists/producers to "capture" their workflow, reduce it to a rule set and stuff it into a smartphone (which they really didn't like for the same good reasons Aboriginal Aussies have around photography). <Skip really complex psychological and philosophical stuff about identity, identification, art, attribution and social participation> What people _actually_ want is "definitive form" (deferred form only works in dietetic contexts like video games =- thus procedural audio is a thing, whereas reactive music is not - although Kent Jolly and the Spore guys almost proved that wrong, and games still seems the natural home of algorithmic music). Put it this way... everyone likes the idea of a "write your own story" game - games have infinite outcomes depending on what the user (player) does. Deferred form works there. But you can't make that work with film. I don't want a Star Wars where each time I watch it I get surprised: maybe Darth Vader and Luke _do_ rule galaxy as father and son, or Luke does a paternity test and it turns out "You're NOT my father!". I want a Star Wars where everything in the universe is as I remember and like it, so I can watch it over and get satisfaction. I want my psychological "noo-noo" to play with. That's how people listen to music, and ultimately it's what they want - even if they say they want interactive smartphone music. This is why market research is a waste of money unless you grok the culture. You can ask them. You can build it. And they still don't come. (BTW, I think this may be first and only post-mortem of RJDJ)

Rochus 5y ago

> In music it's quite usual though to separate the composition, arrangement and performance. Well, that's mostly what classic musicology does (and some Jazz musicologists inspired by classics). At the latest when you only have a lead sheet and the "composer" expects the musicians to add all the rest, the interpretation of the piece is also an essential part of the musical content/"composition". > the use paradigm doesn't appeal It makes sense to first make some proofs of concept (as e.g. done with MuseNet) concerning compositional/performance quality and then adapt the technology to specific use cases. From my humble point of view there was no convincing result in that respect before e.g. BachBot or MuseNet - maybe with the exception of Cope's work, but his algorithms actually require a human selection and optimization step for useful results - it's not fully algorithmic. > Why? Because people don't want their music that way. Having the right ideas is just a small fraction of success; most is pure luck (from my humble experience/observations). But solving an important/interesting problem is not identical with pleasing a mass market. Concerning music composition and performance there are still big unsolved representation problems, i.e. how to model this n-dimensional problem so that all relevant patterns along each subset of dimensions can be properly described and related to other patterns/features. MuseNet has an interesting approach, but it's mostly a spin-off from linguistics and from my point of view still far from what we should have. There is no satisfactory solution since the beginning of music theory and especially since the beginning of the AI era. All we have are only rudimentary projections, makeshifts to have anything at all. That's why a "a clunky old thing like MMA" will never pass a trained musicians "Turing test".

PaulDavisThe1st 5y ago
> That's why a "a clunky old thing like MMA" will never pass a trained musicians "Turing test". Most good musicians will have distinct assessments of other musicians. "Good" here is a metric that involves a combination of time playing, time performing, time spent collaborating and possibly some innate abilities, should such a thing exist. "I really enjoy playing with them, their sense of timing is so great" "They are really good technically but their development ideas just don't go where I go" etc. etc. There's no reason to expect that any computationally based accompaniment system will do any better than the human ones: if it's actually good, it will have a "personality" that won't work for some people; if it doesn't have a "personality", it almost certainly isn't good. There are many exceptionally talented musicians out there, measured by their technical abilities on their instruments. There are far, far fewer musicians that lots of other musicians want to play with.
- TheOtherHobbes 5y ago
  "Good" is context dependent, so any credible backing generation system would have to be aware of context - ideally while playing with others. Which is a whole other level of difficulty compared to pumping out canned MIDI files. In any case, studio session musicians have a very different set of skills to pop headliners. Arrangers have different skills to composers. Producers have some of all of the above, but are there to bridge the differing creative pulls of the artist and the record company and manage the project. Label executives and A&R set goals and make high level editorial choices. When the public hears a finished track, it will have been worked on by tens or sometimes hundreds of people. Some of them - image and marketing consultants, PR people, social media managers - have no musical input of any kind, but still shape the subjective experience of the product. As you say, all of the above are paid because they bring a personality and a unique creative flavour to a project. For a culturally significant and/or successful project, each flavour has to be distinctive but also has to blend successfully. Music is always a composite product. Headline talent only gets you so far. The rest comes from creative tensions between individuals with different goals working in different domains. For most styles - including classical - the public wants a narrative/image/hook fantasy that locates the product in a credible cultural space. Generative music has almost none of this. Even if you create a perfectly polished pop single it will be culturally two dimensional because all of the surrounding context is missing. It will be flavourless - not just musically (most likely) but also culturally. In fact there are spaces where this either doesn't matter much or (rarely) doesn't matter at all. But they're small, specialised, and not particularly interesting to work in. Influential, interesting, highly paid projects demand strong and identifiable personal flavours. And they're far harder to synthesise than musically credible tracks/pieces.

bambax 5y ago

> Because people don't want their music that way. We made the classic mistake of thinking we knew what people wanted because _we_ thought it was cool This is a really great discussion. Can you elaborate a little about what you thought the public wanted, and what you found they actually want? (I make some stochastic / algorithic music myself and am very interested in this.)

nonrandomstring 5y ago
> Can you elaborate a little about what you thought the public wanted, and what you found they actually want? Sure I'll give it a shot, its over a decade ago now and I'm getting old at this. It was circa 2010 IIRC, and we had a a bunch of trailblazers, one Last.fm founder, a Second Life veteran, a top girl from Eve Online, me (procedural audio), and a vision for the delivery of "interactive" pop music - at the point of delivery, using loads of synthesis, but crucially using all the sensors to shape the music. So each "Reactive music" (coined by Michael Breidenbruecker) piece basically used the gyro, temperature, time of day, microphone, light levels and all that jazz... to create parameters for the music. We pioneered all kinds of stuff, like music that adapts to your running speed, driving, location (through GPS or other context awareness), sounds in the environment etcetera. I put a lot into the DSP and philosophy of "deferred form" as an artistic concept. What we thought people wanted was basically "co-creation". We moved legal mountains over licensing stems, derivative works, sharing of UGC by audiences... and working with artists/producers to "capture" their workflow, reduce it to a rule set and stuff it into a smartphone (which they really didn't like for the same good reasons Aboriginal Aussies have around photography). <Skip really complex psychological and philosophical stuff about identity, identification, art, attribution and social participation> What people _actually_ want is "definitive form" (deferred form only works in dietetic contexts like video games =- thus procedural audio is a thing, whereas reactive music is not - although Kent Jolly and the Spore guys almost proved that wrong, and games still seems the natural home of algorithmic music). Put it this way... everyone likes the idea of a "write your own story" game - games have infinite outcomes depending on what the user (player) does. Deferred form works there. But you can't make that work with film. I don't want a Star Wars where each time I watch it I get surprised: maybe Darth Vader and Luke _do_ rule galaxy as father and son, or Luke does a paternity test and it turns out "You're NOT my father!". I want a Star Wars where everything in the universe is as I remember and like it, so I can watch it over and get satisfaction. I want my psychological "noo-noo" to play with. That's how people listen to music, and ultimately it's what they want - even if they say they want interactive smartphone music. This is why market research is a waste of money unless you grok the culture. You can ask them. You can build it. And they still don't come. (BTW, I think this may be first and only post-mortem of RJDJ)