9 ms·

Music Composition with Deep Learning: A Review

125 points by pramodbiligiri 5y ago

bravura 5y ago
I'm working on audio AI, both academic research and as founder of Spooky Labs. We don't have a webpage yet, but we do have clients. We are using deep learning to create rich new synthesizers that sound like they were designed by aliens, as well as novel vocal manipulation techniques. We've spoken to musicians and producers who are excited about new tools, new sounds, and assistants that automate boring parts of the workflow. But when the problem is framed as "music composition", it just leaves me scratching my head. Like, who's clamoring for that? I'm unaware in the history of music any automatically generated music that isn't seen as an oddity. Even if techniques improve, it's not a really sexy sell. People simply want to listen to music created by people, even if AI music were perfect. Only in commercial applications like stock music or jingles is AI composition in demand. I understand that you can move the goalposts and say: "This isn't about total AI composition, it's about co-composition!" But honestly, I think it's just framing the problem wrong to talk about composition, and its lead to some really strange solution-in-search-of-a-problem research agendas. People should thinking about it through the lens of: How do you use AI to create tools that musicians want?
- robbedpeter 5y ago
  As a musician, I'd love automated composition of human level music. Especially if you could navigate and blend and transform the space of styles, artists, and instruments. Co-composition has a place. The reason such tools have historically been viewed as oddities is because they were and are objectively bad. 3 year old prodigies can exceed even the best output of software. In recent years, the models have improved, but the problem space is huge and the algorithms aren't human level yet. When the tools reach human performance levels, people won't care if it's human or software. They'll only care if it's enjoyable. All creative endeavors are at risk of being automated by ai in the next few decades. Software will exceed the best of human creativity, and that's a good thing.
  - tgv 5y ago
    No, it isn't. It'll make people stop making music. Even if all you care about is one particular way to evaluate the end product, it means there will be no new inputs, resulting in the death of creativity. And once that's been reached, it's almost impossible to revive.
    - FranzFerdiNaN 5y ago
      People will most definitely not stop making music, if only because the physical act of playing an instrument can never be replaced by software.
      - tgv 5y ago
        Nobody is talking about playing music. And physical playing has been replaced by software quite a while ago. Not if you play it for your own fun, but almost nobody can distinguish a computer rendering from the real thing on a recording.
      - vixen99 5y ago
        Agreed. Moreover, if a genuinely sentient AI or MI as Minsky called it, comes to pass then it would then be fascinating to listen to it if it cared to compose music; in its absence musical composition is always the result of someone's human experience of being in this universe - necessarily as a human organism. And this is true even if the music is a research exploration of evocative (to a human of course) sound patterns.
      - imperistan 5y ago
        Very True! I play guitar but i’ll never be a Tommy Emmanuel. I don’t care, I play for the joy of it :)
      - TomSwirly 5y ago
        > if only because the physical act of playing an instrument can never be replaced by software. As an instrumentalist, I would love to believe this but more and more this does not appear to be the case. I have talked to many people of different ages who really don't see any difference between someone actually playing an instrument in front of you and listening to pre-recorded music. It's funny because it's in the context of me liking to play live music, and they never seem to realize they're saying, "What you do is meaningless." I think it's that they have no personal experience with playing an instrument, combined with the fact that the dominant forms of pop music now don't have much if any place for instrumentalists.
- dr_dshiv 5y ago
  Other than for research purposes (extending the capabilities of AI) — I 100% agree with your sentiment regarding the relative worthlessness of AI composition. As an aside, it is my understanding that there is no measure of something like consonance/dissonance in complex musical forms. There are models of dissonance in 2 and 3 note chords but even these have gaps. I'm suggesting that the research on "what sounds good to people" is surprisingly immature in contemporary science. This is surprising because that question played a major role in the history of science. For instance, many of the very first experiments conducted at the Royal Society (c1660s) investigated harmony — and arguably the first scientific experiment was designed to evaluate a mathematical model of music (Viz, the 5th century BC Pythagoreans demonstrating their integer ratio theory of harmony by casting bronze chimes at those ratios). So, I'm surprised that there isn't more interest today. That said, there was a recent breakthrough in the science of harmony: an integrated model of vertical and horizontal harmony. https://spj.sciencemag.org/journals/research/2019/2369041/#:~:text=Harmony%20studies%20the%20phenomenon%20of,physics%20and%20mathematics%20for%20centuries https://spj.sciencemag.org/journals/research/2019/2369041/#:....
  - TheOtherHobbes 5y ago
    There are measures, and they're not particularly complex. But consonance/dissonance and "sounds good" are not the point. Music is evocative and social, and it's the semantics of both that are under-researched. You won't get a grasp of either by throwing a corpus into a bucket and fishing things out of it with statistics.
    - dr_dshiv 5y ago
      > You won't get a grasp of either by throwing a corpus into a bucket and fishing things out of it with statistics. We can agree to disagree on that. Or, in any case, it's an empirical question. If we had a large corpus of music labeled by annotated "feels", I think we'd learn an immense amount about how music evokes feelings. (I'm not sure I feel comfortable with the term "semantics" applied to music) In the meantime, I've been building a corpus of the EEG response to music: https://ui.adsabs.harvard.edu/abs/2020arXiv200908793S/abstract https://ui.adsabs.harvard.edu/abs/2020arXiv200908793S/abstra... Regarding the measures of dissonance, I'm only familiar with things like measures of roughness and harmonic entropy. If you know more, I'd appreciate sharing.
- TheOtherHobbes 5y ago
  The same was said about synthesised music in the 40s and early 50s. It was mostly a lab curiosity. Now look at it. In some ways you're not wrong. But only in some ways.
  - deleted 5y ago
    [deleted]
- brigandish 5y ago
  > Like, who's clamoring for that? > even if AI music were perfect I remember Milli Vanilli (who I quite liked, to be honest), and a thousand manufactured bands since who at least make enough of an effort not to get caught out like that again. If that part of the industry (which seems like a large chunk) could get rid of the inconvenience of having to write/find good songs, I expect they would, and get back to exploiting pretty young things with not enough talent. In short, those who value money ahead of music. On the other hand, if it improved the rubbish in the pop chart then I might embrace it!
- dalbasal 5y ago
  >> People simply want to listen to music created by people, even if AI music were perfect. Only in commercial applications like stock music or jingles is AI composition in demand. I understand that you can move the goalposts With new tech, especially automation, moving goalposts isn't cheating. If successful, it almost always gets used in totally different ways than the human equivalent. A common pattern is imagining automation through "robots," like in the jetsons. An automated hotel has a robot desk clerk, robot maid, etc. IRL, an automated hotel may have no check in, rooms designed to be self cleaning and guests must make their own bed. Maybe it's not even a hotel anymore, but a room or pod here and there. Robotic maids act as a stand in, for things that are hard to imagine. >> AI to create tools that musicians want? So... this is a good compass for advanced stages, "productizing." Describe your goals in terms of what it can do for users. For earlier, research oriented stages, the more abstract definition is good. Composition works for that: Music composition (or generation) is the process of creating or writing a new piece of music, defined as a succession of pitches or rhythms, or both, in some definite patterns. How that is ultimately used/sold/productized is certainly up for debate, but that definition sends you on a path that's different from "tools for musicians."
- zozbot234 5y ago
  Current techniques are pretty successful at improvisation or 'noodling', not quite what most folks would call "proper" composition but a good inspiration for it. The paper is not very comprehensive, there's plenty of interesting stuff that it doesn't mention.
- StevenWaterman 5y ago
  I made https://stevenwaterman.uk/musetree https://stevenwaterman.uk/musetree which is a custom frontend for OpenAI's MuseNet. I've used that to generate music that I use on my Twitch streams, games, and other things. I'm not kidding myself that it's as good as human-written music, nor is it truly ai-written music (as there's a lot of human selection in the process), but people frequently comment on the music and enjoy it, without realising it's AI-written Some examples: https://github.com/stevenwaterman/Twitch-Overlay/raw/master/Client-Svelte/public/assets/intro/audio.wav https://github.com/stevenwaterman/Twitch-Overlay/raw/master/... https://github.com/stevenwaterman/NoTimeToStalk/raw/master/client/public/assets/audio/music/HouseStart.mp3 https://github.com/stevenwaterman/NoTimeToStalk/raw/master/c... * Edited to fix the link, my bad
  - stavros 5y ago
    That site is not working for me. Port 443 isn't open.
    - Y_Y 5y ago
      But port 80 is! http://stevenwaterman.uk/musetree/ http://stevenwaterman.uk/musetree/
- stavros 5y ago
  I don't know, I'd pay for something that gave me music I loved, 100% of the time (or, hell, even 2% of the time). Beats sifting through the infinite trove of music just so I can stumble upon something I like by chance. My success rate there is more like 0.1%.
- TaupeRanger 5y ago
  No one is clamoring for it because it isn't good enough yet, same reason no one is asking for AI designed houses, or business logos, or food. But that doesn't mean it can't get there eventually. It's also a very interesting scientific question, wrapped up with big questions about the nature of creativity and the capabilities of machines.
- notahacker 5y ago
  I think apart from the layman's resistance to the idea of nonhumans being creative, the other problem is that the basic theories of composition musicians use are pretty well understood and based on applying some surprisingly simple maths to some instruments whose timbres are very complex (and "what works" is a moving target bound to cultural familiarity with instruments, intonation, musical phrasing and comfort with dissonance; not an AlphaGoZero situation where the ML process can expose the wealth of existing human theory is objectively inferior at achieving a win condition.) Sure, a Deep Learning process with a suitably curated dataset can rediscover the basic principles of Western music theory... but why? Humans understood these things when creating the music that makes up the corpus, and non-AI software can already turn those patterns into musical structures which can be played by accurate synthesis, which only lacks a bit of nuance. The human musician looking for AI-powered shortcuts doesn't want a lot of computation thrown at learning how to approximate a 12 bar blues or arpeggiate chord tones, they want it thrown at a post-production process (or better still, a pedal) that makes their mechanical playing of simple licks approximate BB King or an autotune that makes their average voice in a lower pitch sound like Whitney Houston.
  - OneEyedRobot 5y ago
    >I think apart from the layman's resistance to the idea of nonhumans being creative, What I think is funny is the resistance to the idea that a lot of what people do isn't all that special. It all makes me think of the evolution of supermarket cashiers. It used to be a fairly skilled profession, now people are just used because of their abilities in object manipulation.
- OneEyedRobot 5y ago
  >who's clamoring for that? One use I can see is assist software for someone churning out commercial music. Maybe a standalone music generator (composition plus synth) would be useful for content generators avoiding copyright. As both copyright violation policing gets easier and heavier and programatically generated music becomes easier I have to wonder how those two things will collide.
- grafs50 5y ago
  I can imagine a future where AI can come up with new stylistic concepts, maybe even "predicting" how some music genre will evolves. which is certainly something I'd be interested in seeing explored. Imagine an AI that was able to come up with something remotely close to rock music after being fed 1930s-1940s blues music. It'd be pretty cool to see what it came up with after being fed music from the 2000-2010s.
- zzbzq 5y ago
  Well I agree the term "music composition" is not marketable, I disagree on all other points. Actually I think the only thing pulling for AI is the idea that people would prefer to listen to something generated by a computer. People no longer view human sources as trustworthy, they no longer experience natural things as authentic. They want computer-certified facts, machine-performed actions. You don't even need an AI to make the kind of music people like. You just need to dumb down the music--which is what has happened gradually since the gilded age. Now the charts are topped with monotone "robot" melodies and little to no chord structure, synths using standard basic wave forms, all rhythms quantized to a grid and all voices auto-tuned. You don't need an AI to generate this type of music by machine; ping pong balls in a tumbler would suffice. The role of AI, then, is not that you need it to make the music, but you need an AI to claim authenticity. In the minds of today's people, the computer-made music must be better, must be smarter, than human-made music.
- awolnikowski 5y ago
  I disagree that there isn't already generative music that is enjoyed just for its aesthetic qualities. I'm a huge fan of Brian Eno's work, which is highly generative (and he's even published apps that will simply generate ambient music for you 24/7). "Since I have always preferred making plans to executing them, I have gravitated towards situations and systems that, once set into operation, could create music with little or no intervention on my part. That is to say, I tend towards the roles of planner and programmer, and then become an audience to the results" - Brian Eno I think generative music is particularly effective as ambient background or as part of a larger art installation. Also, if you're working on developing new synthesizers I'd highly recommend you get to know Eno's work.
- randcraw 5y ago
  As a music-theory-unsophisticate, I foresee a lot of potential for deep learning in creating musical arrangements for existing compositions using all kinds of available instruments. How might a Beethoven symphony be arranged for a typical 4 part rock band? What if you added a piano? What other instruments might add further value, esp. toward certain ends, like when learning to play an instrument? Could a deep net learn to accompany adaptively, as only a capable human can today? Could it offer instructional suggestions for improvement, especially subtleties in timing or dynamics? I also see value in transferring musical style for a piece from one genre to another (let's say, from jazz to rock & roll). Or from one set of instruments (or voices) to others. If done well, that could provide a lot of potentially entertaining interpretations or mashups, or at worst some novel medleys. Sinatra raps. Nat King Cole does bluegrass. Because examples of uses like these aren't likely to be plentiful, I suspect these won't be so much supervised learning tasks as reinforcement learning (RL) tasks. The trick then will be to somehow construct useful (and insightful) reward functions for each new use -- basically aesthetic meta-learning for RL. If RL reward functions can learn to write winning advertising jingles, it's hard to imagine what creative doors they won't open, eventually.
- GDC7 5y ago
  > People simply want to listen to music created by people People also want a chunk of that Bob Dylan/Beatles fame without necessarily having the talent or the luck to compose songs like they did. Let the AI do the work and then only be there for the fun part.
- didibus 5y ago
  > People simply want to listen to music created by people I don't think people care who composed the music, I believe people want to see music played and sang by people. Going to a concert where you're just looking at a computer generating sound won't inspire the croud, but if the band plays something composed by an AI, who cares? Even nowadays music has multiple collaborators on a composition, and I almost never pay attention to who they all are.
- Jeff_Brown 5y ago
  The band Yacht used AI to write an album called Chain Tripping that's really good IMO. They transcribed their old music to MIDI, fed it to an AI, had it generate short snippets of new MIDI, and then cut and paste those bits to get a score, which they performed. I suspect the drums and bass are human-written, as they seem more coherent and "whole song-aware" than the other parts, but I have no way to confirm that. https://www.kcrw.com/news/shows/press-play-with-madeleine-brand/using-a-i-to-make-a-music-album/how-yacht-used-a-i-to-make-their-album-chain-tripping https://www.kcrw.com/news/shows/press-play-with-madeleine-br...
- analog31 5y ago
  I believe one of the problems is that we don't know what the goalposts are. People experience music differently, ranging from barely tolerating it, to pursuing it as an academic study, to making massive sacrifices in order to be immersed in it. The conscious lifestyle choice of many full time musicians is bewildering to many techies. This suggests we don't know the "customer" or the "market" at all, or at the very least, that they are fragmented.
- Jeff_Brown 5y ago
  It might be centuries before AI is writing good novels, but music seems a much easier nut to crack. Hell I've seen people sit down at a drum machine and make something enjoyable for minutes in literally seconds. Music doesn't have to be complicated, let alone deeply meaningful, to sound good. AI already makes cool art -- e.g. the Google Deep Dream videos.
inferense 5y ago
ever since I saw the VAE learn to synthesize simple tunes followed by listening to the Hello World by Skygge, I contemplated about the inflection point of the music industry [1][2]. At first some of the music would be a close collaboration of a machine and an artist, but at some point, I wonder if we’d be primarily listening to tunes generated to our specific taste similarly to spotify’s discover weekly. Though it would be all be much closer to what we want to listen to (exploit & explore) at a given time. [1]https://www.youtube.com/watch?v=G5JT16flZwM https://www.youtube.com/watch?v=G5JT16flZwM [2]https://blog.nameshield.com/blog/2018/01/23/hello-world-musical-creation-artificial-intelligence/ https://blog.nameshield.com/blog/2018/01/23/hello-world-musi...
sharikous 5y ago
I wonder why they don't mention the problem of the audio quality of the output. As far as I know the best models work on magnitude spectrograms and have issues with recreating the phase information. Sub-par algorithms like Griffin-Lim are used instead
- thomashop 5y ago
  The review is about composition in the score domain, not synthesis in the audio domain. Check Differential Digital Signal Processing for state-of-the-art data driven synthesis in th audio domain.
  - TomSwirly 5y ago
    It's a very poor "score" domain with no timbral information at all. See my comment elsewhere: https://news.ycombinator.com/item?id=28355582 https://news.ycombinator.com/item?id=28355582
    - thomashop 5y ago
      I was talking about musical scores. They shouldn't have any timbral information.
DrNuke 5y ago
Automated music (art) generation + NFTs = new printing money aka new trading bubble?
PennRobotics 5y ago
The paper begins with David Cope. He evidently has a Youtube channel with algorithmic music. https://www.youtube.com/user/davidhcope/videos https://www.youtube.com/user/davidhcope/videos
- JoeDaDude 5y ago
  David Cope has written several books and journal publications about his automated composition methods, some examples below: https://www.amazon.com/gp/product/0895792567/ref=dbs_a_def_rwt_bibl_vppi_i97 https://www.amazon.com/gp/product/0895792567/ref=dbs_a_def_r... https://ieeexplore.ieee.org/document/6787993 https://ieeexplore.ieee.org/document/6787993
geekamongus 5y ago
It will only be a matter of time until the AI learns that what sounds pleasing to most humans is often some form of, or extract from, Pachelbel's Canon in D.
- bruce343434 5y ago
  It is sickening how often I hear it
shannifin 5y ago
Decent review! I've been fascinated by this topic for a while. I think the real magic is of course in the specifications of the details, but I do think it's innevitable that AI-generated (or assisted) music will come to dominate the field as it democratizes what is currently a more specialized skill set. I've always found the resistance to the idea of AI-generated music a bit odd, but I think it stems from a more philosophical idea about where beauty comes from. For example, it's tempting to imagine that the beauty of a Beethoven symphony comes from Beethoven himself, thus his music facilitates an intense personal emotional connection across time and space. The idea of AI music challenges this notion a bit; if AI "composed" something beautiful, and its decisions were not founded on emotions, where is the beauty coming from? Of course, I'd say the beauty is in the natural phenomenon of music itself; the beauty is in the human brain's ability to have a sense of "music" at all. In this way, a connection to Beethoven through his music is a shared recognition of the beauty of certain natural musical phenomenon. The beauty did not come from Beethoven, rather it was "captured" and shared by Beethoven. As far as the business side goes, I've definitely found that there's interest in at least AI-assisted music composition, both from a melody-generating app I used to sell some years ago (limited as it was), and just collecting emails for a new web app I'm working on at https://www.tunesage.com/ https://www.tunesage.com/ ... I know there are already other services out there too for AI music, and I expect the space will continue to grow. I think it's gonna be awesome. :)
- vincnetas 5y ago
  Im afraid that AI will overwhelm us with perfect amazing music (and other creations) that we will come to a familiar situation that Alan Watts describes : "And you would, naturally as you began on this adventure of dreams, you would fulfill all your wishes. You would have every kind of pleasure you could conceive." And then in this world overwhelmed with perfection we will fall back to that untraceable aspect of human creation that keeps us in awe. Of course we will be much more comfortable in that advanced world in a sense of fulfilled basic needs, but self actualisation (from the pyramid of needs) i guess will never be achieved by AI :)
- yawboakye 5y ago
  The beauty in human creation doesn't rest entirely in the artefact. A large part of it rests in the knowledge (of the beneficiary) that another skilled human has dedicated time, effort, and intelligence to uniquely produce what is in front of them. It doesn't stop at music. You see it everywhere: fashion (hand-made), food (famous chefs), music (live performances), coffee (moka pots), sex (orgasm stimulated by other humans), etc. It isn't a pursuance of the best (or perfection), but of what has more soul in it. And unfortunately AI music lacks any soul.
bjourne 5y ago
It all comes down to generating tokens. The dominant paradigm is to compose the music by generating the next token based on the last few seen tokens, then take that and generate the next token, and so on. No one has been able to significantly improve upon that. All development in the last ten years or so has been about creating ever larger models with more data so that emitted sequences will stay coherent for longer. The problem with this paradigm is that it works amazingly well for short sequences, but often fails to stay coherent for longer sequences. So composing interesting ten seconds long instrumental music is a solved problem - composing full songs three-to-five minutes long is far beyond the reach of current state-of-the-art. Text-generating models suffer from the same problem. E.g. GPT-3 will generate two to three-hundred tokens or so and then it will begin to spout nonsense. That it still works so well is because it's trained with a massive corpus and because its tokens are richer. It operates on the world level, while most music composition models operate on the note level which are more analogous to characters than words. If someone were to be able to figure out how to represent multi-instrumental compositions using "music vectors" (analogous to word vectors) that would probably lead to a major breakthrough.
- pishpash 5y ago
  Probably these would be harmonic units. Unfortunately notes aren't just characters in this. There's also voice leading, and so harmony "words" run into each other and can't be mutually exclusively tokenized. The better analogy might be poetry, and I'm not sure if there are good poetry generators, either.
- OneEyedRobot 5y ago
  By 'token' do you mean a note, a chord, general structure (verse, bridge, etc.), a well known phrase? Music is so full of cliche I don't doubt that someone will write software (probably already has) to assist a composer, generate drum parts or harmony, etc. I like the idea of a Rhythm changes generator. With enough monkeys the Flintstone Theme eventually pops out.
  - snthd 5y ago
    >(probably already has) https://github.com/Impro-Visor/Impro-Visor https://github.com/Impro-Visor/Impro-Visor
streamofdigits 5y ago
A key question imho is this: will algorithmic music composition ever produce anything else than more or less sophisticated musac, effectively recycling a particular musical corpus of a particular music tradition? Producing a memorable, emotionally moving piece is not something that is generally reducible to prescription. If the DL based pattern matching is sufficiently intelligent and produces something worthwhile, will it actually feel different to an existing piece? Music perception and appreciation is a deeply human feature (is there a "purpose" to it or is it actually a piece of redundant brain code? https://brianjump.net/2020/11/02/why-does-music-exist/ https://brianjump.net/2020/11/02/why-does-music-exist/) that has both a biological and cultural basis. There is a feedback loop of contemporary musical sounds a young brain is exposed to and the perception machinery and associated innate pleasurable responses. Think of the countless cultural innovations that underpin human music: tunings, scales, rhythms, genres and the deep link to voice and song. None of those has any (ex-ante) algorithmic feel (to the chagrin of Pythagoreans and other ex-post numerologists). Coming up with music feels literally like plucking beautiful sounds from thin air. In any case its a fascinating research domain...
- PaulDavisThe1st 5y ago
  > A key question imho is this: will algorithmic music composition ever produce anything else than more or less sophisticated musac, effectively recycling a particular musical corpus of a particular music tradition? A related question is: how many human composers manage do this? It seems that we are fairly OK with a large body of composers "effectively recycling a particular musical corpus".
  - streamofdigits 5y ago
    There is indeed a lot of "filler" type production and even the most talented creators are far from always producing unique pieces that touch audiences (let alone endure). But this misses the point: some creators do occasionally hit the magic sweet spot and invent something that makes the rest worthwhile. Heck, some creators invent new genres. A DL type algorithm will not produce a new "classic" except by a "thousand monkeys hitting keyboards" type accident... at least not until we dig far deeper into what makes some music stand-out. And an algorithm as conventionally defined is unlikely to ever produce a new musical style. The true opportunity (as always with algorithms) is the augmentation of creative talent.
YeGoblynQueenne 5y ago
This is an interesting and informative article but to be a bit meta I'm concerned when I see articles like this on HN because they don't usually do a great job of summarising the work that has been done in a certain area before the advent of deep neural nets. This is important because very often, especially when it comes to generative art, the standard approaches used before deep neural nets could do thins that modern deep neural nets cannot do, in particular when it comes to structured generation. For example, alorithmic music is the subject area of generating music with algorithmic approaches, not necessarily using a computer. The wikipedia page seems to be a bit poor in detail but it lists a number of different approaches most of which are not machine larning approaches: https://en.wikipedia.org/wiki/Algorithmic_composition https://en.wikipedia.org/wiki/Algorithmic_composition I'm by no means an expert but that's the point. When a non-expert reads an article like the one above, I fear they may get an impression that neural nets are the first approach ever to generate music, or that they are the best approach ever to generate music, or anyway some kind of misunderstanding that is natural to draw from incomplete information. The thing to try and keep in mind is that computer scientists, and other scientists and creative people, had been able to do amazing things with the tools they had in their disposal long before the advent of deep neural nets. And that there are many such tools that are not deep neural nets. Somehow these amazing things flew under the radar of technies - until deep neural nets came along and suddendly everyone is amazed that "wow, neural nets can do X!". Well, what else can do X? That's something worth trying to find out.
- platz 5y ago
  are you referring to the historic pre-classical period prior to 0 BIM (Before ImageNet)
  - YeGoblynQueenne 5y ago
    I'm talking, for example, about the difference between images of plants generated by L-systems [1] and images of plants generated by GANs [2]. _______________ [1] https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/L-system [2] https://www.easyzoom.com/imageaccess/2128f27845ed4921b3143004dc550c3e https://www.easyzoom.com/imageaccess/2128f27845ed4921b314300...
TomSwirly 5y ago
The article starts really badly with "Music is generally defined as a succession of pitches or rhythms, or both, in some definite patterns." The big three in music are pitch, rhythm and TIMBRE, guys, TIMBRE (the "sound" or "tone" of the piece). Why is Beethoven's Ode To Joy somewhat annoying on the recorder, and can drive a room of hardened Germans to tears when played by a top-notch symphony orchestra? Timbre! In the construction of classical musical pieces, this is controlled with orchestration. It's why a Hans Zimmermann score sounds punchy or etheriel or whatever is called for by the script. I basically stopped there, because, well, if you don't care how your music _sounds_ it isn't going to sound very good.
- PaulDavisThe1st 5y ago
  > The big three in music are pitch, rhythm and TIMBRE, The big three are normally described as melody, harmony and rhythm, and yes, it should be four because timbre is clearly central to the construction and experience of music.
- shoemakersteve 5y ago
  I personally disagree. Timbre will give the music a certain quality, and while it's no doubt a big part of the experience, I think the dynamics of piece will be much more central to the emotional response. If a song has a sax solo that really touches your heart, and you replaced it with a trombone playing the exact same solo instead, the timbre would be completely different, but the emotions that it would evoke would likely be almost identical. The timbre would mainly affect how well one or the other might blend with the rest of the orchestration. If someone played Ode To Joy on a recorder with proper dynamics, I can guarantee they could make it sound good. All my personal opinion of course.
qwerty456127 5y ago
I am eager to be able to choose a classical music piece (e.g. Bach Cello Suite #1 in G) and ask an AI to generate an infinite sequence very close to its style but variating slightly.
adamnemecek 5y ago
I’ve been working on an IDE for music composition http://ngrid.io http://ngrid.io. Launching soon.
gwern 5y ago
Not a single mention of Jukebox.
spywaregorilla 5y ago
I'm not particularly familiar with music creation, but it always seemed to me neural nets would be better at handling music production more than composition. That is, changing the sound to be, say, a trent reznor style output given the original. Does that make any sense?
- thebricksta 5y ago
  As someone with a fair amount of experience in both domains, I'd agree with this. I see a lot more utility in sculpting sound and helping streamline the production side of things than the uses of NNs on the composition side. There already are tools for automatically composing melodies and harmonies, and some of them are actually quite good, but they're hardly used.
thebricksta 5y ago
The best definition I've seen for the success of a piece of music is this: "What emotion is the artist trying to convey and how well does it convey it?" Throughout the composing, arranging, recording, mixing, and mastering process, there are thousands of choices to be made, and the correctness of each choice is entirely linked back to that goal: Does the choice help to convey the emotion, or does it detract from it? To that end, there is no correct choice, no correct or optimal harmony, no correct note, no correct rhythm, no correct timbre. It's all contextual in relation to conveying the desired emotion. I'm really not sure how you could ever train a NN to make choices in that regard without first trying to teach them how to understand the impacts of their choices on the emotions conveyed. At best, you may be able to train a NN to reproduce emotionally-void works in a particular style, and perhaps assign some emotion through the timbres selected (ambient music comes to mind here). Still, this isn't much of an achievement. You could easily codify the rules taught in Music 101 about harmonization and melody composition to a computer and have it spit out bland but pleasant excerpts, no deep learning required.

I'm working on audio AI, both academic research and as founder of Spooky Labs. We don't have a webpage yet, but we do have clients. We are using deep learning to create rich new synthesizers that sound like they were designed by aliens, as well as novel vocal manipulation techniques. We've spoken to musicians and producers who are excited about new tools, new sounds, and assistants that automate boring parts of the workflow. But when the problem is framed as "music composition", it just leaves me scratching my head. Like, who's clamoring for that? I'm unaware in the history of music any automatically generated music that isn't seen as an oddity. Even if techniques improve, it's not a really sexy sell. People simply want to listen to music created by people, even if AI music were perfect. Only in commercial applications like stock music or jingles is AI composition in demand. I understand that you can move the goalposts and say: "This isn't about total AI composition, it's about co-composition!" But honestly, I think it's just framing the problem wrong to talk about composition, and its lead to some really strange solution-in-search-of-a-problem research agendas. People should thinking about it through the lens of: How do you use AI to create tools that musicians want?

robbedpeter 5y ago
As a musician, I'd love automated composition of human level music. Especially if you could navigate and blend and transform the space of styles, artists, and instruments. Co-composition has a place. The reason such tools have historically been viewed as oddities is because they were and are objectively bad. 3 year old prodigies can exceed even the best output of software. In recent years, the models have improved, but the problem space is huge and the algorithms aren't human level yet. When the tools reach human performance levels, people won't care if it's human or software. They'll only care if it's enjoyable. All creative endeavors are at risk of being automated by ai in the next few decades. Software will exceed the best of human creativity, and that's a good thing.
- tgv 5y ago
  No, it isn't. It'll make people stop making music. Even if all you care about is one particular way to evaluate the end product, it means there will be no new inputs, resulting in the death of creativity. And once that's been reached, it's almost impossible to revive.
  - FranzFerdiNaN 5y ago
    People will most definitely not stop making music, if only because the physical act of playing an instrument can never be replaced by software.
    - tgv 5y ago
      Nobody is talking about playing music. And physical playing has been replaced by software quite a while ago. Not if you play it for your own fun, but almost nobody can distinguish a computer rendering from the real thing on a recording.
    - vixen99 5y ago
      Agreed. Moreover, if a genuinely sentient AI or MI as Minsky called it, comes to pass then it would then be fascinating to listen to it if it cared to compose music; in its absence musical composition is always the result of someone's human experience of being in this universe - necessarily as a human organism. And this is true even if the music is a research exploration of evocative (to a human of course) sound patterns.
    - imperistan 5y ago
      Very True! I play guitar but i’ll never be a Tommy Emmanuel. I don’t care, I play for the joy of it :)
    - TomSwirly 5y ago
      > if only because the physical act of playing an instrument can never be replaced by software. As an instrumentalist, I would love to believe this but more and more this does not appear to be the case. I have talked to many people of different ages who really don't see any difference between someone actually playing an instrument in front of you and listening to pre-recorded music. It's funny because it's in the context of me liking to play live music, and they never seem to realize they're saying, "What you do is meaningless." I think it's that they have no personal experience with playing an instrument, combined with the fact that the dominant forms of pop music now don't have much if any place for instrumentalists.
dr_dshiv 5y ago
Other than for research purposes (extending the capabilities of AI) — I 100% agree with your sentiment regarding the relative worthlessness of AI composition. As an aside, it is my understanding that there is no measure of something like consonance/dissonance in complex musical forms. There are models of dissonance in 2 and 3 note chords but even these have gaps. I'm suggesting that the research on "what sounds good to people" is surprisingly immature in contemporary science. This is surprising because that question played a major role in the history of science. For instance, many of the very first experiments conducted at the Royal Society (c1660s) investigated harmony — and arguably the first scientific experiment was designed to evaluate a mathematical model of music (Viz, the 5th century BC Pythagoreans demonstrating their integer ratio theory of harmony by casting bronze chimes at those ratios). So, I'm surprised that there isn't more interest today. That said, there was a recent breakthrough in the science of harmony: an integrated model of vertical and horizontal harmony. https://spj.sciencemag.org/journals/research/2019/2369041/#:~:text=Harmony%20studies%20the%20phenomenon%20of,physics%20and%20mathematics%20for%20centuries https://spj.sciencemag.org/journals/research/2019/2369041/#:....
- TheOtherHobbes 5y ago
  There are measures, and they're not particularly complex. But consonance/dissonance and "sounds good" are not the point. Music is evocative and social, and it's the semantics of both that are under-researched. You won't get a grasp of either by throwing a corpus into a bucket and fishing things out of it with statistics.
  - dr_dshiv 5y ago
    > You won't get a grasp of either by throwing a corpus into a bucket and fishing things out of it with statistics. We can agree to disagree on that. Or, in any case, it's an empirical question. If we had a large corpus of music labeled by annotated "feels", I think we'd learn an immense amount about how music evokes feelings. (I'm not sure I feel comfortable with the term "semantics" applied to music) In the meantime, I've been building a corpus of the EEG response to music: https://ui.adsabs.harvard.edu/abs/2020arXiv200908793S/abstract https://ui.adsabs.harvard.edu/abs/2020arXiv200908793S/abstra... Regarding the measures of dissonance, I'm only familiar with things like measures of roughness and harmonic entropy. If you know more, I'd appreciate sharing.
TheOtherHobbes 5y ago
The same was said about synthesised music in the 40s and early 50s. It was mostly a lab curiosity. Now look at it. In some ways you're not wrong. But only in some ways.
- deleted 5y ago
  [deleted]
brigandish 5y ago
> Like, who's clamoring for that? > even if AI music were perfect I remember Milli Vanilli (who I quite liked, to be honest), and a thousand manufactured bands since who at least make enough of an effort not to get caught out like that again. If that part of the industry (which seems like a large chunk) could get rid of the inconvenience of having to write/find good songs, I expect they would, and get back to exploiting pretty young things with not enough talent. In short, those who value money ahead of music. On the other hand, if it improved the rubbish in the pop chart then I might embrace it!
dalbasal 5y ago
>> People simply want to listen to music created by people, even if AI music were perfect. Only in commercial applications like stock music or jingles is AI composition in demand. I understand that you can move the goalposts With new tech, especially automation, moving goalposts isn't cheating. If successful, it almost always gets used in totally different ways than the human equivalent. A common pattern is imagining automation through "robots," like in the jetsons. An automated hotel has a robot desk clerk, robot maid, etc. IRL, an automated hotel may have no check in, rooms designed to be self cleaning and guests must make their own bed. Maybe it's not even a hotel anymore, but a room or pod here and there. Robotic maids act as a stand in, for things that are hard to imagine. >> AI to create tools that musicians want? So... this is a good compass for advanced stages, "productizing." Describe your goals in terms of what it can do for users. For earlier, research oriented stages, the more abstract definition is good. Composition works for that: Music composition (or generation) is the process of creating or writing a new piece of music, defined as a succession of pitches or rhythms, or both, in some definite patterns. How that is ultimately used/sold/productized is certainly up for debate, but that definition sends you on a path that's different from "tools for musicians."
zozbot234 5y ago
Current techniques are pretty successful at improvisation or 'noodling', not quite what most folks would call "proper" composition but a good inspiration for it. The paper is not very comprehensive, there's plenty of interesting stuff that it doesn't mention.
StevenWaterman 5y ago
I made https://stevenwaterman.uk/musetree https://stevenwaterman.uk/musetree which is a custom frontend for OpenAI's MuseNet. I've used that to generate music that I use on my Twitch streams, games, and other things. I'm not kidding myself that it's as good as human-written music, nor is it truly ai-written music (as there's a lot of human selection in the process), but people frequently comment on the music and enjoy it, without realising it's AI-written Some examples: https://github.com/stevenwaterman/Twitch-Overlay/raw/master/Client-Svelte/public/assets/intro/audio.wav https://github.com/stevenwaterman/Twitch-Overlay/raw/master/... https://github.com/stevenwaterman/NoTimeToStalk/raw/master/client/public/assets/audio/music/HouseStart.mp3 https://github.com/stevenwaterman/NoTimeToStalk/raw/master/c... * Edited to fix the link, my bad
- stavros 5y ago
  That site is not working for me. Port 443 isn't open.
  - Y_Y 5y ago
    But port 80 is! http://stevenwaterman.uk/musetree/ http://stevenwaterman.uk/musetree/
stavros 5y ago
I don't know, I'd pay for something that gave me music I loved, 100% of the time (or, hell, even 2% of the time). Beats sifting through the infinite trove of music just so I can stumble upon something I like by chance. My success rate there is more like 0.1%.
TaupeRanger 5y ago
No one is clamoring for it because it isn't good enough yet, same reason no one is asking for AI designed houses, or business logos, or food. But that doesn't mean it can't get there eventually. It's also a very interesting scientific question, wrapped up with big questions about the nature of creativity and the capabilities of machines.
notahacker 5y ago
I think apart from the layman's resistance to the idea of nonhumans being creative, the other problem is that the basic theories of composition musicians use are pretty well understood and based on applying some surprisingly simple maths to some instruments whose timbres are very complex (and "what works" is a moving target bound to cultural familiarity with instruments, intonation, musical phrasing and comfort with dissonance; not an AlphaGoZero situation where the ML process can expose the wealth of existing human theory is objectively inferior at achieving a win condition.) Sure, a Deep Learning process with a suitably curated dataset can rediscover the basic principles of Western music theory... but why? Humans understood these things when creating the music that makes up the corpus, and non-AI software can already turn those patterns into musical structures which can be played by accurate synthesis, which only lacks a bit of nuance. The human musician looking for AI-powered shortcuts doesn't want a lot of computation thrown at learning how to approximate a 12 bar blues or arpeggiate chord tones, they want it thrown at a post-production process (or better still, a pedal) that makes their mechanical playing of simple licks approximate BB King or an autotune that makes their average voice in a lower pitch sound like Whitney Houston.
- OneEyedRobot 5y ago
  >I think apart from the layman's resistance to the idea of nonhumans being creative, What I think is funny is the resistance to the idea that a lot of what people do isn't all that special. It all makes me think of the evolution of supermarket cashiers. It used to be a fairly skilled profession, now people are just used because of their abilities in object manipulation.
OneEyedRobot 5y ago
>who's clamoring for that? One use I can see is assist software for someone churning out commercial music. Maybe a standalone music generator (composition plus synth) would be useful for content generators avoiding copyright. As both copyright violation policing gets easier and heavier and programatically generated music becomes easier I have to wonder how those two things will collide.
grafs50 5y ago
I can imagine a future where AI can come up with new stylistic concepts, maybe even "predicting" how some music genre will evolves. which is certainly something I'd be interested in seeing explored. Imagine an AI that was able to come up with something remotely close to rock music after being fed 1930s-1940s blues music. It'd be pretty cool to see what it came up with after being fed music from the 2000-2010s.
zzbzq 5y ago
Well I agree the term "music composition" is not marketable, I disagree on all other points. Actually I think the only thing pulling for AI is the idea that people would prefer to listen to something generated by a computer. People no longer view human sources as trustworthy, they no longer experience natural things as authentic. They want computer-certified facts, machine-performed actions. You don't even need an AI to make the kind of music people like. You just need to dumb down the music--which is what has happened gradually since the gilded age. Now the charts are topped with monotone "robot" melodies and little to no chord structure, synths using standard basic wave forms, all rhythms quantized to a grid and all voices auto-tuned. You don't need an AI to generate this type of music by machine; ping pong balls in a tumbler would suffice. The role of AI, then, is not that you need it to make the music, but you need an AI to claim authenticity. In the minds of today's people, the computer-made music must be better, must be smarter, than human-made music.
awolnikowski 5y ago
I disagree that there isn't already generative music that is enjoyed just for its aesthetic qualities. I'm a huge fan of Brian Eno's work, which is highly generative (and he's even published apps that will simply generate ambient music for you 24/7). "Since I have always preferred making plans to executing them, I have gravitated towards situations and systems that, once set into operation, could create music with little or no intervention on my part. That is to say, I tend towards the roles of planner and programmer, and then become an audience to the results" - Brian Eno I think generative music is particularly effective as ambient background or as part of a larger art installation. Also, if you're working on developing new synthesizers I'd highly recommend you get to know Eno's work.
randcraw 5y ago
As a music-theory-unsophisticate, I foresee a lot of potential for deep learning in creating musical arrangements for existing compositions using all kinds of available instruments. How might a Beethoven symphony be arranged for a typical 4 part rock band? What if you added a piano? What other instruments might add further value, esp. toward certain ends, like when learning to play an instrument? Could a deep net learn to accompany adaptively, as only a capable human can today? Could it offer instructional suggestions for improvement, especially subtleties in timing or dynamics? I also see value in transferring musical style for a piece from one genre to another (let's say, from jazz to rock & roll). Or from one set of instruments (or voices) to others. If done well, that could provide a lot of potentially entertaining interpretations or mashups, or at worst some novel medleys. Sinatra raps. Nat King Cole does bluegrass. Because examples of uses like these aren't likely to be plentiful, I suspect these won't be so much supervised learning tasks as reinforcement learning (RL) tasks. The trick then will be to somehow construct useful (and insightful) reward functions for each new use -- basically aesthetic meta-learning for RL. If RL reward functions can learn to write winning advertising jingles, it's hard to imagine what creative doors they won't open, eventually.
GDC7 5y ago
> People simply want to listen to music created by people People also want a chunk of that Bob Dylan/Beatles fame without necessarily having the talent or the luck to compose songs like they did. Let the AI do the work and then only be there for the fun part.
didibus 5y ago
> People simply want to listen to music created by people I don't think people care who composed the music, I believe people want to see music played and sang by people. Going to a concert where you're just looking at a computer generating sound won't inspire the croud, but if the band plays something composed by an AI, who cares? Even nowadays music has multiple collaborators on a composition, and I almost never pay attention to who they all are.
Jeff_Brown 5y ago
The band Yacht used AI to write an album called Chain Tripping that's really good IMO. They transcribed their old music to MIDI, fed it to an AI, had it generate short snippets of new MIDI, and then cut and paste those bits to get a score, which they performed. I suspect the drums and bass are human-written, as they seem more coherent and "whole song-aware" than the other parts, but I have no way to confirm that. https://www.kcrw.com/news/shows/press-play-with-madeleine-brand/using-a-i-to-make-a-music-album/how-yacht-used-a-i-to-make-their-album-chain-tripping https://www.kcrw.com/news/shows/press-play-with-madeleine-br...
analog31 5y ago
I believe one of the problems is that we don't know what the goalposts are. People experience music differently, ranging from barely tolerating it, to pursuing it as an academic study, to making massive sacrifices in order to be immersed in it. The conscious lifestyle choice of many full time musicians is bewildering to many techies. This suggests we don't know the "customer" or the "market" at all, or at the very least, that they are fragmented.
Jeff_Brown 5y ago
It might be centuries before AI is writing good novels, but music seems a much easier nut to crack. Hell I've seen people sit down at a drum machine and make something enjoyable for minutes in literally seconds. Music doesn't have to be complicated, let alone deeply meaningful, to sound good. AI already makes cool art -- e.g. the Google Deep Dream videos.

inferense 5y ago

ever since I saw the VAE learn to synthesize simple tunes followed by listening to the Hello World by Skygge, I contemplated about the inflection point of the music industry [1][2]. At first some of the music would be a close collaboration of a machine and an artist, but at some point, I wonder if we’d be primarily listening to tunes generated to our specific taste similarly to spotify’s discover weekly. Though it would be all be much closer to what we want to listen to (exploit & explore) at a given time. [1]https://www.youtube.com/watch?v=G5JT16flZwM https://www.youtube.com/watch?v=G5JT16flZwM [2]https://blog.nameshield.com/blog/2018/01/23/hello-world-musical-creation-artificial-intelligence/ https://blog.nameshield.com/blog/2018/01/23/hello-world-musi...

sharikous 5y ago

I wonder why they don't mention the problem of the audio quality of the output. As far as I know the best models work on magnitude spectrograms and have issues with recreating the phase information. Sub-par algorithms like Griffin-Lim are used instead

thomashop 5y ago
The review is about composition in the score domain, not synthesis in the audio domain. Check Differential Digital Signal Processing for state-of-the-art data driven synthesis in th audio domain.
- TomSwirly 5y ago
  It's a very poor "score" domain with no timbral information at all. See my comment elsewhere: https://news.ycombinator.com/item?id=28355582 https://news.ycombinator.com/item?id=28355582
  - thomashop 5y ago
    I was talking about musical scores. They shouldn't have any timbral information.

DrNuke 5y ago

Automated music (art) generation + NFTs = new printing money aka new trading bubble?

PennRobotics 5y ago

The paper begins with David Cope. He evidently has a Youtube channel with algorithmic music. https://www.youtube.com/user/davidhcope/videos https://www.youtube.com/user/davidhcope/videos

JoeDaDude 5y ago
David Cope has written several books and journal publications about his automated composition methods, some examples below: https://www.amazon.com/gp/product/0895792567/ref=dbs_a_def_rwt_bibl_vppi_i97 https://www.amazon.com/gp/product/0895792567/ref=dbs_a_def_r... https://ieeexplore.ieee.org/document/6787993 https://ieeexplore.ieee.org/document/6787993

geekamongus 5y ago

It will only be a matter of time until the AI learns that what sounds pleasing to most humans is often some form of, or extract from, Pachelbel's Canon in D.

bruce343434 5y ago
It is sickening how often I hear it

shannifin 5y ago

Decent review! I've been fascinated by this topic for a while. I think the real magic is of course in the specifications of the details, but I do think it's innevitable that AI-generated (or assisted) music will come to dominate the field as it democratizes what is currently a more specialized skill set. I've always found the resistance to the idea of AI-generated music a bit odd, but I think it stems from a more philosophical idea about where beauty comes from. For example, it's tempting to imagine that the beauty of a Beethoven symphony comes from Beethoven himself, thus his music facilitates an intense personal emotional connection across time and space. The idea of AI music challenges this notion a bit; if AI "composed" something beautiful, and its decisions were not founded on emotions, where is the beauty coming from? Of course, I'd say the beauty is in the natural phenomenon of music itself; the beauty is in the human brain's ability to have a sense of "music" at all. In this way, a connection to Beethoven through his music is a shared recognition of the beauty of certain natural musical phenomenon. The beauty did not come from Beethoven, rather it was "captured" and shared by Beethoven. As far as the business side goes, I've definitely found that there's interest in at least AI-assisted music composition, both from a melody-generating app I used to sell some years ago (limited as it was), and just collecting emails for a new web app I'm working on at https://www.tunesage.com/ https://www.tunesage.com/ ... I know there are already other services out there too for AI music, and I expect the space will continue to grow. I think it's gonna be awesome. :)

vincnetas 5y ago
Im afraid that AI will overwhelm us with perfect amazing music (and other creations) that we will come to a familiar situation that Alan Watts describes : "And you would, naturally as you began on this adventure of dreams, you would fulfill all your wishes. You would have every kind of pleasure you could conceive." And then in this world overwhelmed with perfection we will fall back to that untraceable aspect of human creation that keeps us in awe. Of course we will be much more comfortable in that advanced world in a sense of fulfilled basic needs, but self actualisation (from the pyramid of needs) i guess will never be achieved by AI :)
yawboakye 5y ago
The beauty in human creation doesn't rest entirely in the artefact. A large part of it rests in the knowledge (of the beneficiary) that another skilled human has dedicated time, effort, and intelligence to uniquely produce what is in front of them. It doesn't stop at music. You see it everywhere: fashion (hand-made), food (famous chefs), music (live performances), coffee (moka pots), sex (orgasm stimulated by other humans), etc. It isn't a pursuance of the best (or perfection), but of what has more soul in it. And unfortunately AI music lacks any soul.

bjourne 5y ago

It all comes down to generating tokens. The dominant paradigm is to compose the music by generating the next token based on the last few seen tokens, then take that and generate the next token, and so on. No one has been able to significantly improve upon that. All development in the last ten years or so has been about creating ever larger models with more data so that emitted sequences will stay coherent for longer. The problem with this paradigm is that it works amazingly well for short sequences, but often fails to stay coherent for longer sequences. So composing interesting ten seconds long instrumental music is a solved problem - composing full songs three-to-five minutes long is far beyond the reach of current state-of-the-art. Text-generating models suffer from the same problem. E.g. GPT-3 will generate two to three-hundred tokens or so and then it will begin to spout nonsense. That it still works so well is because it's trained with a massive corpus and because its tokens are richer. It operates on the world level, while most music composition models operate on the note level which are more analogous to characters than words. If someone were to be able to figure out how to represent multi-instrumental compositions using "music vectors" (analogous to word vectors) that would probably lead to a major breakthrough.

pishpash 5y ago
Probably these would be harmonic units. Unfortunately notes aren't just characters in this. There's also voice leading, and so harmony "words" run into each other and can't be mutually exclusively tokenized. The better analogy might be poetry, and I'm not sure if there are good poetry generators, either.
OneEyedRobot 5y ago
By 'token' do you mean a note, a chord, general structure (verse, bridge, etc.), a well known phrase? Music is so full of cliche I don't doubt that someone will write software (probably already has) to assist a composer, generate drum parts or harmony, etc. I like the idea of a Rhythm changes generator. With enough monkeys the Flintstone Theme eventually pops out.
- snthd 5y ago
  >(probably already has) https://github.com/Impro-Visor/Impro-Visor https://github.com/Impro-Visor/Impro-Visor

streamofdigits 5y ago

A key question imho is this: will algorithmic music composition ever produce anything else than more or less sophisticated musac, effectively recycling a particular musical corpus of a particular music tradition? Producing a memorable, emotionally moving piece is not something that is generally reducible to prescription. If the DL based pattern matching is sufficiently intelligent and produces something worthwhile, will it actually feel different to an existing piece? Music perception and appreciation is a deeply human feature (is there a "purpose" to it or is it actually a piece of redundant brain code? https://brianjump.net/2020/11/02/why-does-music-exist/ https://brianjump.net/2020/11/02/why-does-music-exist/) that has both a biological and cultural basis. There is a feedback loop of contemporary musical sounds a young brain is exposed to and the perception machinery and associated innate pleasurable responses. Think of the countless cultural innovations that underpin human music: tunings, scales, rhythms, genres and the deep link to voice and song. None of those has any (ex-ante) algorithmic feel (to the chagrin of Pythagoreans and other ex-post numerologists). Coming up with music feels literally like plucking beautiful sounds from thin air. In any case its a fascinating research domain...

PaulDavisThe1st 5y ago
> A key question imho is this: will algorithmic music composition ever produce anything else than more or less sophisticated musac, effectively recycling a particular musical corpus of a particular music tradition? A related question is: how many human composers manage do this? It seems that we are fairly OK with a large body of composers "effectively recycling a particular musical corpus".
- streamofdigits 5y ago
  There is indeed a lot of "filler" type production and even the most talented creators are far from always producing unique pieces that touch audiences (let alone endure). But this misses the point: some creators do occasionally hit the magic sweet spot and invent something that makes the rest worthwhile. Heck, some creators invent new genres. A DL type algorithm will not produce a new "classic" except by a "thousand monkeys hitting keyboards" type accident... at least not until we dig far deeper into what makes some music stand-out. And an algorithm as conventionally defined is unlikely to ever produce a new musical style. The true opportunity (as always with algorithms) is the augmentation of creative talent.

YeGoblynQueenne 5y ago

This is an interesting and informative article but to be a bit meta I'm concerned when I see articles like this on HN because they don't usually do a great job of summarising the work that has been done in a certain area before the advent of deep neural nets. This is important because very often, especially when it comes to generative art, the standard approaches used before deep neural nets could do thins that modern deep neural nets cannot do, in particular when it comes to structured generation. For example, alorithmic music is the subject area of generating music with algorithmic approaches, not necessarily using a computer. The wikipedia page seems to be a bit poor in detail but it lists a number of different approaches most of which are not machine larning approaches: https://en.wikipedia.org/wiki/Algorithmic_composition https://en.wikipedia.org/wiki/Algorithmic_composition I'm by no means an expert but that's the point. When a non-expert reads an article like the one above, I fear they may get an impression that neural nets are the first approach ever to generate music, or that they are the best approach ever to generate music, or anyway some kind of misunderstanding that is natural to draw from incomplete information. The thing to try and keep in mind is that computer scientists, and other scientists and creative people, had been able to do amazing things with the tools they had in their disposal long before the advent of deep neural nets. And that there are many such tools that are not deep neural nets. Somehow these amazing things flew under the radar of technies - until deep neural nets came along and suddendly everyone is amazed that "wow, neural nets can do X!". Well, what else can do X? That's something worth trying to find out.

platz 5y ago
are you referring to the historic pre-classical period prior to 0 BIM (Before ImageNet)
- YeGoblynQueenne 5y ago
  I'm talking, for example, about the difference between images of plants generated by L-systems [1] and images of plants generated by GANs [2]. _______________ [1] https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/L-system [2] https://www.easyzoom.com/imageaccess/2128f27845ed4921b3143004dc550c3e https://www.easyzoom.com/imageaccess/2128f27845ed4921b314300...

TomSwirly 5y ago

The article starts really badly with "Music is generally defined as a succession of pitches or rhythms, or both, in some definite patterns." The big three in music are pitch, rhythm and TIMBRE, guys, TIMBRE (the "sound" or "tone" of the piece). Why is Beethoven's Ode To Joy somewhat annoying on the recorder, and can drive a room of hardened Germans to tears when played by a top-notch symphony orchestra? Timbre! In the construction of classical musical pieces, this is controlled with orchestration. It's why a Hans Zimmermann score sounds punchy or etheriel or whatever is called for by the script. I basically stopped there, because, well, if you don't care how your music _sounds_ it isn't going to sound very good.

PaulDavisThe1st 5y ago
> The big three in music are pitch, rhythm and TIMBRE, The big three are normally described as melody, harmony and rhythm, and yes, it should be four because timbre is clearly central to the construction and experience of music.
shoemakersteve 5y ago
I personally disagree. Timbre will give the music a certain quality, and while it's no doubt a big part of the experience, I think the dynamics of piece will be much more central to the emotional response. If a song has a sax solo that really touches your heart, and you replaced it with a trombone playing the exact same solo instead, the timbre would be completely different, but the emotions that it would evoke would likely be almost identical. The timbre would mainly affect how well one or the other might blend with the rest of the orchestration. If someone played Ode To Joy on a recorder with proper dynamics, I can guarantee they could make it sound good. All my personal opinion of course.

qwerty456127 5y ago

I am eager to be able to choose a classical music piece (e.g. Bach Cello Suite #1 in G) and ask an AI to generate an infinite sequence very close to its style but variating slightly.

adamnemecek 5y ago

I’ve been working on an IDE for music composition http://ngrid.io http://ngrid.io. Launching soon.

gwern 5y ago

Not a single mention of Jukebox.

spywaregorilla 5y ago

I'm not particularly familiar with music creation, but it always seemed to me neural nets would be better at handling music production more than composition. That is, changing the sound to be, say, a trent reznor style output given the original. Does that make any sense?

thebricksta 5y ago
As someone with a fair amount of experience in both domains, I'd agree with this. I see a lot more utility in sculpting sound and helping streamline the production side of things than the uses of NNs on the composition side. There already are tools for automatically composing melodies and harmonies, and some of them are actually quite good, but they're hardly used.

thebricksta 5y ago

The best definition I've seen for the success of a piece of music is this: "What emotion is the artist trying to convey and how well does it convey it?" Throughout the composing, arranging, recording, mixing, and mastering process, there are thousands of choices to be made, and the correctness of each choice is entirely linked back to that goal: Does the choice help to convey the emotion, or does it detract from it? To that end, there is no correct choice, no correct or optimal harmony, no correct note, no correct rhythm, no correct timbre. It's all contextual in relation to conveying the desired emotion. I'm really not sure how you could ever train a NN to make choices in that regard without first trying to teach them how to understand the impacts of their choices on the emotions conveyed. At best, you may be able to train a NN to reproduce emotionally-void works in a particular style, and perhaps assign some emotion through the timbres selected (ambient music comes to mind here). Still, this isn't much of an achievement. You could easily codify the rules taught in Music 101 about harmonization and melody composition to a computer and have it spit out bland but pleasant excerpts, no deep learning required.