10 ms·

A.I. Duet: A piano that responds to you

262 points by dsr12 10y ago

krosaen 10y ago
I like the concept and the framing of the demo is really cool, but the actual music generated in response to me doing basic things like playing a blues riff or a pentatonic scale was usually just a repeat of some fragment of what I was playing. It felt like something that could be built using a basic heuristic...
- standardly 10y ago
  Agreed, it seems to either play a note you've played or pick a random interval. The interval and timing perhaps aren't truly random but it makes no difference to the listener's ear. Doesn't appear to have any concept of, or the ability to learn, scales, rhythm, or melody (or chords). Neat idea, but akin to playing piano with a 5-year-old (perhaps not as fun).
  - GrinningFool 10y ago
    Of course that may change as people could be expanding its training through their interactions with it.
- raverbashing 10y ago
  Yeah, had the same impression It seems it almost gets the concept of tone, but barely And for some reason it worked on Firefox, but not Chrome (it doesn't even load for me)
- hellofunk 10y ago
  Could just be a limitation of the training resources available.
  - 6stringmerc 10y ago
    I don't think there's a practical limitation on training resources with respect to Music. Maybe the authors didn't get into any of the rubric, but it's certainly out there. Gobs of it.
    - hellofunk 10y ago
      By resources, I meant training time, CPU/GPU resources, not availability of data.
      - imaginenore 10y ago
        At Google? Ha.
- deleted 10y ago
  [deleted]
- zodPod 10y ago
  It almost felt random at times.
- svantana 10y ago
  Yeah either the examples from the video are cherry-picked, or something's wrong with the deployment, because what I get is way worse than what's in the video. I really don't like this new trend of throwing Deep Learning at anything without really checking whether a simple model (e.g. markov chains in this case) would give equal results. Because using a simple model wouldn't be "cool" or newsworthy...
  - ocdtrekkie 10y ago
    When you've described the entire new vision of your company as being "doing things with deep learning", you aren't going to bother seeing if the simpler solution works. Just isn't good for business.
    - fest 10y ago
      But then a competitor comes along, doing the same thing with vastly less complex model and capable of offering cheaper/faster service because of that.
  - sb8244 10y ago
    Here's a relevant excerpt from research I did on college into this. FWIW, my research was specifically into genetic algorithms and generating similar musical styles as the input music. It didn't perform particularly well but had some surprises. From my research, a Markov model could potentially be viable for this demo in particular. However, it wouldn't be viable for longer generation of music where you're trying to produce an actual song. The biggest problem being that it doesn't have the concept of melody in it. --- research after here --- A Markov model utilized by Herfort and Rehberger [17] uses the immediate previous note in order to determine the next note against a set of probabilities. This is a fairly simple algorithm as the set of rules is not large, but it is extremely short-sighted by nature. This “first order” Markov chain is not capable of determining a probability based on a large sequence of pitches. To counteract this, “N-order” Markov chain models can be determined to look more deeply into the past, but the complexity of the rule set increases exponentially with added depth. Further, every aspect of music that is to be modeled would require its own Markov model. For instance, there would need to be a model for pitch, duration, etc. This leads to a potentially huge Markov model rule set where all chains are independent of each other. Thus, Markov chains are good at composing musical melodies on the micro level (such as a measure) but fail to produce quality music at the macro level. [17] Rehberger, Leopold. “Composing music using AI algorithms.” GRIN Publishing GmbH. 2004. <http://www.grin.com/en/e-book/108991/composing-music-using-ai-algorithms> http://www.grin.com/en/e-book/108991/composing-music-using-a.... Source: https://github.com/sb8244/GeneticMusicComposition https://github.com/sb8244/GeneticMusicComposition
- LoSboccacc 10y ago
  even going a simple jingle bells loop (I know, I haven't studied much) get's a pretty much nonsensical repeat response. on a side note I'd like to have an option to put ai on mute and have the duet run after a session, since it's extremely disorienting for a noob to play with random noise on the feedback loop
dominotw 10y ago
just tried this on my midi keyboard( web midi is awesome!) but the "AI music" response was very underwhelming and sounded bad. I would be interested if "AI" can help me with different timbres based on the physical environment that music is playing using various sensor like air pressure, humidity, room resonance, maybe even heartbeat, perspiration ect. Make that a new dimension of information coded into the song along with pitch, timing, amp and timbre.
rrherr 10y ago
Style transfer and generative adversarial networks have worked well with images. How can these be applied to music?
- increment_i 10y ago
  It does seem like an interesting edge to explore doesn't it? ML techniques combined with spectral analysis could yield some interesting outcomes. The datasets wouldn't be too hard to find either - most people have pretty extensive digital music libraries.
- iamwil 10y ago
  Is that how they did it? Or is it just training some RNNs to repeat the same style back?
- iverjo 10y ago
  Sony has been working on style (genre) transfer in music lately: https://youtu.be/I9M8l2guPSo?t=21m30s https://youtu.be/I9M8l2guPSo?t=21m30s
6stringmerc 10y ago
Definitely a work in progress. Was like having a goddamn cat following my fingers around the piano. No sense of rhythm or tonality whatsoever. Like, it objectively made whatever I was playing sound worse. It may "respond" but it's a lot more like throwing electricity through a dead body than working with a competent duet partner.
- Ajedi32 10y ago
  FWIW, it's not really meant to be playing alongside you. The idea is for you to play a short tune, then wait for the AI to respond with a tune of its own based off of yours.
  - 6stringmerc 10y ago
    Well, FWIW as well I tried exactly that as well and it was like it could barely follow what I did (rhythm was off) and then it went in other directions that were not pleasing to the ear. Nor made much of any sense musically. So, again, it's like a cat or child was mimicking my motions just enough to make me dislike the result.
- vegabook 10y ago
  Haha...love the cat and dead body idioms. Captures the fail perfectly.
ovulator 10y ago
I'm not sure if it is a reflection of my own lack of understanding of music, but it sounded like chaos to me. Garbage in garbage out though.
catnaroek 10y ago
To whoever implemented this AI: Can you actually play music yourself? Your AI can't. I tried to play “Satori Maiden” (link: https://www.youtube.com/watch?v=0c9bea_thaM https://www.youtube.com/watch?v=0c9bea_thaM ), and your AI only replied with unmusical nonsense.
- hellofunk 10y ago
  Go easy man. It's an "experiment" for a reason.
  - catnaroek 10y ago
    I'm opposed to this trend of trying to replace actual understanding of the problem at hand with massive amounts of computation. If you want to write a program that writes music, then learn how music is actually written first. --- @hellofunk: If you want the ML algorithm to actually learn how to write music, at the very least, the programmer must be able to express the goal of producing musical sequences of notes.
    - jgalloway___ 10y ago
      It's opensource bro. Translate your problem to a requirement and impress us.
      - 6stringmerc 10y ago
        Look, it couldn't even follow a basic tempo when I was playing. That's not even close to being a "duet" as described. Maybe whoever wrote it has a limited grasp of English, because that experiment is a far, far, far, far, far, far, far cry from a "Duet" experience. Well, one where you don't look over at your partner and wonder if the have any clue how music works before pushing them off the piano bench.
        frogpelt 10y ago
        That's too many fars.
wufufufu 10y ago
I played the A-section for Heart and Soul (the song from the movie "Big") and its response was enough to tell me it was not going to be playing actual duets. I'd be interested in a version that has learned accompanying portions to many popular melodies. If I could play the right hand part of a piece and it played the left hand I'd be very impressed -- also this would actually be useful software for aspiring pianists.
jgalloway___ 10y ago
As an experiment I think this is well done. It's fun to mash the keys and see what happens. If there was a feedback loop to register if the user was happy or displeased with the results via facial recognition or other input with expanded training data it could become quite interesting I think. Thanks for this!
ssharp 10y ago
My immediate reaction was to try and confuse it, so I played "Roll Call" from the Thomas show (kids are obsessed !) It's a bit chromatic so I thought it would have trouble, which it did. Next, I tried putzing around on the white keys but made sure to never hit a F or F#. I wanted to see if it would try and guess which key I was playing in, C or G major. It seemed to guess C most of the time, though one time I tried hinting at G by starting out on G, it got confused and starting playing both F and F#. I was just clicking with a mouse, so rhythm was hard to judge but overall this thing did a really bad job "complimenting" my playing!
- jff 10y ago
  You can use your keyboard, the middle row maps to the white keys and the upper row maps to black keys. Works better than a mouse for tempo, IMO.
  - coroxout 10y ago
    Are these standard keys for some other piece of music software? I keep getting confused because I'm so used to the old-style tracker keys (I keep hitting J for Bb and getting B, which isn't helping my C minor improv). Also one octave isn't great, lag is killing the rhythm, and it keeps not waiting for me to finish. Neat idea though. (I'm curious to know whether the training set for the version shown in the demo is the same as for the live web version, as the demo sounded way better than anything I'm getting out of it - though obviously they would have cherrypicked the results for the video.)
    - fryguy 10y ago
      It seems to be the same keyboard mapping that Reason uses: https://youtu.be/7maQ1zjvb-M?t=28 https://youtu.be/7maQ1zjvb-M?t=28
jgalloway___ 10y ago
For those looking for harmony try letting the machine lead .. if you mimic it's style and complement with a hint of your own rhythm it can be satisfying.
amelius 10y ago
I think Google can learn something from Microsoft here: [1] [1] https://www.youtube.com/watch?v=3oGFogwcx-E https://www.youtube.com/watch?v=3oGFogwcx-E
dnprock 10y ago
I think it'd sound more interesting if the computer compliments what I play. For example, if I play in C key, the computer tries to guess the chord progression and play the rhythm that matches what I'm playing.
stevehiehn 10y ago
This is cool! I've been trying to use ML to automate the entire production pipeline. Choose chords, synths, loops, mix and add FX. http://signalsandsorcery.com/ http://signalsandsorcery.com/ Im finding extracting audio features from loops the hardest part. The selection of notes is actually not as difficult as choosing a good timbre for the notes.
jrowley 10y ago
I want to hear one trained on Monk, Bill Evans, etc.
- zhemao 10y ago
  I get the feeling this one was trained on Cecil Taylor.
deleted 10y ago
[deleted]
stevehiehn 10y ago
I've had the most success in my own experiments using many dedicated Models. The first and most important model is a NN that generates a chord progression. After that is much easier to select good notes! http://signalsandsorcery.com/ http://signalsandsorcery.com/
- tyingq 10y ago
  Markov chains might do better for progressions :)
  - stevehiehn 10y ago
    I have tried just using N-grams which is pretty good but it doesn't have any structure. My next experiment will be to use the 'word2vec' algorithm the same way it it used for NLP. Meaning first create relationships between chords then create sequences with RNN.
cocktailpeanuts 10y ago
I know this is meant to be a "cool experiment", but from trying this out for a while my impression is: I could just build a random sequence generator and tell the world "I made this with tensorflow" and people would believe. I mean, none of the response sounded like music. Which is fine, but then again, what is the machine learning for, if it just sounds like random keystrokes?
- deleted 10y ago
  [deleted]
- sordina 10y ago
  I think that this response is confused. What is your point? That people can't differentiate random notes from a duet? That people would believe that anything associated with tensorflow was imbued with sophisticated behavior? How does that relate to this experiment? It's clearly not just playing random notes in the example video...
  - gfody 10y ago
    The AI is clearly improvising musically based on the input, it's far from random notes. It's actually pretty cool and I look forward to seeing where it goes from here. The person you're responding to is just an asshole.
    - MattRix 10y ago
      What's so hard to understand here? The music it plays sounds awful! I have no doubt te tech behind it is interesting but the results are pretty bad.
      - gfody 10y ago
        I played with this for a while and never got the impression that it was random notes. A few times it played something quite musical which I found interesting and cool. There seems to be a lot of haters in here for what I can only imagine is some sort of sour grapes jealousy or some shit.. maybe it's the production quality of this video is out of proportion for the effort put into the experiment, I dunno but the experiment itself is cool.
y04nn 10y ago
Make me think about the movie "Electric Dreams" https://youtu.be/aVE8taDWmMc https://youtu.be/aVE8taDWmMc
cyberferret 10y ago
I was excited to play with this, but ultimately left disappointed. I was hoping that my initial melodies would be expanded upon and that I could build a series of riffs that would become increasingly complex but based on the same starting melody, but in the end, the responses from the AI seemed to be fairly random, and didn't seem to be built upon the previous sequences at all. The AI also changed keys a couple of times, which was disconcerting.
camtarn 10y ago
Me: plays a couple of minor chords AI: long pause; holds down one random key for several seconds; boops another random key "Nailed it." After spending a few minutes playing, I did see evidence of it occasionally tracking my chords (as long as they were very common - e.g. c major, a minor) and tempo (but only matching tempo, as in vaguely matching how fast I was playing, not actually playing in time with me). I played an extended chord sequence and it made one or two nice contributions, but mostly it was just random noise. I wonder if this is due to lack of training data? Maybe it needs an actual data set of people specifically trading musical phrases with each other, rather than just a set of general music. It's an amazing idea, and I can actually see it being quite interesting for things like musical phrase creation - supply the first half of a phrase and see what the machine comes up with for the second half, and tweak as appropriate. But currently it's a bit of a case of over promising and under delivering.
return0 10y ago
I believe ML can do better than this.
jpalla 10y ago
If you are interested in real-time generative music, check out IRCAM's project SoMax/OMax: http://repmus.ircam.fr/omax/home http://repmus.ircam.fr/omax/home Sample (software is piano trained with Bill Evans corpus): http://www.dailymotion.com/video/x2j39hf_somax-with-remi-fox-at-the-ircam-open-doors-june-2014_music http://www.dailymotion.com/video/x2j39hf_somax-with-remi-fox...
deleted 10y ago
[deleted]
biot 10y ago
This is fun. It actually feels like I'm taking turns with someone else on the other end of the screen who takes what I play and sends back a riff on the notes. The trick is (as the little intro shows) to only play for a few seconds, don't play simultaneous notes, and then stop and let the AI respond. Once the AI has finished, you can try again. If the AI starts playing before you're done, you've screwed up and should make your little sequence of notes shorter. Basically, the title of this is misleading... if you keep playing expecting the AI to join you in a simultaneous duet, it'll sound like cats fighting inside a piano. Try playing the "Dukes of Hazard" horn: on your keyboard, press (without overlapping the keypresses) "LJGGGHJKLLLJ" and let the AI respond. More often than not, it'll come back with a vaguely pleasing riff on that sequence using mostly the same notes and in the same tempo.
shams93 10y ago
I'm getting some good results but I'm also a clasically trained musician so if you haven't studied music it may not give you a pleasing result but if you are a trained musician you may get more mileage.
_andromeda_ 10y ago
Amazing. Oh, the possibilities.
algomusic 10y ago
Even more sophisticated duets with computationally creative computer: CIM sessions 2017 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort=dd&shelf_id=0&view=0 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort...
algomusic 10y ago
Even more sophisticated duets with a computationally creative computer: CIM Sessions 2017. https://www.youtube.com/user/QCGInteractiveMusic/videos?sort=dd&shelf_id=0&view=0 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort...
erinjerri1678 10y ago
I tried this at Google Tensorflow Summit and it was fun!

262 points by dsr12 10y ago

I like the concept and the framing of the demo is really cool, but the actual music generated in response to me doing basic things like playing a blues riff or a pentatonic scale was usually just a repeat of some fragment of what I was playing. It felt like something that could be built using a basic heuristic...

standardly 10y ago
Agreed, it seems to either play a note you've played or pick a random interval. The interval and timing perhaps aren't truly random but it makes no difference to the listener's ear. Doesn't appear to have any concept of, or the ability to learn, scales, rhythm, or melody (or chords). Neat idea, but akin to playing piano with a 5-year-old (perhaps not as fun).
- GrinningFool 10y ago
  Of course that may change as people could be expanding its training through their interactions with it.
raverbashing 10y ago
Yeah, had the same impression It seems it almost gets the concept of tone, but barely And for some reason it worked on Firefox, but not Chrome (it doesn't even load for me)
hellofunk 10y ago
Could just be a limitation of the training resources available.
- 6stringmerc 10y ago
  I don't think there's a practical limitation on training resources with respect to Music. Maybe the authors didn't get into any of the rubric, but it's certainly out there. Gobs of it.
  - hellofunk 10y ago
    By resources, I meant training time, CPU/GPU resources, not availability of data.
    - imaginenore 10y ago
      At Google? Ha.
deleted 10y ago
[deleted]
zodPod 10y ago
It almost felt random at times.
svantana 10y ago
Yeah either the examples from the video are cherry-picked, or something's wrong with the deployment, because what I get is way worse than what's in the video. I really don't like this new trend of throwing Deep Learning at anything without really checking whether a simple model (e.g. markov chains in this case) would give equal results. Because using a simple model wouldn't be "cool" or newsworthy...
- ocdtrekkie 10y ago
  When you've described the entire new vision of your company as being "doing things with deep learning", you aren't going to bother seeing if the simpler solution works. Just isn't good for business.
  - fest 10y ago
    But then a competitor comes along, doing the same thing with vastly less complex model and capable of offering cheaper/faster service because of that.
- sb8244 10y ago
  Here's a relevant excerpt from research I did on college into this. FWIW, my research was specifically into genetic algorithms and generating similar musical styles as the input music. It didn't perform particularly well but had some surprises. From my research, a Markov model could potentially be viable for this demo in particular. However, it wouldn't be viable for longer generation of music where you're trying to produce an actual song. The biggest problem being that it doesn't have the concept of melody in it. --- research after here --- A Markov model utilized by Herfort and Rehberger [17] uses the immediate previous note in order to determine the next note against a set of probabilities. This is a fairly simple algorithm as the set of rules is not large, but it is extremely short-sighted by nature. This “first order” Markov chain is not capable of determining a probability based on a large sequence of pitches. To counteract this, “N-order” Markov chain models can be determined to look more deeply into the past, but the complexity of the rule set increases exponentially with added depth. Further, every aspect of music that is to be modeled would require its own Markov model. For instance, there would need to be a model for pitch, duration, etc. This leads to a potentially huge Markov model rule set where all chains are independent of each other. Thus, Markov chains are good at composing musical melodies on the micro level (such as a measure) but fail to produce quality music at the macro level. [17] Rehberger, Leopold. “Composing music using AI algorithms.” GRIN Publishing GmbH. 2004. <http://www.grin.com/en/e-book/108991/composing-music-using-ai-algorithms> http://www.grin.com/en/e-book/108991/composing-music-using-a.... Source: https://github.com/sb8244/GeneticMusicComposition https://github.com/sb8244/GeneticMusicComposition
LoSboccacc 10y ago
even going a simple jingle bells loop (I know, I haven't studied much) get's a pretty much nonsensical repeat response. on a side note I'd like to have an option to put ai on mute and have the duet run after a session, since it's extremely disorienting for a noob to play with random noise on the feedback loop

dominotw 10y ago

just tried this on my midi keyboard( web midi is awesome!) but the "AI music" response was very underwhelming and sounded bad. I would be interested if "AI" can help me with different timbres based on the physical environment that music is playing using various sensor like air pressure, humidity, room resonance, maybe even heartbeat, perspiration ect. Make that a new dimension of information coded into the song along with pitch, timing, amp and timbre.

rrherr 10y ago

Style transfer and generative adversarial networks have worked well with images. How can these be applied to music?

increment_i 10y ago
It does seem like an interesting edge to explore doesn't it? ML techniques combined with spectral analysis could yield some interesting outcomes. The datasets wouldn't be too hard to find either - most people have pretty extensive digital music libraries.
iamwil 10y ago
Is that how they did it? Or is it just training some RNNs to repeat the same style back?
iverjo 10y ago
Sony has been working on style (genre) transfer in music lately: https://youtu.be/I9M8l2guPSo?t=21m30s https://youtu.be/I9M8l2guPSo?t=21m30s

6stringmerc 10y ago

Definitely a work in progress. Was like having a goddamn cat following my fingers around the piano. No sense of rhythm or tonality whatsoever. Like, it objectively made whatever I was playing sound worse. It may "respond" but it's a lot more like throwing electricity through a dead body than working with a competent duet partner.

Ajedi32 10y ago
FWIW, it's not really meant to be playing alongside you. The idea is for you to play a short tune, then wait for the AI to respond with a tune of its own based off of yours.
- 6stringmerc 10y ago
  Well, FWIW as well I tried exactly that as well and it was like it could barely follow what I did (rhythm was off) and then it went in other directions that were not pleasing to the ear. Nor made much of any sense musically. So, again, it's like a cat or child was mimicking my motions just enough to make me dislike the result.
vegabook 10y ago
Haha...love the cat and dead body idioms. Captures the fail perfectly.

ovulator 10y ago

I'm not sure if it is a reflection of my own lack of understanding of music, but it sounded like chaos to me. Garbage in garbage out though.

catnaroek 10y ago

To whoever implemented this AI: Can you actually play music yourself? Your AI can't. I tried to play “Satori Maiden” (link: https://www.youtube.com/watch?v=0c9bea_thaM https://www.youtube.com/watch?v=0c9bea_thaM ), and your AI only replied with unmusical nonsense.

hellofunk 10y ago
Go easy man. It's an "experiment" for a reason.
- catnaroek 10y ago
  I'm opposed to this trend of trying to replace actual understanding of the problem at hand with massive amounts of computation. If you want to write a program that writes music, then learn how music is actually written first. --- @hellofunk: If you want the ML algorithm to actually learn how to write music, at the very least, the programmer must be able to express the goal of producing musical sequences of notes.
  - jgalloway___ 10y ago
    It's opensource bro. Translate your problem to a requirement and impress us.
    - 6stringmerc 10y ago
      Look, it couldn't even follow a basic tempo when I was playing. That's not even close to being a "duet" as described. Maybe whoever wrote it has a limited grasp of English, because that experiment is a far, far, far, far, far, far, far cry from a "Duet" experience. Well, one where you don't look over at your partner and wonder if the have any clue how music works before pushing them off the piano bench.
      - frogpelt 10y ago
        That's too many fars.

wufufufu 10y ago

I played the A-section for Heart and Soul (the song from the movie "Big") and its response was enough to tell me it was not going to be playing actual duets. I'd be interested in a version that has learned accompanying portions to many popular melodies. If I could play the right hand part of a piece and it played the left hand I'd be very impressed -- also this would actually be useful software for aspiring pianists.

jgalloway___ 10y ago

As an experiment I think this is well done. It's fun to mash the keys and see what happens. If there was a feedback loop to register if the user was happy or displeased with the results via facial recognition or other input with expanded training data it could become quite interesting I think. Thanks for this!

ssharp 10y ago

My immediate reaction was to try and confuse it, so I played "Roll Call" from the Thomas show (kids are obsessed !) It's a bit chromatic so I thought it would have trouble, which it did. Next, I tried putzing around on the white keys but made sure to never hit a F or F#. I wanted to see if it would try and guess which key I was playing in, C or G major. It seemed to guess C most of the time, though one time I tried hinting at G by starting out on G, it got confused and starting playing both F and F#. I was just clicking with a mouse, so rhythm was hard to judge but overall this thing did a really bad job "complimenting" my playing!

jff 10y ago
You can use your keyboard, the middle row maps to the white keys and the upper row maps to black keys. Works better than a mouse for tempo, IMO.
- coroxout 10y ago
  Are these standard keys for some other piece of music software? I keep getting confused because I'm so used to the old-style tracker keys (I keep hitting J for Bb and getting B, which isn't helping my C minor improv). Also one octave isn't great, lag is killing the rhythm, and it keeps not waiting for me to finish. Neat idea though. (I'm curious to know whether the training set for the version shown in the demo is the same as for the live web version, as the demo sounded way better than anything I'm getting out of it - though obviously they would have cherrypicked the results for the video.)
  - fryguy 10y ago
    It seems to be the same keyboard mapping that Reason uses: https://youtu.be/7maQ1zjvb-M?t=28 https://youtu.be/7maQ1zjvb-M?t=28

jgalloway___ 10y ago

For those looking for harmony try letting the machine lead .. if you mimic it's style and complement with a hint of your own rhythm it can be satisfying.

amelius 10y ago

I think Google can learn something from Microsoft here: [1] [1] https://www.youtube.com/watch?v=3oGFogwcx-E https://www.youtube.com/watch?v=3oGFogwcx-E

dnprock 10y ago

I think it'd sound more interesting if the computer compliments what I play. For example, if I play in C key, the computer tries to guess the chord progression and play the rhythm that matches what I'm playing.

stevehiehn 10y ago

This is cool! I've been trying to use ML to automate the entire production pipeline. Choose chords, synths, loops, mix and add FX. http://signalsandsorcery.com/ http://signalsandsorcery.com/ Im finding extracting audio features from loops the hardest part. The selection of notes is actually not as difficult as choosing a good timbre for the notes.

jrowley 10y ago

I want to hear one trained on Monk, Bill Evans, etc.

zhemao 10y ago
I get the feeling this one was trained on Cecil Taylor.

deleted 10y ago

[deleted]

stevehiehn 10y ago

I've had the most success in my own experiments using many dedicated Models. The first and most important model is a NN that generates a chord progression. After that is much easier to select good notes! http://signalsandsorcery.com/ http://signalsandsorcery.com/

tyingq 10y ago
Markov chains might do better for progressions :)
- stevehiehn 10y ago
  I have tried just using N-grams which is pretty good but it doesn't have any structure. My next experiment will be to use the 'word2vec' algorithm the same way it it used for NLP. Meaning first create relationships between chords then create sequences with RNN.

cocktailpeanuts 10y ago

I know this is meant to be a "cool experiment", but from trying this out for a while my impression is: I could just build a random sequence generator and tell the world "I made this with tensorflow" and people would believe. I mean, none of the response sounded like music. Which is fine, but then again, what is the machine learning for, if it just sounds like random keystrokes?

deleted 10y ago
[deleted]
sordina 10y ago
I think that this response is confused. What is your point? That people can't differentiate random notes from a duet? That people would believe that anything associated with tensorflow was imbued with sophisticated behavior? How does that relate to this experiment? It's clearly not just playing random notes in the example video...
- gfody 10y ago
  The AI is clearly improvising musically based on the input, it's far from random notes. It's actually pretty cool and I look forward to seeing where it goes from here. The person you're responding to is just an asshole.
  - MattRix 10y ago
    What's so hard to understand here? The music it plays sounds awful! I have no doubt te tech behind it is interesting but the results are pretty bad.
    - gfody 10y ago
      I played with this for a while and never got the impression that it was random notes. A few times it played something quite musical which I found interesting and cool. There seems to be a lot of haters in here for what I can only imagine is some sort of sour grapes jealousy or some shit.. maybe it's the production quality of this video is out of proportion for the effort put into the experiment, I dunno but the experiment itself is cool.

y04nn 10y ago

Make me think about the movie "Electric Dreams" https://youtu.be/aVE8taDWmMc https://youtu.be/aVE8taDWmMc

cyberferret 10y ago

I was excited to play with this, but ultimately left disappointed. I was hoping that my initial melodies would be expanded upon and that I could build a series of riffs that would become increasingly complex but based on the same starting melody, but in the end, the responses from the AI seemed to be fairly random, and didn't seem to be built upon the previous sequences at all. The AI also changed keys a couple of times, which was disconcerting.

camtarn 10y ago

Me: plays a couple of minor chords AI: long pause; holds down one random key for several seconds; boops another random key "Nailed it." After spending a few minutes playing, I did see evidence of it occasionally tracking my chords (as long as they were very common - e.g. c major, a minor) and tempo (but only matching tempo, as in vaguely matching how fast I was playing, not actually playing in time with me). I played an extended chord sequence and it made one or two nice contributions, but mostly it was just random noise. I wonder if this is due to lack of training data? Maybe it needs an actual data set of people specifically trading musical phrases with each other, rather than just a set of general music. It's an amazing idea, and I can actually see it being quite interesting for things like musical phrase creation - supply the first half of a phrase and see what the machine comes up with for the second half, and tweak as appropriate. But currently it's a bit of a case of over promising and under delivering.

return0 10y ago

I believe ML can do better than this.

jpalla 10y ago

If you are interested in real-time generative music, check out IRCAM's project SoMax/OMax: http://repmus.ircam.fr/omax/home http://repmus.ircam.fr/omax/home Sample (software is piano trained with Bill Evans corpus): http://www.dailymotion.com/video/x2j39hf_somax-with-remi-fox-at-the-ircam-open-doors-june-2014_music http://www.dailymotion.com/video/x2j39hf_somax-with-remi-fox...

deleted 10y ago

[deleted]

biot 10y ago

This is fun. It actually feels like I'm taking turns with someone else on the other end of the screen who takes what I play and sends back a riff on the notes. The trick is (as the little intro shows) to only play for a few seconds, don't play simultaneous notes, and then stop and let the AI respond. Once the AI has finished, you can try again. If the AI starts playing before you're done, you've screwed up and should make your little sequence of notes shorter. Basically, the title of this is misleading... if you keep playing expecting the AI to join you in a simultaneous duet, it'll sound like cats fighting inside a piano. Try playing the "Dukes of Hazard" horn: on your keyboard, press (without overlapping the keypresses) "LJGGGHJKLLLJ" and let the AI respond. More often than not, it'll come back with a vaguely pleasing riff on that sequence using mostly the same notes and in the same tempo.

shams93 10y ago

I'm getting some good results but I'm also a clasically trained musician so if you haven't studied music it may not give you a pleasing result but if you are a trained musician you may get more mileage.

_andromeda_ 10y ago

Amazing. Oh, the possibilities.

algomusic 10y ago

Even more sophisticated duets with computationally creative computer: CIM sessions 2017 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort=dd&shelf_id=0&view=0 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort...

algomusic 10y ago

Even more sophisticated duets with a computationally creative computer: CIM Sessions 2017. https://www.youtube.com/user/QCGInteractiveMusic/videos?sort=dd&shelf_id=0&view=0 https://www.youtube.com/user/QCGInteractiveMusic/videos?sort...

erinjerri1678 10y ago

I tried this at Google Tensorflow Summit and it was fun!