5 ms·

Ok wow, this is interesting. This Guy is quite an ambitious musician. Unfortunately I don't think the "performance" of these cultured neurons will ever live up

Ok wow, this is interesting. This Guy is quite an ambitious musician. Unfortunately I don't think the "performance" of these cultured neurons will ever live up to his expectations, but I give him props for trying something novel. To summarize... Guy Ben-Ary took a sample of his skin cells, and with the help of a couple different biology labs induced skin cells into pluripotent stem cells, then differentiated those cells into neurons. The neurons were plated on a multi-electrode array dish, so electrical signals could be sent to the neurons, and received by an ephys amplifier / recording device. At first I thought he was attempting to train these neural networks to synthesize music, but it actually doesn't seem like that's being attempted. It looks like he's just sort of crudely prodding the neurons to fire (banging on a drum kit mic'd up with outputs to the multi-electrode array dish); at the same time he's amplifying the neural activity, and passing it through some filters in at attempt to make it sound as musical as 10,000 stochastically crackling neurons can be. Here's an example of what that sounds like: https://youtu.be/CWb_7mYLa6g https://youtu.be/CWb_7mYLa6g

subroutine 9y ago
continued... As a neurobiologist/electrophysiologist I suspected this random crackling is what it would sound like, since that's what activity from my cultured neurons sounds like. Any good neuro/psych student should remember this Nobel prize winning Hubel & Weisel experiment: https://youtu.be/8VdFf3egwfg https://youtu.be/8VdFf3egwfg I think this approach to synthesizing music from neural activity is completely misguided. But there are a few tweaks that might salvage the project... 1. First, they need to focus on the output of far fewer neurons. I know ya'll are experts in machine learning-type neural nets - the current "CellF" setup would be like sending the weights of an untrained 10k-node neural net to your laptop speakers, and just looping over those weights MCMC style. I think they would be better served to isolate the output a few neurons and translate those activation patterns into prefabricated musical samples. For example if the neuron fires at 60 Hz, the sample on MIDI channel 1 is activated (or something like that). continuing...
- subroutine 9y ago
  2. I get they are experimental musicians and composers chomping at the bit to finally play their finest arcane musique tintinnabulations. But they should first try to provide simple, structured musical inputs (repetitive drum taps would be the best, or square waves of particular frequencies; but at the very least simple repetitive melodies). At least until they can determine if these neurons are reacting in any predictable way to the microelectrode inputs. In one of the videos it basically looks like they mic'd up a petri dish on stage with a bunch of avant garde musicians working their hardest. That's actually pretty hilarious. Anyway, they should hold off on that until after they figure out whether these 'trans' neurons can even hear them. 3. They should try to train these neurons. Instead of microarray dishes though, I'd use optogenetics. From my experience it's much easier to evoke predictable signals using fiber optical input. It would be best to pass their instruments through a computer program though, that translated their sounds into pulse frequencies known to induce synaptic potentiation or synaptic depression. Currently, they are just adding noise to an already noisy system. All that said, this is still a cool project. And FWIW, they are much better neuroscientists than I am a musician.
- svantana 9y ago
  > I think this approach to synthesizing music from neural activity is completely misguided. Perhaps I am overly cynical, but I suspect the artist himself knew it would produce noise basically, but it doesn't matter because this is the type of thing that gets attention and that's the real currency in art. Similarly, there was an analog synth module that came out a while back that was driven by radioactive decay [1]. Both of these could easily be simulated by a few lines of code and nobody could tell the difference. http://www.factmag.com/2017/01/16/geiger-counter-synth-module-video/ http://www.factmag.com/2017/01/16/geiger-counter-synth-modul...
  - subroutine 9y ago
    Yeah I agree. I think he achieved his goal, for sure. Somewhere between the induction of pluripotent cells and plating these stem cells on an MEA dish for neuron differentiation I forgot this guy was preparing an art performance piece.
bigiain 9y ago
FWIW, I saw/hear this in Sydney about a year ago - and a Dorkbot presentation from the artist. It doesn't make mainstream music - if your idea of music doesn't include noise-art you might be disappointed, but while the random clickyness of the "cell's performance" is extremely abstract, he (at least in Sydney) chooses accompanying musicians who're perfectly happy to play against that. The show I saw was an absolutely wonderful piece with pianist Chris Abrahams playing along with and reacting to the effectively random sounds produced by the weird and vaguly ethically curious machinery. (Chris plays - amongst other things, with a band called The Necks, who're kinda similar in that they play hour-long improvised pieces where the piano, bass,and drums just evolve long form soundscapes live on stage. Not everybody's cup of tea, but I find them wonderful to listen to. I have tickets to see them again in a couple of weeks...)
- plaguuuuuu 9y ago
  I had no idea that happened. Now I'm sad. Chris Abrahams is great.
  - bigiain 9y ago
    There's a The Necks gig at The Opera House as part of Vivid on (checks calendar) June 4th... He's also playing Camelot Lounge in July with Melanie Oxley.
    - plaguuuuuu 9y ago
      Nice. I might go to that actually :-)
      - bigiain 9y ago
        Did you end up going? It was _great!_ (The Vivid show last night that is...)
- subroutine 9y ago
  After reading my comments from earlier today - I didn't intend to sound so critical. On the contrary, I'm fucking impressed. Halfway through the video I was thinking to myself, damn I know UC cog/neuro PIs that would be embarrassed to share current project notes with this artist Guy Ben-Ary. I went from 'this should be amusing', to 'omg this musician is really about to demo training dissociated neural cultures to synthesize Tchaikovsky'. Even though it didn't remotely turn out that way, I probably will reach out to him (there are a few off-the-rack methods for marshalling neural activity, which may lead to more interesting outputs than stochastic crackling). Though I suspect you're absolutely right - he achieved his primary goal of creating "noise-art". ---- an aside on the modern academic musical vanguard. I recently attended an event put on by UCSD music department grads; not what I expected. I'm going in thinking - sweet, these doctoral students def were first-chair musicians in HS, classically trained since birth, playing in chamber quartets on the weekend; now they're all probably exploring their talents composing electronic and synthesized music, maybe (hopefully) some rock elements. It's gunna be great! ... The first act was essentially some kid rolling an empty Coca-Cola bottle (with a tiny microphone inside) between his foot and a slab of concrete for about 8 minutes, backed by the faint whirls and chortles of children whispering. So, just a heads-up on that front for anyone missing it. ---- edit: oh and I forgot to mention, there's something to be said about the coolness factor of jamming-out on stage with a neural network that used to be your skin cells.

subroutine 9y ago

continued... As a neurobiologist/electrophysiologist I suspected this random crackling is what it would sound like, since that's what activity from my cultured neurons sounds like. Any good neuro/psych student should remember this Nobel prize winning Hubel & Weisel experiment: https://youtu.be/8VdFf3egwfg https://youtu.be/8VdFf3egwfg I think this approach to synthesizing music from neural activity is completely misguided. But there are a few tweaks that might salvage the project... 1. First, they need to focus on the output of far fewer neurons. I know ya'll are experts in machine learning-type neural nets - the current "CellF" setup would be like sending the weights of an untrained 10k-node neural net to your laptop speakers, and just looping over those weights MCMC style. I think they would be better served to isolate the output a few neurons and translate those activation patterns into prefabricated musical samples. For example if the neuron fires at 60 Hz, the sample on MIDI channel 1 is activated (or something like that). continuing...

subroutine 9y ago
2. I get they are experimental musicians and composers chomping at the bit to finally play their finest arcane musique tintinnabulations. But they should first try to provide simple, structured musical inputs (repetitive drum taps would be the best, or square waves of particular frequencies; but at the very least simple repetitive melodies). At least until they can determine if these neurons are reacting in any predictable way to the microelectrode inputs. In one of the videos it basically looks like they mic'd up a petri dish on stage with a bunch of avant garde musicians working their hardest. That's actually pretty hilarious. Anyway, they should hold off on that until after they figure out whether these 'trans' neurons can even hear them. 3. They should try to train these neurons. Instead of microarray dishes though, I'd use optogenetics. From my experience it's much easier to evoke predictable signals using fiber optical input. It would be best to pass their instruments through a computer program though, that translated their sounds into pulse frequencies known to induce synaptic potentiation or synaptic depression. Currently, they are just adding noise to an already noisy system. All that said, this is still a cool project. And FWIW, they are much better neuroscientists than I am a musician.
svantana 9y ago
> I think this approach to synthesizing music from neural activity is completely misguided. Perhaps I am overly cynical, but I suspect the artist himself knew it would produce noise basically, but it doesn't matter because this is the type of thing that gets attention and that's the real currency in art. Similarly, there was an analog synth module that came out a while back that was driven by radioactive decay [1]. Both of these could easily be simulated by a few lines of code and nobody could tell the difference. http://www.factmag.com/2017/01/16/geiger-counter-synth-module-video/ http://www.factmag.com/2017/01/16/geiger-counter-synth-modul...
- subroutine 9y ago
  Yeah I agree. I think he achieved his goal, for sure. Somewhere between the induction of pluripotent cells and plating these stem cells on an MEA dish for neuron differentiation I forgot this guy was preparing an art performance piece.

bigiain 9y ago

FWIW, I saw/hear this in Sydney about a year ago - and a Dorkbot presentation from the artist. It doesn't make mainstream music - if your idea of music doesn't include noise-art you might be disappointed, but while the random clickyness of the "cell's performance" is extremely abstract, he (at least in Sydney) chooses accompanying musicians who're perfectly happy to play against that. The show I saw was an absolutely wonderful piece with pianist Chris Abrahams playing along with and reacting to the effectively random sounds produced by the weird and vaguly ethically curious machinery. (Chris plays - amongst other things, with a band called The Necks, who're kinda similar in that they play hour-long improvised pieces where the piano, bass,and drums just evolve long form soundscapes live on stage. Not everybody's cup of tea, but I find them wonderful to listen to. I have tickets to see them again in a couple of weeks...)

plaguuuuuu 9y ago
I had no idea that happened. Now I'm sad. Chris Abrahams is great.
- bigiain 9y ago
  There's a The Necks gig at The Opera House as part of Vivid on (checks calendar) June 4th... He's also playing Camelot Lounge in July with Melanie Oxley.
  - plaguuuuuu 9y ago
    Nice. I might go to that actually :-)
    - bigiain 9y ago
      Did you end up going? It was _great!_ (The Vivid show last night that is...)
subroutine 9y ago
After reading my comments from earlier today - I didn't intend to sound so critical. On the contrary, I'm fucking impressed. Halfway through the video I was thinking to myself, damn I know UC cog/neuro PIs that would be embarrassed to share current project notes with this artist Guy Ben-Ary. I went from 'this should be amusing', to 'omg this musician is really about to demo training dissociated neural cultures to synthesize Tchaikovsky'. Even though it didn't remotely turn out that way, I probably will reach out to him (there are a few off-the-rack methods for marshalling neural activity, which may lead to more interesting outputs than stochastic crackling). Though I suspect you're absolutely right - he achieved his primary goal of creating "noise-art". ---- an aside on the modern academic musical vanguard. I recently attended an event put on by UCSD music department grads; not what I expected. I'm going in thinking - sweet, these doctoral students def were first-chair musicians in HS, classically trained since birth, playing in chamber quartets on the weekend; now they're all probably exploring their talents composing electronic and synthesized music, maybe (hopefully) some rock elements. It's gunna be great! ... The first act was essentially some kid rolling an empty Coca-Cola bottle (with a tiny microphone inside) between his foot and a slab of concrete for about 8 minutes, backed by the faint whirls and chortles of children whispering. So, just a heads-up on that front for anyone missing it. ---- edit: oh and I forgot to mention, there's something to be said about the coolness factor of jamming-out on stage with a neural network that used to be your skin cells.