7 ms·

Experimental analog synth controlled by stem cell neural network

43 points by glitcher 9y ago

21 9y ago
I think the ethical aspect of this is interesting. Are you allowed to experiment on your own cells? How will this interact with various laws regarding human experimentation.
- hobarrera 9y ago
  I was thinking the same thing. I kinda dislike that the called it "HIS external brain". They gathered cells and created a new organism using it a a base. It's not "him" any more. If it had been an embryo, and they let it grow 10 years (eg: into a child), would they still call it part of him? My example scenario is quite absurd, I know, I'm merely attempting to show that where we draw the line [if at all] is extremely fuzzy. More on point; can this organism feel pain? The amount of neurons seems to be in the same order of magnitude as flies.
  - subroutine 9y ago
    You guys might be interested in a court case known as Moore v. Regents of the University of California, https://en.wikipedia.org/wiki/Moore_v._Regents_of_the_University_of_California https://en.wikipedia.org/wiki/Moore_v._Regents_of_the_Univer... > A was a landmark Supreme Court of California decision filed on July 9, 1990, which dealt with the issue of property rights to one's own cells taken in samples by doctors or researchers. In 1976, John Moore was treated for hairy cell leukemia by physician David Golde, a cancer researcher at the UCLA Medical Center. Moore's cancer cells were later developed into a cell line that was commercialized by Golde and UCLA. The California Supreme Court ruled that a hospital patient's discarded blood and tissue samples are not his personal property and that individuals do not have rights to a share in the profits earned from commercial products or research derived from their cells. To my knowledge there is no case law for "non-discarded" cells.
subroutine 9y ago
Ok wow, this is interesting. This Guy is quite an ambitious musician. Unfortunately I don't think the "performance" of these cultured neurons will ever live up to his expectations, but I give him props for trying something novel. To summarize... Guy Ben-Ary took a sample of his skin cells, and with the help of a couple different biology labs induced skin cells into pluripotent stem cells, then differentiated those cells into neurons. The neurons were plated on a multi-electrode array dish, so electrical signals could be sent to the neurons, and received by an ephys amplifier / recording device. At first I thought he was attempting to train these neural networks to synthesize music, but it actually doesn't seem like that's being attempted. It looks like he's just sort of crudely prodding the neurons to fire (banging on a drum kit mic'd up with outputs to the multi-electrode array dish); at the same time he's amplifying the neural activity, and passing it through some filters in at attempt to make it sound as musical as 10,000 stochastically crackling neurons can be. Here's an example of what that sounds like: https://youtu.be/CWb_7mYLa6g https://youtu.be/CWb_7mYLa6g
- subroutine 9y ago
  continued... As a neurobiologist/electrophysiologist I suspected this random crackling is what it would sound like, since that's what activity from my cultured neurons sounds like. Any good neuro/psych student should remember this Nobel prize winning Hubel & Weisel experiment: https://youtu.be/8VdFf3egwfg https://youtu.be/8VdFf3egwfg I think this approach to synthesizing music from neural activity is completely misguided. But there are a few tweaks that might salvage the project... 1. First, they need to focus on the output of far fewer neurons. I know ya'll are experts in machine learning-type neural nets - the current "CellF" setup would be like sending the weights of an untrained 10k-node neural net to your laptop speakers, and just looping over those weights MCMC style. I think they would be better served to isolate the output a few neurons and translate those activation patterns into prefabricated musical samples. For example if the neuron fires at 60 Hz, the sample on MIDI channel 1 is activated (or something like that). continuing...
  - subroutine 9y ago
    2. I get they are experimental musicians and composers chomping at the bit to finally play their finest arcane musique tintinnabulations. But they should first try to provide simple, structured musical inputs (repetitive drum taps would be the best, or square waves of particular frequencies; but at the very least simple repetitive melodies). At least until they can determine if these neurons are reacting in any predictable way to the microelectrode inputs. In one of the videos it basically looks like they mic'd up a petri dish on stage with a bunch of avant garde musicians working their hardest. That's actually pretty hilarious. Anyway, they should hold off on that until after they figure out whether these 'trans' neurons can even hear them. 3. They should try to train these neurons. Instead of microarray dishes though, I'd use optogenetics. From my experience it's much easier to evoke predictable signals using fiber optical input. It would be best to pass their instruments through a computer program though, that translated their sounds into pulse frequencies known to induce synaptic potentiation or synaptic depression. Currently, they are just adding noise to an already noisy system. All that said, this is still a cool project. And FWIW, they are much better neuroscientists than I am a musician.

43 points by glitcher 9y ago

21 9y ago

I think the ethical aspect of this is interesting. Are you allowed to experiment on your own cells? How will this interact with various laws regarding human experimentation.

hobarrera 9y ago
I was thinking the same thing. I kinda dislike that the called it "HIS external brain". They gathered cells and created a new organism using it a a base. It's not "him" any more. If it had been an embryo, and they let it grow 10 years (eg: into a child), would they still call it part of him? My example scenario is quite absurd, I know, I'm merely attempting to show that where we draw the line [if at all] is extremely fuzzy. More on point; can this organism feel pain? The amount of neurons seems to be in the same order of magnitude as flies.
- subroutine 9y ago
  You guys might be interested in a court case known as Moore v. Regents of the University of California, https://en.wikipedia.org/wiki/Moore_v._Regents_of_the_University_of_California https://en.wikipedia.org/wiki/Moore_v._Regents_of_the_Univer... > A was a landmark Supreme Court of California decision filed on July 9, 1990, which dealt with the issue of property rights to one's own cells taken in samples by doctors or researchers. In 1976, John Moore was treated for hairy cell leukemia by physician David Golde, a cancer researcher at the UCLA Medical Center. Moore's cancer cells were later developed into a cell line that was commercialized by Golde and UCLA. The California Supreme Court ruled that a hospital patient's discarded blood and tissue samples are not his personal property and that individuals do not have rights to a share in the profits earned from commercial products or research derived from their cells. To my knowledge there is no case law for "non-discarded" cells.

subroutine 9y ago

Ok wow, this is interesting. This Guy is quite an ambitious musician. Unfortunately I don't think the "performance" of these cultured neurons will ever live up to his expectations, but I give him props for trying something novel. To summarize... Guy Ben-Ary took a sample of his skin cells, and with the help of a couple different biology labs induced skin cells into pluripotent stem cells, then differentiated those cells into neurons. The neurons were plated on a multi-electrode array dish, so electrical signals could be sent to the neurons, and received by an ephys amplifier / recording device. At first I thought he was attempting to train these neural networks to synthesize music, but it actually doesn't seem like that's being attempted. It looks like he's just sort of crudely prodding the neurons to fire (banging on a drum kit mic'd up with outputs to the multi-electrode array dish); at the same time he's amplifying the neural activity, and passing it through some filters in at attempt to make it sound as musical as 10,000 stochastically crackling neurons can be. Here's an example of what that sounds like: https://youtu.be/CWb_7mYLa6g https://youtu.be/CWb_7mYLa6g

subroutine 9y ago
continued... As a neurobiologist/electrophysiologist I suspected this random crackling is what it would sound like, since that's what activity from my cultured neurons sounds like. Any good neuro/psych student should remember this Nobel prize winning Hubel & Weisel experiment: https://youtu.be/8VdFf3egwfg https://youtu.be/8VdFf3egwfg I think this approach to synthesizing music from neural activity is completely misguided. But there are a few tweaks that might salvage the project... 1. First, they need to focus on the output of far fewer neurons. I know ya'll are experts in machine learning-type neural nets - the current "CellF" setup would be like sending the weights of an untrained 10k-node neural net to your laptop speakers, and just looping over those weights MCMC style. I think they would be better served to isolate the output a few neurons and translate those activation patterns into prefabricated musical samples. For example if the neuron fires at 60 Hz, the sample on MIDI channel 1 is activated (or something like that). continuing...
- subroutine 9y ago
  2. I get they are experimental musicians and composers chomping at the bit to finally play their finest arcane musique tintinnabulations. But they should first try to provide simple, structured musical inputs (repetitive drum taps would be the best, or square waves of particular frequencies; but at the very least simple repetitive melodies). At least until they can determine if these neurons are reacting in any predictable way to the microelectrode inputs. In one of the videos it basically looks like they mic'd up a petri dish on stage with a bunch of avant garde musicians working their hardest. That's actually pretty hilarious. Anyway, they should hold off on that until after they figure out whether these 'trans' neurons can even hear them. 3. They should try to train these neurons. Instead of microarray dishes though, I'd use optogenetics. From my experience it's much easier to evoke predictable signals using fiber optical input. It would be best to pass their instruments through a computer program though, that translated their sounds into pulse frequencies known to induce synaptic potentiation or synaptic depression. Currently, they are just adding noise to an already noisy system. All that said, this is still a cool project. And FWIW, they are much better neuroscientists than I am a musician.