4 ms·

Augmenting biological intelligence with RL in C.elegans using optogenetics [pdf]

113 points by signa11 2y ago

MrGuts 2y ago
"Know How, Can Do", by Michael Blumlein www.michaelblumlein.com/miscellany/knowhowcando_blumlein.pdf
- taneq 2y ago
  After the first couple of pages... is this Flowers for Algerworm? I'm gonna need to emotionally prepare myself if so. :P
- EMCymatics 2y ago
  404 error
hehdhdjehehegwv 2y ago
Sell NVIDIA, buy buckets of worms, profit.
- bugbuddy 2y ago
  Your next Apple will come with worms inside.
  - hehdhdjehehegwv 2y ago
    I guess that would make AppleCare a bucket of compost.
refibrillator 2y ago
There are some neat videos and images in the repo: https://github.com/ccli3896/RLWorms https://github.com/ccli3896/RLWorms Basically they took a worm and hijacked the genetic machinery such that a subset of neurons could be activated or inhibited via a light sensitive protein. Then they hooked up an unsupervised RL algorithm to a camera, and it learned how to control the lights to direct the worm towards a goal like food. The learned algorithms were able to navigate obstacles and generally enhanced the capability of the worm while coexisting with the remaining uncontrolled neurons.
- jb1991 2y ago
  > unsupervised RL algorithm I’m curious, is this an actual term? In machine learning you have three paradigms: supervised, unsupervised, and reinforcement learning. I never heard of combining two of these terms in this way before.
  - dr_kiszonka 2y ago
    It is a thing: https://masteringurlb.github.io/ https://masteringurlb.github.io/
    - jb1991 2y ago
      The term is not used in this article, however.
- lawlessone 2y ago
  > and it learned how to control the lights to direct the worm towards a goal like food. Oh good. biobots..
ramraj07 2y ago
Okay here are some background facts: 1. The worm is 1 mm long and few hundred microns across. 2. It has a fully characterized nervous system - 302 well defined neurons that have been mapped and named in exquisite detail. We even know which neurons connect to which! Have for decades. Still can’t tell how they function though. 3. The authors are expressing optogenetics proteins in various sets of neurons. Line 1 expresses them for example in 8 neurons known to play a role in chemotaxis (moves towards food). 4. They just shine a flash of light on the worm which would activate all these neurons together. This is probably still as blunt as putting an electrode into a mouse or monkey brain in some rough area and zapping them. I was hoping for slightly precise control. I’ve been waiting for optogenetics to enter worms - mainly because they’re like the cheapest and easiest to work with creatures and I always believed understanding the simplest organism first is the right way to understanding any complex system.
- bbor 2y ago
  Thanks for taking the time, very interesting! So the headline really isn’t clickbait. That’s… that’s incredible. I thought the models were optimizing some pre-birth (hatching?) interventions, but from your comment and the very first of the many figures, it sounds like the models were intervening realtime. This is the world’s first true digital cyborg, no? At least on a cognitive level rather than muscular/sensory/etc? We presented a hybrid system that used deep RL to interact with an animal's nervous system to achieve a task following a reward signal. Agents customized themselves to specific and diverse sites of neural integration, and the combined system retained the animal's ability to flexibly integrate information in new environments. FWIW the authors implications at the beginning and end about applying this to large mammals for its own sake seems dubiously motivated at best, and like something we should never encourage doing to other people. Literally hooking up your brain to remote processing would presumably mean you can longer tell what parts of your subconscious perceptions are due to your personality and perspectives and biases, and what parts are artificial… I’m not sure I could think of a closer technological parallel to “selling one’s soul to the devil”. You’d be consigning yourself to (potential? Inevitable?) ego death, in the long term. But obv those parts are included for rhetorical purposes, and likely encouraged by the powers that be (journal editors?) to generate interest among laymen and professionals both. So I’m absolutely not hating the players, here! This will no doubt be a critical tool for all sorts of studies going forward, though TBH I’m blanking on how they would be designed to get useful info from this. I guess “lesion organoid X and replace it with an ML network, train 1000 versions, and see what kinds of networks end up best replicating nominal behavior?”

113 points by signa11 2y ago

MrGuts 2y ago

"Know How, Can Do", by Michael Blumlein www.michaelblumlein.com/miscellany/knowhowcando_blumlein.pdf

taneq 2y ago
After the first couple of pages... is this Flowers for Algerworm? I'm gonna need to emotionally prepare myself if so. :P
EMCymatics 2y ago
404 error

hehdhdjehehegwv 2y ago

Sell NVIDIA, buy buckets of worms, profit.

bugbuddy 2y ago
Your next Apple will come with worms inside.
- hehdhdjehehegwv 2y ago
  I guess that would make AppleCare a bucket of compost.

refibrillator 2y ago

There are some neat videos and images in the repo: https://github.com/ccli3896/RLWorms https://github.com/ccli3896/RLWorms Basically they took a worm and hijacked the genetic machinery such that a subset of neurons could be activated or inhibited via a light sensitive protein. Then they hooked up an unsupervised RL algorithm to a camera, and it learned how to control the lights to direct the worm towards a goal like food. The learned algorithms were able to navigate obstacles and generally enhanced the capability of the worm while coexisting with the remaining uncontrolled neurons.

jb1991 2y ago
> unsupervised RL algorithm I’m curious, is this an actual term? In machine learning you have three paradigms: supervised, unsupervised, and reinforcement learning. I never heard of combining two of these terms in this way before.
- dr_kiszonka 2y ago
  It is a thing: https://masteringurlb.github.io/ https://masteringurlb.github.io/
  - jb1991 2y ago
    The term is not used in this article, however.
lawlessone 2y ago
> and it learned how to control the lights to direct the worm towards a goal like food. Oh good. biobots..

ramraj07 2y ago

Okay here are some background facts: 1. The worm is 1 mm long and few hundred microns across. 2. It has a fully characterized nervous system - 302 well defined neurons that have been mapped and named in exquisite detail. We even know which neurons connect to which! Have for decades. Still can’t tell how they function though. 3. The authors are expressing optogenetics proteins in various sets of neurons. Line 1 expresses them for example in 8 neurons known to play a role in chemotaxis (moves towards food). 4. They just shine a flash of light on the worm which would activate all these neurons together. This is probably still as blunt as putting an electrode into a mouse or monkey brain in some rough area and zapping them. I was hoping for slightly precise control. I’ve been waiting for optogenetics to enter worms - mainly because they’re like the cheapest and easiest to work with creatures and I always believed understanding the simplest organism first is the right way to understanding any complex system.

bbor 2y ago
Thanks for taking the time, very interesting! So the headline really isn’t clickbait. That’s… that’s incredible. I thought the models were optimizing some pre-birth (hatching?) interventions, but from your comment and the very first of the many figures, it sounds like the models were intervening realtime. This is the world’s first true digital cyborg, no? At least on a cognitive level rather than muscular/sensory/etc? We presented a hybrid system that used deep RL to interact with an animal's nervous system to achieve a task following a reward signal. Agents customized themselves to specific and diverse sites of neural integration, and the combined system retained the animal's ability to flexibly integrate information in new environments. FWIW the authors implications at the beginning and end about applying this to large mammals for its own sake seems dubiously motivated at best, and like something we should never encourage doing to other people. Literally hooking up your brain to remote processing would presumably mean you can longer tell what parts of your subconscious perceptions are due to your personality and perspectives and biases, and what parts are artificial… I’m not sure I could think of a closer technological parallel to “selling one’s soul to the devil”. You’d be consigning yourself to (potential? Inevitable?) ego death, in the long term. But obv those parts are included for rhetorical purposes, and likely encouraged by the powers that be (journal editors?) to generate interest among laymen and professionals both. So I’m absolutely not hating the players, here! This will no doubt be a critical tool for all sorts of studies going forward, though TBH I’m blanking on how they would be designed to get useful info from this. I guess “lesion organoid X and replace it with an ML network, train 1000 versions, and see what kinds of networks end up best replicating nominal behavior?”