3 ms·

I would like to see the middleware between sensors and motors be open source. To me, it seems like the complexity in prosthetics is not the mechanics, but inte

I would like to see the middleware between sensors and motors be open source. To me, it seems like the complexity in prosthetics is not the mechanics, but interpreting the signals properly to control the device. It seems like a relatively rudimentary neural net with a lot more electrical sensors than today (maybe 16 to 256, something on that order) combined with some of the new sensors like light transmission or even maybe ultrasound to "see" the muscles under the skin, could be mapped to a graph with roughly the same degrees of freedom as the human hand. Then maybe something like KNN could sift the data and map it to the motors. I’m not very familiar with data mining yet, but surely a lot could be done with a 700 MHz Raspberry Pie and a few GB of SSD storage. I’m thinking that giving the algorithm some notion of time by recording 1/60 of a second slices or something in that range might really improve accuracy. Just throw data and processing at it rather than trying to fully understand how nerves work. Doing the reverse, to stimulate nerves based on how much power each motor is using, should be simpler. It could be sloppier because the human mind would learn to interpret the signals better than a computer. The software should be available for free online and work with commodity hardware, and then the prosthetic designer could customize the limb for each user. Plus it could have the ability to upload random samples to help train the next generation AI. As long as I am dreaming, we also need a moonshot to standardize nerve implants. It would go inside the limb somewhere and then use something like 802.11 powered by inductance coils above and beneath the skin, with a faraday cage built into the arm to prevent interference. Actually now that I say that, I wonder if the noise produced by the body’s movements would be statistically significant enough for another input into a neural net. Anyway, chemical markers on the terminals could urge the nerves to grow into the terminals. Or use a bit of foreign nerve tissue that’s been grown on a scaffolding in the device in the lab, and then implant it and put the patient on immune suppression drugs until the nerves grow together, and then take them off gradually until the body dissolves the foreign nerves and has a direct connection to the sensor. Another idea would be to press a comb-shaped device with hundreds of microscopic channels through the nerve bundle and read the voltage through each channel. It should be manufactured for longevity and to read the most amount of data possible, and just let the programmers work out how to sift through it. Once we had good data, training a neural net to do the mapping to actuators seems like a basic problem. Like maybe sample a few hours’ worth of movements and put the whole thing online like the Netflix contest until someone has programmed a near-perfect AI. Or a series of contests, each aiming to raise accuracy by 10% or something.

lelf 13y ago
> a lot more electrical sensors than today (maybe 16 to 256, something on that order) And you will basically have brain-controlled port. Then emulating normal(-ish) human hand becomes kinda boring thing to do. Because you can use a “different” hand. Or plug in into music synthesiser. Endless possibilities. > Once we had good data, training a neural net to do the mapping to actuators seems like a basic problem You don't strictly need it. Human brain is pretty ok with learning all kind of weird stuff.
- klipt 13y ago
  > You don't strictly need it. Human brain is pretty ok with learning all kind of weird stuff. Sure brains are plastic, but learning takes time so if you can map nerve outputs to hand in a way that generalizes across people, they could start using it faster.
Balgair 13y ago
Ok, wow, this is my area of expertise actually (well, in process at least). Lets go through your post and comb it out. I really love the enthusiasm and ideas. My biggest caution going forward though: The body really dislikes foreign objects, like really. >"...it seems like the complexity in prosthetic is not the mechanics, but interpreting the signals properly to control the device." You couldn't be more correct! Dr. Richard Weir at CU Denver[0] has some great work in 3-D printed hands. The lack of controls into the hands is a major concern. The issue is you only have so many muscles in the arm to control all the complex motions of a hand. His group is working on the eigen-states of hand movement, trying to decompose them into the number of muscles in your arm. Why the arm? Well, it makes sense to focus there, as that is where patients' minds are focused. The method is myocardial sensors that sense the nerve firings in the peripheral nervous system and then can send out that data. You basically implant them in a muscle fiber and they last a long time. >... a lot more electrical sensors than today (maybe 16 to 256, something on that order)... This would be awesome. But, focusing on a prosthetic hand, we don't have that many muscles in the arm to control that many[1]. The hand is much more complicated than the arm. Decomposing gestures, as Dr. Weir is doing now is a good step in that direction. >...Just throw data and processing at it rather than trying to fully understand how nerves work. How true this is! Much is being done in the Big Data era of neuroscience that we live in. Many groups are trying this tactic[2]. The thought is to just describe what the system of a brain does, not how it works. Think Gaussians, not Markov chains. >...Doing the reverse, to stimulate nerves based on how much power each motor is using, should be simpler. Feedback is very important to prostheses and helps improve care and quality of life. However, directly stimulating nerves is difficult. Your fingertips have much better 'resolution' than your elbow does[3]. Also, your skin also becomes accustomed to the input and you will ignore that stimulus pretty quickly. Like how you notice the smell of the food when you walk in the door, but don't within 10 minutes of standing next to the stove. >...It could be sloppier because the human mind would learn to interpret the signals better than a computer. The body does acclimate to the stimuli, but as I said before, it also starts to ignore as well. Also, people are all different. Genes can account for a lot of the differences, as well as environment, but also even culture and gender[4] >...As long as I am dreaming, we also need a moonshot to standardize nerve implants. I'm not sure what you mean here. Each person's situation is very different and personal. Standardizing is difficult with medicine.. Skin color, size, weight, culture, gender, job, etc all greatly vary. For example, a scuba instructor's situation is a lot different than a 1st grade teacher's. The scuba guy has to deal with greatly varying pressure and water. The teacher has to have the new hand be fit for children. The nerve implants will have to follow their situations. >...a faraday cage built into the arm to prevent interference. Your blood is mainly salt water, which is very conductive, so there is not real need to cage things. In fact, you want to encase everything in silicone to prevent leaks and make it all non-bioreactive. >...the noise produced by the body’s movements would be statistically significant enough for another input into a neural net. The noise in a muscle is hard to figure out. Again, it depends on the person, the muscles, the salt content in the blood, etc. In a nerve, the picture is much the same. I will quote without citation that the firing rates of nerves increase with temperature to the 4th power (I'd love a citation, if anyone can find one, my brain is just borking now) >...Anyway, chemical markers on the terminals could urge the nerves to grow into the terminals. This is non-trivial and may not be possible. Those markers often are only present in a real soup, and the behavior of nerve growth is not well understood (at least by me, haha). I'll admit it's possible, but a lot of engineering is needed to do it. >...put the patient on immune suppression drugs until the nerves grow together, Wow, that is not a good idea. Going on immunosuppressants is not taken lightly, and you'd have a heck of a time convincing people to take that big of a risk just for a hand or something. Also, they won't be necessary in the peripheral nervous system, just the central nervous system. >...Another idea would be to press a comb-shaped device with hundreds of microscopic channels through the nerve bundle and read the voltage through each channel. Dr. Theodore Berger at USC is working on a device like this already![5] >...It should be manufactured for longevity and to read the most amount of data possible, and just let the programmers work out how to sift through it. If you'd like to join up, many neuroscience departments really favor physics, CS, and EE majors for grad study these days, at least it seems to me like they are. The peripheral nervous system is a real good area for these neural ideas. But the central nervous system (CNS) is not. The reason is glial scarring[6]. You body really really does not like things in the CNS. This is why paralysis happens mainly. The scar prevents the nerves in your spine from regrowing and making the legs and whatnot work again (sort-of). This takes place in your brain to a huge degree. I used to work with place cells[7](very cool cells) and everyday we would have to 'burn off' the scarring by passing current through the electrode bundle. It turns out, you are your own worst enemy here. Ideas like opto-genetics[8] can bypass the scarring issues, but they also have a host of others. Again, great ides and enthusiasm! I love it! If you see a future with neuroscience, the field is wide open these days and there seems to be a bit of money floating around. Apply to a progam or 12 and you'll have a great time. [0]http://www.ucdenver.edu/about/newsroom/newsreleases/Pages/Bioengineer-developing-prosthetic-hand-and-fingers-that-perform-full-range-movement.aspx http://www.ucdenver.edu/about/newsroom/newsreleases/Pages/Bi... [1]http://www.innerbody.com/anatomy/muscular/arm-hand http://www.innerbody.com/anatomy/muscular/arm-hand [2]It's easter, I'm not counting how many! You do it. [3]http://visalakshiramani.files.wordpress.com/2010/07/homunculus.png http://visalakshiramani.files.wordpress.com/2010/07/homuncul... [4]http://ccr.sagepub.com/content/34/2/135.abstract http://ccr.sagepub.com/content/34/2/135.abstract [5]http://www.usc.edu/programs/neuroscience/faculty/profile.php?fid=23 http://www.usc.edu/programs/neuroscience/faculty/profile.php... [6]https://en.wikipedia.org/wiki/Glial_scar https://en.wikipedia.org/wiki/Glial_scar [7]https://en.wikipedia.org/wiki/Place_cell https://en.wikipedia.org/wiki/Place_cell [8]https://en.wikipedia.org/wiki/Glial_scar https://en.wikipedia.org/wiki/Glial_scar

lelf 13y ago

> a lot more electrical sensors than today (maybe 16 to 256, something on that order) And you will basically have brain-controlled port. Then emulating normal(-ish) human hand becomes kinda boring thing to do. Because you can use a “different” hand. Or plug in into music synthesiser. Endless possibilities. > Once we had good data, training a neural net to do the mapping to actuators seems like a basic problem You don't strictly need it. Human brain is pretty ok with learning all kind of weird stuff.

klipt 13y ago
> You don't strictly need it. Human brain is pretty ok with learning all kind of weird stuff. Sure brains are plastic, but learning takes time so if you can map nerve outputs to hand in a way that generalizes across people, they could start using it faster.

Balgair 13y ago

Ok, wow, this is my area of expertise actually (well, in process at least). Lets go through your post and comb it out. I really love the enthusiasm and ideas. My biggest caution going forward though: The body really dislikes foreign objects, like really. >"...it seems like the complexity in prosthetic is not the mechanics, but interpreting the signals properly to control the device." You couldn't be more correct! Dr. Richard Weir at CU Denver[0] has some great work in 3-D printed hands. The lack of controls into the hands is a major concern. The issue is you only have so many muscles in the arm to control all the complex motions of a hand. His group is working on the eigen-states of hand movement, trying to decompose them into the number of muscles in your arm. Why the arm? Well, it makes sense to focus there, as that is where patients' minds are focused. The method is myocardial sensors that sense the nerve firings in the peripheral nervous system and then can send out that data. You basically implant them in a muscle fiber and they last a long time. >... a lot more electrical sensors than today (maybe 16 to 256, something on that order)... This would be awesome. But, focusing on a prosthetic hand, we don't have that many muscles in the arm to control that many[1]. The hand is much more complicated than the arm. Decomposing gestures, as Dr. Weir is doing now is a good step in that direction. >...Just throw data and processing at it rather than trying to fully understand how nerves work. How true this is! Much is being done in the Big Data era of neuroscience that we live in. Many groups are trying this tactic[2]. The thought is to just describe what the system of a brain does, not how it works. Think Gaussians, not Markov chains. >...Doing the reverse, to stimulate nerves based on how much power each motor is using, should be simpler. Feedback is very important to prostheses and helps improve care and quality of life. However, directly stimulating nerves is difficult. Your fingertips have much better 'resolution' than your elbow does[3]. Also, your skin also becomes accustomed to the input and you will ignore that stimulus pretty quickly. Like how you notice the smell of the food when you walk in the door, but don't within 10 minutes of standing next to the stove. >...It could be sloppier because the human mind would learn to interpret the signals better than a computer. The body does acclimate to the stimuli, but as I said before, it also starts to ignore as well. Also, people are all different. Genes can account for a lot of the differences, as well as environment, but also even culture and gender[4] >...As long as I am dreaming, we also need a moonshot to standardize nerve implants. I'm not sure what you mean here. Each person's situation is very different and personal. Standardizing is difficult with medicine.. Skin color, size, weight, culture, gender, job, etc all greatly vary. For example, a scuba instructor's situation is a lot different than a 1st grade teacher's. The scuba guy has to deal with greatly varying pressure and water. The teacher has to have the new hand be fit for children. The nerve implants will have to follow their situations. >...a faraday cage built into the arm to prevent interference. Your blood is mainly salt water, which is very conductive, so there is not real need to cage things. In fact, you want to encase everything in silicone to prevent leaks and make it all non-bioreactive. >...the noise produced by the body’s movements would be statistically significant enough for another input into a neural net. The noise in a muscle is hard to figure out. Again, it depends on the person, the muscles, the salt content in the blood, etc. In a nerve, the picture is much the same. I will quote without citation that the firing rates of nerves increase with temperature to the 4th power (I'd love a citation, if anyone can find one, my brain is just borking now) >...Anyway, chemical markers on the terminals could urge the nerves to grow into the terminals. This is non-trivial and may not be possible. Those markers often are only present in a real soup, and the behavior of nerve growth is not well understood (at least by me, haha). I'll admit it's possible, but a lot of engineering is needed to do it. >...put the patient on immune suppression drugs until the nerves grow together, Wow, that is not a good idea. Going on immunosuppressants is not taken lightly, and you'd have a heck of a time convincing people to take that big of a risk just for a hand or something. Also, they won't be necessary in the peripheral nervous system, just the central nervous system. >...Another idea would be to press a comb-shaped device with hundreds of microscopic channels through the nerve bundle and read the voltage through each channel. Dr. Theodore Berger at USC is working on a device like this already![5] >...It should be manufactured for longevity and to read the most amount of data possible, and just let the programmers work out how to sift through it. If you'd like to join up, many neuroscience departments really favor physics, CS, and EE majors for grad study these days, at least it seems to me like they are. The peripheral nervous system is a real good area for these neural ideas. But the central nervous system (CNS) is not. The reason is glial scarring[6]. You body really really does not like things in the CNS. This is why paralysis happens mainly. The scar prevents the nerves in your spine from regrowing and making the legs and whatnot work again (sort-of). This takes place in your brain to a huge degree. I used to work with place cells[7](very cool cells) and everyday we would have to 'burn off' the scarring by passing current through the electrode bundle. It turns out, you are your own worst enemy here. Ideas like opto-genetics[8] can bypass the scarring issues, but they also have a host of others. Again, great ides and enthusiasm! I love it! If you see a future with neuroscience, the field is wide open these days and there seems to be a bit of money floating around. Apply to a progam or 12 and you'll have a great time. [0]http://www.ucdenver.edu/about/newsroom/newsreleases/Pages/Bioengineer-developing-prosthetic-hand-and-fingers-that-perform-full-range-movement.aspx http://www.ucdenver.edu/about/newsroom/newsreleases/Pages/Bi... [1]http://www.innerbody.com/anatomy/muscular/arm-hand http://www.innerbody.com/anatomy/muscular/arm-hand [2]It's easter, I'm not counting how many! You do it. [3]http://visalakshiramani.files.wordpress.com/2010/07/homunculus.png http://visalakshiramani.files.wordpress.com/2010/07/homuncul... [4]http://ccr.sagepub.com/content/34/2/135.abstract http://ccr.sagepub.com/content/34/2/135.abstract [5]http://www.usc.edu/programs/neuroscience/faculty/profile.php?fid=23 http://www.usc.edu/programs/neuroscience/faculty/profile.php... [6]https://en.wikipedia.org/wiki/Glial_scar https://en.wikipedia.org/wiki/Glial_scar [7]https://en.wikipedia.org/wiki/Place_cell https://en.wikipedia.org/wiki/Place_cell [8]https://en.wikipedia.org/wiki/Glial_scar https://en.wikipedia.org/wiki/Glial_scar