7 ms·
Hacking my arm prosthesis to plug into a synth: thought-controlled music [video]
- ggggtez 7y agoVery cool, skip to the end to hear it in action.
- caymanjim 7y agoDon't skip to the end. It's not about the final product.
- berryg 7y agoIndeed, very cool. I kept thinking that with an Arduino he probably could have build a first prototype with first building a custom circuit. Just nitpicking.
- kregasaurusrex 7y agoIt's great that the representatives from the company sent him the relevant parts of the schematic to help him connect his arm to the interface board. Products aren't often built with accessibility with mind and seeing DIY projects like this inspire better solutions for these groups.
- pmoriarty 7y agoNot only that, but it must be incredibly rewarding to work on something that actually helps people. In addition, they got to work on something that's really cutting edge or even futuristic in a way that most technology work isn't. It's a huge win all around, and the world be a much better place if more people in tech did stuff like this than figuring out more effective ways of getting people to click ads or spy on them.
- 4NDR10D 7y agoGreat concept and interesting demo, however the title is pretty misleading. This isn't what I would call "thought controlled" more than the old Thalmic Myo or any EMG [0] is thought controlled. This is reading muscle signals with electrodes and translating those muscle activations into a variable voltage signal. One could argue that because you're not reading mechanical action it could be considered "thought control" but I'll leave that up to the individual to gauge for themselves. [0] https://techcrunch.com/2018/12/10/ctrl-labs-first-dev-kit-is-a-gesture-tracking-neural-controller/ https://techcrunch.com/2018/12/10/ctrl-labs-first-dev-kit-is...
- goldenkey 7y agoMost cheaper EEG headbands are misadvertised and actually measure skull muscle activation instead of actual brain signals, so there's that..
- aryik 7y agoHonestly one of the coolest things I’ve seen on HN. It’s a great display of a ton of things that have just recently (10-20 years) become accessible to the masses. 3D printing for the enclosure, the internet for the schematics, and advanced prosthetics to translate the muscle signal into voltage combined with tried and true technologies like relatively simple circuit design and modular synths. Thanks for sharing!
- mhh__ 7y agoThe thing that's really struck me (which is potentially saying a lot - or nothing - because I'm still young) is how cheap PCB Fabrication (and Assembly!) at a hobbyist level has gotten recently e.g. An 8-layer ENIG, double silkscreen, a few gold fingers for $40 to $50 shipped is pretty damn cheap Think of all the projects to never get round to making!
- takeda 7y agoI don't think making PCB was ever expensive, perhaps making a professionally looking one. In old times you had a piece of plastic with a copper layer. You put wax on it, then you could use a needle to remove wax in right places (you basically print the inverse of the circuit) Place that PCB in acid, and after the copper gets dissolved you clean it up, and have printed PCB. There are many techniques people have, and modern ones involves printers, but you kind of always could do one at low cost. It might just not look that nice.
- kube-system 7y agoDoing a complex 8 layer board would have been traditionally expensive... or nonexistent, depending on how long ago we’re talking. DIY accessible stuff is usually single layer or dual layer through-hole stuff.
- takeda 7y agomy bad, now after reading your comment I noticed the 8 layer. Yeah I meant only simple PCB.
- cronix 7y agoThis is really fascinating to me. I've often wondered what exactly makes an artist/musician. Is it the person who can create it in their head, or the person who can actually write it down and/or actually play it on an instrument? Up until now, it's always been the latter. Not everybody can read/write music or play an actual instrument, but most can create it in their head. Like, I'm sure most of you can create a simple tune and hum it, and don't know how to read/write music or play an instrument. This kind of thing would allow you to. I wonder how many hidden artists throughout time have created brilliant masterpieces, but just could never express it.
- pmoriarty 7y agoI'd encourage you to read Brian Eno's "Composers as Gardeners"[1], which talks about a different way of looking at composition, which is pretty different from the usual way people think about it. Many non-musicians are under the impression that composers think up music in their head and then write it down, and while some do work that way to some extent (with they myth of Mozart working like that in the movie Amadeus being probably the most well known and most extreme case of composition being completely like that), for most composers (including Mozart) this is much more of an iterative, experimental process. In recent decades, and especially with the advent of electronic music and computers, a lot more randomness and elements not fully or directly under the composer's control have entered in (though, again, this goes back at least back to the age of Mozart, with his dice game[2]). There's a whole field of algorithmic composition[3] where music-generation becomes either semi-automated or even completely automated, and the composer's role is one of writing or choosing the algorithms, and of choosing various other aspects of the music (such as which instruments or sounds get used) but without pre-determining the final work in totality or in advance. Brian Eno talks about this much more eloquently and deeply than I can, so I again encourage you to read his talk. [1] - https://www.edge.org/conversation/composers-as-gardeners https://www.edge.org/conversation/composers-as-gardeners [2] - https://en.wikipedia.org/wiki/Musikalisches_W%C3%BCrfelspiel https://en.wikipedia.org/wiki/Musikalisches_W%C3%BCrfelspiel [3] - https://en.wikipedia.org/wiki/Algorithmic_composition https://en.wikipedia.org/wiki/Algorithmic_composition
- enisberk 7y agoIt is really cool, indeed. Bravo for the creativity and the effort. Facebook recently acquired a company called ctrl-labs that develops a neural interface. Their vision is making computer interface more natural. Bertolt, in the video says “The thing is for me, that is such a natural thing to do, I do not really have to think about it. I just do it, it is zero effort.”(6.23) I love that, I hope we can have such computer interfaces in near future.
- Thorentis 7y agoThis is incredible. I think the next step from here - technology wise I mean - is figuring out how devices can tap into this without needing to lose a limb. Implants that can read the signals from a limb to control something else could be the bridge in between what we have now, and brain-computer interfaces. You know those devices that severely disabled people like the late Stephen Hawking used to interact with his computer? They require only a very small amount of muscle movement to control a switch etc. What if an implant could be put directly into a muscle or a group of muscles (or a a nerve ending somewhere more suitable) so that a tiny movement could trigger some action? It would essentially give humans a set of re-programmable multi-function buttons. Connect it to your phone over bluetooth, and you can setup an action to be triggered by it. How would you filter out regular/involuntary input vs. intentional input? Not sure! But this avenue of development is exciting.
- wefarrell 7y agoAlready exists: https://spectrum.ieee.org/the-human-os/biomedical/bionics/mindcontrolled-robotic-third-arm-gives-new-meaning-to-multitasking https://spectrum.ieee.org/the-human-os/biomedical/bionics/mi...
- pmoriarty 7y agoAnother cool device, which is not an arm and is for input rather than control, is one that lets people see through their tongues.[1] It's currently targeted at the blind, but I'd imagine seeing people could use it just as well if it picked up, say, infrared, ultraviolet, x-rays, radio waves, ionizing radiation or virtually anything else normally undetectable to human senses. The potential of human sensory enhancement is truly staggering. [1] - https://www.newyorker.com/magazine/2017/05/15/seeing-with-your-tongue https://www.newyorker.com/magazine/2017/05/15/seeing-with-yo...
- emiliobumachar 7y agoIf it was cheap and easy enough, I'd seriously try it as a sighted person, to read while resting my eyes, which routinely get strained by the end of the day. Possibly decoupled from the camera and getting the signal from an adapter which itself gets text. https://www.wicab.com/brainport-vision-pro https://www.wicab.com/brainport-vision-pro "Ready to buy or try a BrainPort Vision Pro? Fill out the patient survey here to determine if you are a good candidate. A Certified Trainer will contact you to discuss the next steps!" Oh, never mind... I got from the quoted text that it would probably be prohibitively expensive, and that if I tried to find out more I'd probably divert professionals from helping actual blind people in need of this tech .
- daenz 7y agobrb cutting off my arm
- sachdevap 7y agoI know this is meant to be humor, but realistically, as I understand it, you can just tap into the muscular signals with electrodes - that does not need any such dramatic step.
- pmoriarty 7y agoIn case anyone's wondering what that cool synthesizer that he's using is, it's a Eurorack[1] modular synthesizer[2]. If you are at all interested in this, I'd strongly encourage you to check out Muffwiggler[3], a huge forum for modular synthesizer enthusiasts. Also useful is the Modular Grid[4], a gigantic database of thousands of amazing modules (and which also lets you design your own modular setup using these modules). VCV Rack[4] is also great for playing around with software versions of some of these modules, which can be pretty expensive (the guy in the video has likely invested thousands and thousands of dollars in to his Eurorack setup, while VCV Rack is free). On the other hand, you can also build your own Eurorack modules (see the DIY forum on Muffwiggler for that), which can be a lot cheaper. Something else I was thinking about was that if the main problem the guy in the video was trying to solve was that the knobs and switches on his modules were too difficult to manipulate with his prosthesis, then he could have tried the HackMe Vectr[5][6], which would have let him simply wave his whole hand around above the module to send out three voltages from his hand movements in three dimensional space. Mind control is, of course, way cooler, and doesn't require movement at all. Finally, I was also thinking about how his modular interface might have been designed with opto-isolators, to minimize the risk of voltage inadvertently being sent back from the modular in to his arm, should he plug his cable in to an output rather than an input. Someone with more of an electronics background than me can probably comment on whether this would be a good idea. [1] - https://en.wikipedia.org/wiki/Eurorack https://en.wikipedia.org/wiki/Eurorack [2] - https://en.wikipedia.org/wiki/Modular_synthesizer https://en.wikipedia.org/wiki/Modular_synthesizer [3] - https://muffwiggler.com/forum/ https://muffwiggler.com/forum/ [4] - https://vcvrack.com/ https://vcvrack.com/ [5] - https://www.youtube.com/watch?v=HSoVBz-1eQ0 https://www.youtube.com/watch?v=HSoVBz-1eQ0 [6] - http://hackmeopen.com/Vectr/ http://hackmeopen.com/Vectr/
- mxmilkb 7y agoSee also the JACK audio and LV2 plugin form of CV - https://llllllll.co/t/jack-lv2-cv-ports/29428 https://llllllll.co/t/jack-lv2-cv-ports/29428
- tomduncalf 7y agoVery cool, I didn’t know this existed! Is it Linux only?
- caymanjim 7y agoThe thing that caught me the most about this video has nothing to do with the hack (which is awesome). What impressed me was how natural the prosthesis is, and how naturally he moves with it. Specifically, he grasps his non-prosthetic wrist with his prosthetic hand, casually, while he's talking. It shows how he's able to use the prosthesis without thinking about it. That and the fact that the prosthesis has the same dimensions as a natural limb, and doesn't require any bulky power attachment. I'm guessing this is standard now, or at least not experimental. I didn't realize prosthetics had advanced this far.
- pmoriarty 7y agoAnother interesting video along these lines: [1], of a girl with two proshetic arms. [1] - https://www.youtube.com/watch?v=4eMQuhxNDJM https://www.youtube.com/watch?v=4eMQuhxNDJM
- rad_gruchalski 7y agoThis is incredible, indeed. And must be well understood technology by now. There is for example this man who build prosthetics with lego: https://www.reuters.com/article/us-spain-lego-prosthetic-arm/brick-by-lego-brick-teen-builds-his-own-prosthetic-arm-idUSKCN1PW155 https://www.reuters.com/article/us-spain-lego-prosthetic-arm....
- holri 7y ago"the fact that the prosthesis has the same dimensions as a natural limb, and doesn't require any bulky power attachment." This is standard in arm prothesis at least since the 1960ies.
- Ididntdothis 7y agoI think there is a big advantage in having some engineering/hacking minded people being the users of medical devices like this arm prosthesis. I work in medical devices and I always think how good it would be to have test patients who have a deep understanding of our devices. I am sure progress could be accelerated a lot if you had patients who can give better feedback. I know it’s not FDA approved but if I had device myself I certainly would play with the programming. Obviously the level of experimentation depends on the device. If a malfunction can do serious harm you probably don’t want to experiment much. But there are a lot of devices that won’t kill you if something goes wrong.
- liveoneggs 7y agoreal user feedback? do you live in la-la land or something? We're going to need an outside agency, a giant game of telephone, and a bunch of contrived questionnaires to spend that kind of money.
- econcon 7y agoI've always wondered why we've learn a specific music instrument and I say this after learning flute (simple bamboo one I bought from India) Even without music theory or knowledge of specific instrument we are able to hum or vocally replicate a piece of music - why is it not possible to directly transform it into some instrument? Is it not possible to hum > grab the pattern > generate instrument music with that? I mean i can articulate smth best with my voice which is with me since I've learned to generate voice on my own so why we don't exploit it with help of some software?
- esistgut 7y agoI had a similar thought until I actually started learning an instrument and my SO, flutist graduated from an Italian conservatory, made me start singing with the right intonation (singing the same notes you are playing on the instrument) and rhythm (division and subdivision alongside a metronome). Long story short: singing is hard and requires real effort. There is a good chance that what you think you can articulate is probably far away from minimal quality on both pitch and time.
- ddingus 7y agoWhile that's true enough for many people, adding a feedback step and some practice would improve matters quickly. There is also a good chance someone can learn to encode quite a bit into vocalizations.
- holri 7y ago"some practice would improve matters quickly" Anyone who seriously learned an instrument or to sing, knows that it takes very hard practice for years to improve.
- ddingus 7y agoI have done that myself. Both vocal and instrument. Yes, but vocalizations may well improve more quickly. Frankly, I have toyed with this kind of thing. Software could do a lot, in particular, allow the user to adapt their expressions in various ways.
- kristianov 7y agoInitially I was wondering why his prosthesis has ARM processor...
- allovernow 7y agoThat's awesome. I figure its only a matter of time before I'll be hard pressed not to make the difficult decision of giving up my arms for the eventual advantage of a prosthetic, and the seamless way he is able to control the synth brings that horizon much closer. Combine this with a little ML and you could probably have a general purpose interface trainable in a few seconds/minutes to interact with any suitable output based on your personal signals! Imagine piloting a vehicle without moving a muscle, or playing a video game, or interacting with someone remotely... Hell, now that I think of it, there's no reason you have to cut off your arms to do so. Is anyone aware of any open source prosthetic tech/code that one could conceivably hook up to functioning appendages?
- rntksi 7y agoDid this remind anybody of Exapunks? It's a game that has a defined assembly-like language, you hack into things to do other things. The catch is some of those things you hack into are actually your nerves, so that you can still function properly. Pretty cool game by Zachtronics
- yayitswei 7y agoIf anyone owns a Neurosky (brain wave reader that fits on your head) and wants to try out something similar, you can try a simple demo that I wrote: https://github.com/yayitswei/hairtime https://github.com/yayitswei/hairtime