7 ms·
I developed an RSI a few years ago, and I had to leave my job. It was devastating to lose both my career and my passion to an injury. Since then, I've been wor
by mwiethoff 7y ago
I developed an RSI a few years ago, and I had to leave my job. It was devastating to lose both my career and my passion to an injury.
Since then, I've been working with a friend on a new voice coding app called Serenade [1] that aims to enable anyone to program by voice. With Serenade, you can speak natural English voice commands like "delete second function" and "add class person". Not only can this be faster than typing, but it also means you don't have to memorize the syntax details of every language or a bunch of editor keyboard shortcuts.
We found that cloud speech APIs and programs like Dragon weren't accurate enough for common programming words, so we built a custom speech engine (based on Kaldi [2]) that's designed specifically for coding. The app is still early, but we think the future of programming is working with these higher-level inputs rather than typing out code entirely by hand.
We're looking for people to give feedback, so if anyone is interested in giving it a try, you can download Serenade at [3] or email me at matt@serenade.ai.
[1] https://serenade.ai https://serenade.ai
[2] http://kaldi-asr.org http://kaldi-asr.org
[3] https://serenade.ai/download https://serenade.ai/download
- hilyen 7y agoDefinitely interested. I’ll check it out. Same boat, software engineer, my index and middle finger main knuckles on both hands inflame from the slightest use even after years of therapy, doctors visits, etc. Also tendinitis in both elbows/arms. Note to those who are pushing off fixing your ergonomics, don’t. If you feel pain, numbing, tingling; do not ignore it. Listen to your body, don’t be like me.
- elevenoh 7y agoHad a similar issue. Then I went zerocarb and it faded fairly quickly.
- cheez 7y ago0 carb is the thing!
- throwaway61292 7y agoCan you elaborate on somethings you wish you had done. I get a bit of pain after half a day of coding, mostly my right pinky after hitting enter multiple times. Also get pain after certain activities like rolling food into a ball for my toddlers. I’ve joint pains here and there and the doctors have not yet found the cause and think it’s early onset of arthritis and have been told to take pain killers. I’m 39 by the way! Edit: added age
- LukeShu 7y agoI (25yo), also had pain in my right pinky and wrist, starting about 2 years ago. I spoke with a friend who had surgery for Carpel Tunnel Syndrome, and knew a lot more about this than me. What he told me: * Carpel Tunnel Syndrome affects the thumb/index-finger; pain in the pinky side of your hand could be caused by uthe ulnar tunnel or the cubital tunnel. * He does sets of 10 of these stretches every hour http://2.bp.blogspot.com/-cl-rpkmgPqI/UeUWdH4oMUI/AAAAAAAAAJw/lMj843282OE/s1600/CTS_BLOG.jpg http://2.bp.blogspot.com/-cl-rpkmgPqI/UeUWdH4oMUI/AAAAAAAAAJ... https://s-media-cache-ak0.pinimg.com/originals/8a/a4/2c/8aa42cc5c059cabf9aeb989914ca79ea.jpg https://s-media-cache-ak0.pinimg.com/originals/8a/a4/2c/8aa4... * "One of the biggest things you can do is splint your wrist at night. This is one of the best wrist braces I've found: https://www.amazon.com/Mueller-Fitted-Wrist-Brace-Number/dp/B002NLGNW8/ https://www.amazon.com/Mueller-Fitted-Wrist-Brace-Number/dp/... " My experience: - I got a foot pedal, and put Alt, Ctrl, Shift, and Enter on it (and for a while pulled those keycaps off the right side of my keyboard, to force myself to learn to use the foot pedal). This allowed me to keep using a computer while I recovered, and allowed me to avoid stressing my pinky after I recovered. I wrote a bit about that on Reddit https://old.reddit.com/r/emacs/comments/7remed/has_anybody_used_foot_pedals/dswhzkt/ https://old.reddit.com/r/emacs/comments/7remed/has_anybody_u... . - The tendon gliding exercises were very helpful. The other stretches didn't seem to make much difference. While I was recovering, I tried to do them hourly. I now only do them if it's acting up. - I second his endorsement of those wrist braces. When using them, I wear them at night, and when typing if I can (I can use a desk keyboard with them, but can't use my laptop with them). While I was recovering, I used them all the time, now I only use them if it's acting up. - More recently, I got a Keyboardio Model 01, which puts the modifiers on the thumbs, not the pinkies. The foot pedal is entirely unnecessary with it; it puts much less strain on my hands. http://2.bp.blogspot.com/-cl-rpkmgPqI/UeUWdH4oMUI/AAAAAAAAAJw/lMj84328 http://2.bp.blogspot.com/-cl-rpkmgPqI/UeUWdH4oMUI/AAAAAAAAAJ... Nowadays, my wrist acts up maybe once a month or two.
- bspammer 7y agoI'm 23 and have been using computers very heavily for the last 10 or so years - probably an average of 6 hours a day. During a particularly intensive day of typing last week I started getting pain in my right hand. I had to resort to hunt-and-peck index finger typing to get the feature I was working on finished. I'm horrified, it feels like I'm way too early in my career for something like this to happen, but I guess that's what happens when you do something so repetitive for tens of thousands of hours. What in particular have you found success with that helps? I'm kinda in denial right now, it's already gotten better over Saturday but I know I shouldn't be ignoring it.
- pknopf 7y agoI learned to use mouse with my other hand. I was literally forced to one day, out of pain. Took me 2 days to feel normal with it. Nowadays, I juggle back and forth, depending on comfort.
- digsy 7y agoAlso same boat. I've found that a mix of proper ergonomics, regular breaks, CBD oil (for inflammation), wrist braces and physical therapy exercises can do a lot of good.
- hilyen 7y agowhere do you get your cbd oil / what brand? Thanks
- throwaway3714 7y agoFor those in this thread, try mechanical keyboards without much actuation force, like Cherry MX clear. Also try programmable keyboards or custom keymap layouts, which will allow you to map Return to the caps lock key to balance RSI a bit. I have issues with my right index which I almost completely blame on Diablo 1/2, swapping left + right click helps a lot. I’ve also tried and wouldn’t recommend: Dvorak (no noticeable difference) Egrodox (extremely hard to learn) Maaaaybeee tiled window managers, dunno where I stand on that one but it probably doesn’t help a lot ergonomics wise.
- throwaway3714 7y agoI’ll also mention that cherry mx blue are much to heavy for me, and brown hurt after a week of furious typing. My problem with clears is that if I rest my hands too much I press keys because they’re so light
- melling 7y agoClears require more force than Browns? https://ergodox-ez.com/pages/keyswitches https://ergodox-ez.com/pages/keyswitches
- throwaway_391 7y agoGrantparent poster here. Hrm, you're right, I remember purchasing keys 'lighter' than browns, possibly silvers. They're very light, but I'd probably prefer 'too light' over 'too heavy'.
- Nyubis 7y agoYou may have Gateron Clears, which are lighter than Gateron Browns. Gateron Browns are basically equivalent to Cherry MX Browns, but Gateron Clears are entirely different from Cherry MX Clears. This has tripped me up more than once.
- bootlooped 7y agoI really like my ergodox. I can't go back to a normal keyboard. The only upgrade I can think of is going to a kinesis advantage or dactyl. It took about a week to get comfortable with mine, longer until I was better with it than a normal keyboard.
- jsjolen 7y agoIf what you have is actually tendinitis in your elbows then that's fixable through physical therapy (progressively heavier slow loading of the joint in the form of concentric-only single-joint movements).
- xtagon 7y agoThat's incredible! Does it work with any programming language?
- csomar 7y agoI gave it a shot. Here is my feedback: 1- It doesn't recgonize my voice very well. I know that my accent is not that good, but Google has around 9~% accuracy. This had maybe 10%! 2- Why a VSCode plugin? Why not make it at the OS level. So I can type with my voice anywhere I want. 3- Being able to customize the commands myself.
- mwiethoff 7y agoThanks for the feedback! We’re actively working on improving the speech engine, apologies it didn’t work well for you. We’re starting with editor plugins, but eventually want to expand to other apps as well. Definitely want to support customization down the line too.
- escapologybb 7y agoI just emailed you, this definitely looks promising!
- thrownaway954 7y agolittle constructive criticism... no where on the site or in the document does it mention what the supported languages are. After a little bit I saw that this was only for python, but I think it would be better for you cause if you not only said what languages are supported, but maybe also how you can add other languages.
- czechdeveloper 7y agoIt's directly on home page: Multiple languages. Use the same commands across multiple languages: Python, JavaScript, Java, and more on the way.
- navinsylvester 7y agoI was thinking along these lines. Wish i had time but good that you have made considerable progress. My motivation was different though. I wanted to break barrier to code, make cleaner/efficient/consistent code, automated tests ingestion and for the ability to port code to other languages from metadata. If you could focus also on those lines too - it would be useful for the general community too.
- RexM 7y agoIs this open source? I’d love to play with it and see what it would take to support C# in Visual Studio.
- Justsignedup 7y agoDoing the good work. It keeps me up at night thinking that I might be in this position. So little has to go wrong. I was actually VERY close to being in this same boat, one of my nerves in my shoulder was pinched so badly that it started to take damage. Took me a few months to regain full use of my left (dominant) hand with lots of physical therapy. Seeing this gives me real hope that there is hope for everyone regardless of a sudden life altering possibility.
- mattigames 7y agoJust today I heard about this guy with motor issues who can play competitive fighting games using only his tongue (not directly, using it to push his cheeks/skin) I would imagine combining something like that with voice recognition would be the best approach, something like press X before saying (on your mic) a variable name, or press Y before saying a control structure, such approach would remove a lot of ambiguity because the number of possible interpretations by the speech recognition software would be greatly reduced.
- cheez 7y agoEmacs programmer here, any chance it would be available there?
- Sir_Cmpwn 7y agoWhat's the licensing for this? I feel very strongly that accessibility technologies must be free/open source software, so that the already disadvantaged community which relies on them can work together on improving them to give each other a leg up, or to adapt it to their specific circumstances.
- shkkmo 7y agoThere is no license listed in the github repos for the VSCode and Atom plugins[0]. As such they are not currently open source. Open source seems like a major advantage with assistive technologies and something that the community should insist on in the products it uses. However, insisting that only free (as in beer) software is acceptable seems to unnecessarily restrict the potential new software that is needed. [0] https://github.com/serenadeai https://github.com/serenadeai
- Sir_Cmpwn 7y agoI meant free as in freedom in this case, sorry for the confusion.
- mortdeus 7y agoSounds like the foundation to building Jarvis.
- thinkersilver 7y agoThe snippets feature looks quite powerful and particularly like the enumeration conversion example. Are there any plans of making these kinda of transformations extensible? I've recently gone into speech assisted video editing and came across Kald about a month ago, was interested specifically in its diarization feature ... it's quite daunting. How did you get up to speed on it ... I've talked to a few people who've worked with it but it took them months to feel comfortable with it.
- goatforce5 7y agoI worked with a guy in 1999 who had severe RSI. He had spent a ton of time training whatever the highest pro version of Dragon Dictate was at the time, and then even more time setting up a bunch of macros to do complicated/repetitive tasks. It was impressive how fast he could 'type' (that is to say, it was really, really fast - faster than a keyboard) for the things he had trained it for. But, 20 years ago, none of the non-RSI people were tempted to follow his lead having seen how much work it took to get it set up. But, fast forwarding 20 years, I can totally imagine that a product such as yours that is designed from the ground up for programmer types could be significantly faster for text input, but hopefully without all of that burdensome training and setup before you can be productive.
- grogenaut 7y agoReading the docs it struck me that your not far off from a vim or emacs like abstraction around text processing. Hopefully you are building this so it's easy to add commands in other languages and not a ton of specialized code. thinking something like command + regex matcher... Or really if you can integrating with the language server spec you'd have a quite powerful system if you just have users learn the ast parts of that spec as commands. Didn't get to play with it as my computer died again right after I installed it. Not your fault, main ssd desoldered and cracked connector a week back while I'm in a month long road trip. I've been hobbling along on live USB distros (surprisingly fast) until last night when the boot USB torched. I have rsi flare-ups that cause my fingertips and blades of hands to go somewhat numb or hurt. Usually goes away when I use my good ergo setup (appeared when I played about 40 hours of pubg over a weekend) and get some exercise around my hands / forearms.
- hhas01 7y agoCurrently I’m in Language Design Hell trying to devise a syntax that is as easy to speak as it is to type, but if/when I get that cracked I’ll take a look as this looks very relevant to my interests. (Related: https://news.ycombinator.com/item?id=20674515 https://news.ycombinator.com/item?id=20674515)