4 ms·

As a developer with carpal and cubital tunnel syndrome, I'm extremely excited about the possibility of voice programming. However my experience was that it's s

As a developer with carpal and cubital tunnel syndrome, I'm extremely excited about the possibility of voice programming. However my experience was that it's somewhere between non-existent and impractical when I last tried it. It was actually the inspiration for my undergraduate research which explored (scratched the surface, really) the use of Lojban[0] as an "interactive metaprogramming language". The benefits of Lojban are that it is significantly more expressive than making sounds that are mapped to vim-like commands via a hacked natural language recognition engine, is isomorphic between text and speech (you can speak mathematical expressions with precision), is phonologically and grammatically unambiguous, and is standardized/has a community -- even if it's a small one. I still think it's a paradigm shift and would be indescribably more efficient and powerful than traditional code exploration and editing. And I would love to continue, but it's such a massive project and working on something that foreign alone for long enough is really isolating because it's hard to explain. Lojban also has its fair share of problems for this and would need to be further standardized and scoped out for computers, so there's really not a clear path forward in any case. Regardless, for those _currently_ suffering from hand and arm issues preventing them from coding and/or working, my advice is: 1) don't panic. Go on antidepressants and/or see a therapist if you need to; it's going to take a while to recover 2) rest as much as possible and make sure to get a lot of sleep 3) wear wrist braces to sleep to prevent further damage in the short term (consult a doctor and all that, you want to avoid muscle atrophy so don't use them for too long without starting some kind of muscle strengthening program) 4) invest in proper tools (standing desk + ergonomic keyboard is like magic for me, I can actually type again) 5) gain weight if you're on the low-side of normal weight -- this helped my cubital tunnel syndrome quite a bit by giving my ulnar nerve more protection And finally, don't give up hope; I'm able to work full time and don't wear wrist braces to sleep at all anymore after a little over a year. [0]: https://en.wikipedia.org/wiki/Lojban https://en.wikipedia.org/wiki/Lojban

DoingIsLearning 8y ago
Just as anecdotal evidence, I was surprised to find out that a Radiologist friend of mine spends the majority of his work day 'writing' MR reports by speaking to (from what I understood) some modded 'Dragon' natural speaking software. He also assured me this is pretty much the norm for all his colleagues as well. Although anecdotal, this is real-world evidence, that you can write something that is, highly technical, with a lot of domain-specific knowledge, to professional standard, via voice dictation. The question then becomes, is there a market, for example, for a "C++ aware" or "Python aware" modded voice application? How big would the market be? How much would people pay? Could it be covered by health insurance in an RSI case?
- maoeurk 8y ago
  That's a great point! There are quite a few specialized products for voice recognition and in fact domain specific voice recognition should surely be easier than trying to recognize general natural language. This is especially true, in the case of programming languages, because they tend to be so structured. There are a few major differences between most uses of voice recognition and programming though: The biggest issue is that most coding is not producing code; it's navigating and editing existing code in a non-linear way. I think that makes it extremely difficult because it needs to be integrated with static (or dynamic!) analysis of a codebase. And once we take that step, we should be able to turn programming from shuffling text around to a more conversational (not unlike REPL-based) development. There are also issues with the ability to pronounce things in some programming languages and being able to abstractly refer to these (often not linear) things. Because voice recognition is slower in terms of commands per second (as far as I've seen), we'd need more powerful and expressive ways of communicating intent to reach parity with keyboard-based programming. What I was hoping for with "interactive metaprogramming" was to be able to do project-wide queries and edits. It seemed that trying to find a voice-based equivalent to the way we program with text and keyboards was never going to be a great fit and that, really, we should have more powerful tools for interacting with code anyways. Lojban provides a lot of fertile ground for these ideas, I think. For example, Lojban has a really powerful way of talking about space, time, relative positions, anaphora, and meta-linguistic correction/negation. I can imagine that being incredibly useful for programming by voice. And I think the conversational style is key because the biggest issue with voice-based interfaces is presentation of relevant information. Lojban is based on relationships so it would be very natural to query the language itself with datalog-like evaluation, which provides an elegant solution to the hidden commands problem. edit: In regards to the price, I'd be willing to pay quite a lot. Losing the ability to program was one of the darkest times in my life. Soul-crushing, really. I was completely lost, unable to work and unable to enjoy myself. I remember when I thought $300-350 for a keyboard was expensive when looking at the Kinesis Advantage 2, but today, I'd be happy to pay 10 or 100 times the price knowing what it did for me. It's the difference between being able to work and enjoy myself (not being in constant pain from trying to work anyways) and laying around feeling hopeless and worthless.
singularity2001 8y ago
I can fully support advice 2) 4) An ergonomic keyboard will work wonders. 1) NEVER take meds that mess with your brain! That's such an American vice. 3) ? 5) yes, but building muscles is better than building fat (can be combined;)
- maoeurk 8y ago
  Those are fair points! I got too excited and didn't explain what I was talking about clearly -- so it isn't great general advice. To clarify for those who may need it: I think my case was a bit exceptional in how bad it got. I went from having no hand problems at all to having acute pain to the point where I couldn't function in daily life in a matter of weeks. Things such as working, driving, or opening doors with doorknobs became essentially impossible for me. I couldn't even sit in most chairs or sleep without my wrists being in constant pain. 2, 4, and 5 are the most useful because they're preventative. 3 is to deal with the physical trauma and 1 is do deal with the mental trauma. I agree with you that being on psychoactive drugs is not ideal, but when your life becomes nothing but sitting around in pain, I think that they're a reasonable short-term solution. Although non-hand based exercise is surely better if you can manage it, I found that running with my wrists taped up was fine and helped as much or more than the drugs, and I stopped the drugs after a few months when I felt more able to deal with it.
hjpimentel 8y ago
How long ago was this? It certainly is not impractical now. I wrote my PhD thesis, along with most of the software using VoiceCode.io. These days I’m on Talon. Dictation has basically been my sole key input device for three years. Like anything, it takes practice.
- maoeurk 8y ago
  About 2-3 years ago. Thank you! I had not seen either of these; I'll look into them and start a deep dive trying them out this weekend!

DoingIsLearning 8y ago

Just as anecdotal evidence, I was surprised to find out that a Radiologist friend of mine spends the majority of his work day 'writing' MR reports by speaking to (from what I understood) some modded 'Dragon' natural speaking software. He also assured me this is pretty much the norm for all his colleagues as well. Although anecdotal, this is real-world evidence, that you can write something that is, highly technical, with a lot of domain-specific knowledge, to professional standard, via voice dictation. The question then becomes, is there a market, for example, for a "C++ aware" or "Python aware" modded voice application? How big would the market be? How much would people pay? Could it be covered by health insurance in an RSI case?

maoeurk 8y ago
That's a great point! There are quite a few specialized products for voice recognition and in fact domain specific voice recognition should surely be easier than trying to recognize general natural language. This is especially true, in the case of programming languages, because they tend to be so structured. There are a few major differences between most uses of voice recognition and programming though: The biggest issue is that most coding is not producing code; it's navigating and editing existing code in a non-linear way. I think that makes it extremely difficult because it needs to be integrated with static (or dynamic!) analysis of a codebase. And once we take that step, we should be able to turn programming from shuffling text around to a more conversational (not unlike REPL-based) development. There are also issues with the ability to pronounce things in some programming languages and being able to abstractly refer to these (often not linear) things. Because voice recognition is slower in terms of commands per second (as far as I've seen), we'd need more powerful and expressive ways of communicating intent to reach parity with keyboard-based programming. What I was hoping for with "interactive metaprogramming" was to be able to do project-wide queries and edits. It seemed that trying to find a voice-based equivalent to the way we program with text and keyboards was never going to be a great fit and that, really, we should have more powerful tools for interacting with code anyways. Lojban provides a lot of fertile ground for these ideas, I think. For example, Lojban has a really powerful way of talking about space, time, relative positions, anaphora, and meta-linguistic correction/negation. I can imagine that being incredibly useful for programming by voice. And I think the conversational style is key because the biggest issue with voice-based interfaces is presentation of relevant information. Lojban is based on relationships so it would be very natural to query the language itself with datalog-like evaluation, which provides an elegant solution to the hidden commands problem. edit: In regards to the price, I'd be willing to pay quite a lot. Losing the ability to program was one of the darkest times in my life. Soul-crushing, really. I was completely lost, unable to work and unable to enjoy myself. I remember when I thought $300-350 for a keyboard was expensive when looking at the Kinesis Advantage 2, but today, I'd be happy to pay 10 or 100 times the price knowing what it did for me. It's the difference between being able to work and enjoy myself (not being in constant pain from trying to work anyways) and laying around feeling hopeless and worthless.

singularity2001 8y ago

I can fully support advice 2) 4) An ergonomic keyboard will work wonders. 1) NEVER take meds that mess with your brain! That's such an American vice. 3) ? 5) yes, but building muscles is better than building fat (can be combined;)

maoeurk 8y ago
Those are fair points! I got too excited and didn't explain what I was talking about clearly -- so it isn't great general advice. To clarify for those who may need it: I think my case was a bit exceptional in how bad it got. I went from having no hand problems at all to having acute pain to the point where I couldn't function in daily life in a matter of weeks. Things such as working, driving, or opening doors with doorknobs became essentially impossible for me. I couldn't even sit in most chairs or sleep without my wrists being in constant pain. 2, 4, and 5 are the most useful because they're preventative. 3 is to deal with the physical trauma and 1 is do deal with the mental trauma. I agree with you that being on psychoactive drugs is not ideal, but when your life becomes nothing but sitting around in pain, I think that they're a reasonable short-term solution. Although non-hand based exercise is surely better if you can manage it, I found that running with my wrists taped up was fine and helped as much or more than the drugs, and I stopped the drugs after a few months when I felt more able to deal with it.

hjpimentel 8y ago

How long ago was this? It certainly is not impractical now. I wrote my PhD thesis, along with most of the software using VoiceCode.io. These days I’m on Talon. Dictation has basically been my sole key input device for three years. Like anything, it takes practice.

maoeurk 8y ago
About 2-3 years ago. Thank you! I had not seen either of these; I'll look into them and start a deep dive trying them out this weekend!