17 ms·

On Voice Coding

316 points by buchuki 7y ago

melling 7y ago
“I type fast enough that I have never had much use for snippets in my text editor. Perhaps if I’d used them more effectively, I would not have developed the RSI symptoms that I have!" Often we have discussions about advanced code completion on HN. Many developers feel they don’t need it, or that it gets in their way, for example. Reading stories like this convinces me even more that our editors (tools) need to be smarter. There is so much repetition is coding, it’s hard to believe we can’t do better. This tool is often mentioned: https://tabnine.com/ https://tabnine.com/
- busterarm 7y ago
  I've been typing just about 4-16 hours a day going on 30 years now without RSI. Fingers crossed.
  - melling 7y ago
    Isn’t that sort of like saying, I smoked 2 packs of cigarettes for 30 years, fingers crossed? Some people are more susceptible to problems. This was on HN several years ago. http://www.looknohands.me/ http://www.looknohands.me/ You can be in your 20’s and have a problem.
    - busterarm 7y ago
      I also worked with someone in a remote capacity who insisted on using Dragon Naturally Speaking and his communication on technical topics was unintelligible. It clearly wasn't working, but he insisted and everyone who worked with him suffered for it. And now he's not working in tech anymore and is a field engineer doing repetitive labor with his hands...
    - magicalhippo 7y ago
      I actively try to switch things up, depending on if I feel something is starting to get uncomfortable. If my mouse hand starts to give signals, I'll switch to using keyboard more. Regardless I'll switch seating position and hand orientation throughout the day. So in that respect it's unlike smoking I guess.
- keymone 7y ago
  We should use better languages. Better understood and integrated with the editor and better at expressing every level of abstraction. coughpretty much any lispcough
  - ulisesrmzroche 7y ago
    Or you could take a short break every so often What we really need are better programmers
    - keymone 7y ago
      Tools matter. First ascent of Eiger’s north face took few days and team of 4. Last I heard Ueli Steck did it few years ago solo in less than 2 hours. Maybe he was better but not by that much. His tools were unquestionably better.
      - shrimp_emoji 7y ago
        If better tools also make "worse" programmers (i.e., people saner and happier and uninitiated with the brain death rigors of struggling with awful/no tools), tools also matter more the better they get.
        keymone 7y ago
        If a tool makes worse programmer, it’s not better I would say.
- lunixbochs 7y ago
  Deep TabNine looks so good it’s downright spooky: https://tabnine.com/blog/deep https://tabnine.com/blog/deep
  - singularity2001 7y ago
    This will result in a whole new class of copy paste bugs. Great anyways
- est 7y ago
  I found that Kite is comparable with Tabnine, at least with Python. https://kite.com/integrations/kite-vs-tabnine/ https://kite.com/integrations/kite-vs-tabnine/
  - Gaelan 7y ago
    Is Kite's behavior still as shady as it used to be?
  - lunixbochs 7y ago
    Because this says “single token completion” as the tabnine example gif I’m guessing it isn’t comparing to deep tabnine at all, which is on an entirely different level
  - 90litshi99 7y ago
    the memory usage comparison is accurate for tabnine, at least in my case. local-only tab nine wasn't practical. and using a cloud server for an autocomplete seems unnecessary.
- short_sells_poo 7y ago
  I like the idea of TabNine, unfortunately it doesn't seem to be super well supported :( There are no news on whether it is being actively developed and the current implementation is unusable in a corporate environment because it can't dial home through a proxy so it refuses to activate the license. It's a shame too because it was basically a "shut up and take my money" reaction from me. I'd pay for this product. I'd pay good money for this product.
  - tcoff91 7y ago
    It has a completely local option which is the default.
    - short_sells_poo 7y ago
      Yes, but it still needs to talk to it's servers once to unlock the restricted version. It's all nice and dandy that after the one time activation it can function fully locally, but I never get to that point because the pointless "feature" of activation can't get through the corporate proxy. So to recap: 1. There is currently no paid license 2. The free version nevertheless needs to be activated online to unlock the full power (otherwise there are severe limitations) 3. It can't handle proxies, so you can't activate it at all in corporate environments
- thrower123 7y ago
  I am relatively sure that you could tell that my keyboard is primarily used for coding C# in Visual Studio just by looking at which keys are the most worn and which carry a significant layer of dust on them. The keys that complete an Intellisense selection (space, period, semi-colon, Enter) are nearly ground down to nubs, and the open-paren and open-bracket keys are worn smooth, while the corresponding close keys indicate near complete disuse. Similarly for F5 (Start Debugger) and F10 (Step Over) compared to the rest of the F-keys.
ijidak 7y ago
FYI, I thought my programming career was over due to RSI. Now, I only type while wearing long-sleeves. And of course, I still have to take regular breaks. I no longer suffer RSI symptoms. I'm guessing because it increases blood flow to the area and perhaps the warmth helps keep ligaments and muscles flexible and loose. Simple solution, but took a while to figure out. Hopefully this helps someone reading this.
- meddlepal 7y ago
  How do the long sleeves help? Cushion?
  - geewee 7y ago
    Probably also warmth, increasing blood flow. Particularly important if you also have cold hands.
    - Wistar 7y ago
      Rapid Reboot makes compression (peristaltic?) massaging boots and the like for athletes. Maybe something similar but heated, massaging sleeves, can be developed to help with keyboard RSI? https://rapidreboot.com/?gclid=EAIaIQobChMImbn228fq5wIVjMVkCh0fOAySEAAYASAAEgJGZfD_BwE https://rapidreboot.com/?gclid=EAIaIQobChMImbn228fq5wIVjMVkC...
- deleted 7y ago
  [deleted]
- reportgunner 7y ago
  I remember reading some guy's article where he said wearing fingerless gloves helped him with RSI because he was actually sitting right below the AC fan the whole time.
  - geewee 7y ago
    Probably this one: https://codewithoutrules.com/2016/11/18/rsi-solution/ https://codewithoutrules.com/2016/11/18/rsi-solution/
rijoja 7y ago
How well would this work in an open office setting, especially if several people are using it?
- PaulRobinson 7y ago
  I suspect this setup would not be ideal as background noise getting worse would be annoying for all concerned and even more challenging than an open plan office already is, and your colleagues might not appreciate this unless you're already surrounded by people talking all day (e.g. support or sales teams on calls). That aside, in terms of worrying about your mic picking up other people's voices and the voice dictation getting confused, most dedicated microphones these days (i.e. not ones that are built into your phone's headphones), are pretty good at background noise reduction. I've not used the one OP recommends - I'd never have considered a table based mic like that before - but the noise reduction on the Plantronics Blackwire 3215 headset I use is so good that if I move the mic boom a few inches up or down away from my mouth, people can't really hear me on calls. It's superb at getting rid of background noises, and if somebody else was in my home office using voice dictation it would not be picked up by my headset.
  - rijoja 7y ago
    Very interesting. But what about people who simply just rather use their fingers? Would you not rather type than speak if possible?
    - PaulRobinson 7y ago
      I think I personally would, but then I would rather hear some of my colleagues talking rather than using their mechanical keyboards in an open plan office. I work mostly from home, so I'm interested in this as a programming method to see how or if it changes the way I approach it as work.
- deleted 7y ago
  [deleted]
- lunixbochs 7y ago
pfraze 7y ago
I was curious what this might look like and found a couple videos on YouTube https://www.youtube.com/watch?v=fBhBqlQj00Q https://www.youtube.com/watch?v=fBhBqlQj00Q https://www.youtube.com/watch?v=hGPNs5C1Lp0 https://www.youtube.com/watch?v=hGPNs5C1Lp0
- lunixbochs 7y ago
  Some of the voice coding videos I’ve found are a little contrived, or seem practiced (Even some of mine were just copying code from one window to another or demoing one small feature). In this playlist, I do a lot of code from scratch, by competing in Advent of Code 2018 for the first week: https://www.youtube.com/playlist?list=PL2wTcyeSmhsZuHqbSlLAxkroWIuIpExwx&feature=share https://www.youtube.com/playlist?list=PL2wTcyeSmhsZuHqbSlLAx... As I start each day exactly when the problem is revealed, without any prior knowledge of the task, everything about these videos including debugging broken code is completely real.
- gitgreen 7y ago
  Here is a very early(2007) and unintentionally humorous attempt to script in Perl with voice recognition. https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec
  - lunixbochs 7y ago
    I know it’s funny, but I hate the prevalence of this video so much, because people reference it as a reason voice programming doesn’t work, and I think it discourages people who may have otherwise had success with voice programming. You should consider linking to the middle of Emily Shea’s PerlCon video where she first plays this video then shows that it’s actually possible to dictate the same code effectively.
    - gitgreen 7y ago
      I don't know which video you're referring to and a cursory search didn't turn it up. Perhaps you could link it?
dwiel 7y ago
I've been voice coding for about 5 years now. For those of you not on windows, I use talon voice on mac (linux version is in beta). It works quite well and I'm at least as productive writing code by voice than I ever was by hand. I was someone who would spend the time to get my emacs and then later vim configs highly optimized, but there is something liberating about not constraining yourself to key bindings. I used to type gcC to comment a python class in vim, now I say comment class. For commands you type frequently to get into muscle memory this isn't a huge gain, but for all the things you don't use regularly, it's so much easier for me to remember normal words than keyboard shortcuts.
- lunixbochs 7y ago
  At this point all of the projects mentioned in this thread (caster/talon/serenade) have some option for supporting the three main (win/lin/mac) platforms.
  - gleb 7y ago
    How do you get/request access to Talon on Linux?
    - lunixbochs 7y ago
      You can follow the Talon website in my HN profile to the Donate link (Patreon) and sign up for the beta tier then join the Slack to get access. Or go straight to the Slack and make your case for not being able to support the beta on Patreon at this time.
lunixbochs 7y ago
This post starts out talking about expecting to spend around $1,000. There are at least two cross-platform projects where the biggest expense is a microphone instead of software. 1. My project, Talon. Windows/Linux/Mac support, and a first party local speech recognition engine that is pretty good and getting better. It’s free, but the engine is in a private beta (which is $15/mo to support development, optional if there’s a financial issue). 2. Serenade. They are VC backed. Currently free, unsure about their longer term plans. They use cloud based recognition.
- pumanoir 7y ago
  I have intermittent RSI and I've been using Talon for 1.5 years for programming (web) with intellij and spacemacs. I reckon I'm as productive as using my hands if I use only voice. When my hands don't hurt and i mix talon with my hands I feel i can do more than I could do with my hands alone. Thanks, lunixbochs. Talon is great.
  - bobakanoosh 7y ago
    Can you give some examples of how you intermix the two? I'm curious in trying this out.
    - pumanoir 7y ago
      When using vim keys for example, to jump to the beginning of the line I press ^ then press 'dw' keys delete first word in a line. So I do the caret with my voice and the other press with my hand which is faster than doing it by hand alone.
      - lunixbochs 7y ago
        Interesting, is that something you’d consider using the noise input for once I finish that? (As noise will be lower latency, you’ll have about 40 noises to map, and it’s “out of band” with speech)
qixxiq 7y ago
It's still surprising to see Dragon Speech Recognition as the recommended (and only) choice here. Is anyone working on decent speech recognition for Mac/Linux or know good resources for that? The ideal output is a stream of what could have been said, as well as some alternatives, each with a confidence. Every alternative I've tried has not been as effective as the version of Dragon I used from 2011. I think the focus on accents and training is a big thing here -- I'm happy to spend a couple hours training it for better results.
- tbenst 7y ago
  The best alternative right now for voice coding is https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar. wav2letter is also under use by some, although requires more effort to setup
  - lunixbochs 7y ago
    > wav2letter is also under use by some, although requires more effort to setup This isn’t quite accurate. To my knowledge the options are: you are either using it in Talon and it just works, or you want to use it outside Talon and you will need to write entirely new glue code to add support for it in your project of choice.
- deleted 7y ago
  [deleted]
- lunixbochs 7y ago
  The Talon beta ships with wav2letter and a really good many-accent English model that can handle both arbitrary commands and free form English. All of my trained models and some information is posted here: https://talonvoice.com/research/ https://talonvoice.com/research/
  - woodson 7y ago
    Are you going to release the speech data you collect at https://speech.talonvoice.com/ https://speech.talonvoice.com/ or is it proprietary?
random3 7y ago
I had my second surgery just a few months ago. I can type again, but each time I had to use my left hand for months. Initially, it feels like your brain doesn't function properly anymore (not mentioning the psychological effort you have to make in order to be focused on work when you feel your hands are falling apart). Keyboard speed is directly related to how fast you can move your hands to support your thought flow. I tried Kinesis and even a split vertical keyboard (KeyboardIo) but none avoided the pain and numbness that came with typing. The other problem with thumb-cluster keyboards is that your IDE productivity goes to zero. I was faster with just my left hand on a regular keyboard than with both. I think this would be fixable with a good amount of time remapping shortcuts, etc. Now that my hand works again, I think I should start spending time getting used to my KeyboardIO and at least try to buy some time. The "voice coding" space is maybe not a mess, but far from great or even acceptable. However, there seem to be more recent efforts to make better tools. I would definitely check https://serenade.ai/ https://serenade.ai/ out. The main problem, I think is that "voice coding" is too much focused on editor typing which they can't do right as, when combined with code syntax, it becomes too complex. Instead, they should focus on higher level actions (which btw, Serenade does) along with a different approach to typing. I think Vim is a good example of where editing should be. IntelliJ refactoring is where voice coding should start. With all the AI buzz, it's unbelievable how bad voice recognition is. I'm not talking about "Siri set an alarm", but instead separating context from tone, not having to say things 2-3 times having good response latency, etc. Lastly, I wish there was simple voice assistance for code navigation - like go to definition, find usages, etc. This is much simpler to "parse" than code structure. Unfortunately, this is not even tackled by any tool as far as I've seen.
- mwiethoff 7y ago
  I’m one of the creators of Serenade—thanks for mentioning us! We totally agree about the need for higher-level layers of abstraction, and we’re working on some of the code navigation functionality you mentioned right now. If you have any other ideas or feedback, we’d love to chat more, I’m matt@serenade.ai.
  - nshm 7y ago
    Among all demos above Serenade seems the only product that does the right thing. Why the hell I should say aloud "colon" or "quote" where tool can put them automatically.
AstralStorm 7y ago
This article could use a better title. Voice coding makes me think of audio compression for voice. This is programming using voice recognition instead.
- jakemal 7y ago
  Isn't that voice encoding?
  - aidenn0 7y ago
    Speech coding actually (though a "vocoder" may be used as part of speech coding): https://en.wikipedia.org/wiki/Speech_coding https://en.wikipedia.org/wiki/Speech_coding
- fizixer 7y ago
  I agree. 'Voice coding' does not make any sense, no offense, even if it wasn't a technical term in a completely different discipline. It should be 'voice-activated coding', VAC, or 'voice-activated programming' (VAP) or something.
- lunixbochs 7y ago
  A small nitpick when talking about voice programming is there’s a superset of “voice control”. When your hands are hurt you don’t just want to type in your editor and terminal, there’s chat programs, application switching, web browsing, and other tasks it would be nice to offload from your hands too. So the clearest way to represent some of this may be to encode the meaning of “voice input/control capable of programming” in a title. We might need a new name for this kind of input to best represent it.
- bradenb 7y ago
  I think this is a bit nitpicky. I figured what it meant pretty immediately but even if I hadn't the actual article would have cleared it up quickly.
dotasm 7y ago
Wow DragonFly, that’s a name I haven’t heard in a while. I wonder if the speech to text space will change now that ML has gained popularity.
- lunixbochs 7y ago
  There’s recently Common Voice from Mozilla, which is a huge free English dataset (1500 hours and growing), and wer_are_we [1] has shown really impressive accuracy increases in published research the past few years. Exciting times. [1] https://github.com/syhw/wer_are_we https://github.com/syhw/wer_are_we
lbj 7y ago
Just a quick shout out to Microsoft (no affiliation) - Their Sculpt keyboard has taken all the discomfort out of 15hour coding sessions. If you're a professional developer, get one.
- yreg 7y ago
  I'd recommend avoiding 15-hour coding sessions over using an ergonomic keyboard.
  - lbj 7y ago
    Probably better. But when the motivation grabs me, I cant stop.
- lunixbochs 7y ago
  The Sculpt and Kensington expert trackball (big trackball, not thumb trackball), while not fixing my issues completely, did improve things for me quite a bit. That said, barring genetic lottery, as the sibling said: enough 15 hour sessions are going to give you RSI no matter what you’re typing on. There’s no way to do that every day, and exercise, take breaks, and sleep enough. Balance is important.
  - lbj 7y ago
    I agree. Worst streak was 18 months of those session nearly daily. Was a bit rough at the end.
- war1025 7y ago
  > taken all the discomfort out of 15hour coding sessions I can't imagine working 15 hours straight would be comfortable for me no matter what tools I was using.
- jedberg 7y ago
  I've been using the Microsoft Natural Ergo for 20+ years. It's the very first thing I ask for when I get a new job, or if they don't supply keyboards, the first thing I bring from home. I've never had any RSI type symptoms or even fatigue after long typing sessions. The sculpt seems to just be a fancier wireless version of the same thing (although I haven't tried it so I could be wrong).
cjbassi 7y ago
For those on Linux, I've been working on a Talon inspired voice coding program called Osprey that uses the Google Cloud speech to text API: https://github.com/osprey-voice/osprey https://github.com/osprey-voice/osprey. It's still very much a work in progress but it's already been working very well for me and I'm actually using it to type out this response right now.
- nshm 7y ago
  Why would you work with google when there are much more accurate open source speech recognizers based on Kaldi? With that specific usecase it is very easy to beat Google on accuracy.
  - lunixbochs 7y ago
    I think google as the primary engine results in stuff like this: https://github.com/osprey-voice/osprey-starter-pack/blob/master/src/text/formatters.py#L8 https://github.com/osprey-voice/osprey-starter-pack/blob/mas... On the other hand it’s probably better at general (non command) English.
    - cjbassi 7y ago
      Yeah it uses a lot of machine learning and context based inference, which is great for dictating phrases but less so for commands.
  - cjbassi 7y ago
    Actually I don't think I ended up testing kaldi because it seemed difficult to set up but I'll give it a try now that you mention it.
    - nshm 7y ago
      Ok, if you want to start with Kaldi it is probably easier to check kaldi-active-grammar mention above or https://github.com/alphacep/vosk-api https://github.com/alphacep/vosk-api
totalthrowaway 7y ago
Example (with Perl!): https://www.youtube.com/watch?v=Mz3JeYfBTcY https://www.youtube.com/watch?v=Mz3JeYfBTcY
azinman2 7y ago
You gotta be careful not to get RSI in your vocal chords. I almost did when I tried voice coding years ago. For me at least, it tends to shift to whenever “work” is being done. Same thing happened when I experimented with eye tracking.
est 7y ago
Off topic, but I really wish to find out any Alexa-like "smart speakers" capable of voice programming. For example: 1. I would like to command the speaker listen for a keyword like the Fizz Buzz Test[1] if I counted to certain number. 2. Ask the speaker to remind me of something when hearing certain topics during a conversation. Much like the "if" keyword in text based computer programming languages. 3. Program a poem into the speaker over the microphone, tutor my kids to memorize it, correct the wrong parts. Share the snippet to other parents. program simple home made riddles and tests over voice. 4. The ability to store certain list/map structure as global variables. e.g. asking the speaker, who is the second oldest son in this family? Who got up first this morning? 5. Voice memos and search engine. Stored and indexed securely offline on my home NAS. [1]: https://wiki.c2.com/?FizzBuzzTest https://wiki.c2.com/?FizzBuzzTest
- lunixbochs 7y ago
  I would probably start with the seeed studio ReSpeaker array hardware wise. Otherwise, can you write up a more specific use case list with specific commands and responses? This sounds fun and I can probably help you make this happen.
  - est 7y ago
    > write up a more specific use case list with specific commands and responses "no swear words" rule for kids. The speaker need to maintain a global counter for kid_id/word counter for every time a swear word is heard. Will reset every week for rewards/penalty, etc. The parents (as admin) could add/remove the swear words list, and can configure how long the counter need to be rotated. A non-trivial example, I am an ESL speaker, trying to teach my kid to learn English. But had little patience with the grammar and pronunciation. (My Bad) I'd like to record a simple poem e.g. The Moon by Robert Stevenson, ask my kid to listen to it without need to stare at any screen, the aim is to recite it correctly, the programmable speaker would correct any missing words with stronger emphasizes next time, and count any badly spoken words with a counter, test it several rounds to see if my kid improves and recite the full text correctly. There are many family games that can be played around a programmable speaker, e.g. as the dungeon master. The story teller, the on-demand background music composer, etc.
russellbeattie 7y ago
Just want to point out a very accurate, yet completely inelegant solution to voice text input: Saying each letter and symbol individually. If you haven't tried it, you'll think I'm crazy, but it's amazing how fast computers can recognize individual letter names. You can just blurt them all out. I discovered this while entering a domain name by voice - just spell it out and poof, no problem, no corrections. Not sure I'd want to spend all day doing it that way, but rather than fighting with voice recognition for misunderstood homonyms, just fall back on individual keys.
- lunixbochs 7y ago
  You’re exactly right, and this is something I put a lot of work into! I designed a new phonetic alphabet - the “Talon alphabet” - (which isn’t proprietary to Talon, people have used it with Caster as well). The trick is one-syllable words to represent each letter in the alphabet, but they have high phoneme diversity (sub-syllable unique sounds) and care was taken to ensure they can be differentiated even when saying them “too fast” in a stream without gaps, and I tested them extensively to make sure I picked words that didn’t clog up your mouth too much. The result is I can type 58wpm for short bursts using the alphabet alone, which is enough to play ztype, write entire words that would be hard to dictate, and control vim directly without an additional command set. The base alphabet is: air bat cap drum each fine gust harp sit jury crunch look made near odd pit quench red sun trap urge vest whale plex yank zip Go ahead, see how fast you can read the alphabet in order. Then try reading it even faster, blurring the word boundaries together a bit. You can still tell the words apart! Some people change a couple of words if they don’t work well in their accent, but the base idea holds strong.
  - Hasnep 7y ago
    How do you pronounce jury as one syllable word? I'm finding it difficult to imagine.
    - shagbag 7y ago
      You could pronounce it as jree. Like tree with a j.
mfrye0 7y ago
I went through a similar situation and ended up using https://voicecode.io https://voicecode.io for a few years. It worked well enough for me, but I always found it hard using it in a public setting. Ironically, I solved my RSI issue after coming across a book on HN. Everyone seemed to recommend the Mind Body Prescription, and after trying it, it did the trick. https://www.amazon.com/Mindbody-Prescription-Healing-Body-Pain/dp/0446675156 https://www.amazon.com/Mindbody-Prescription-Healing-Body-Pa... Edit: I've been wanting to try this project for several years now: https://www.ctrl-labs.com https://www.ctrl-labs.com. I think something like this is the solution moving forward vs voice coding.
LexiconCode 7y ago
Greetings Everyone, I help maintain the Caster project. The key difference from other solutions out there as we seek to support a completely open source voice coding stack. Open source is only way to go long term if you're going to being using a tool for most of your life. Fortunately for some it acts as a bridge until their RSI symptoms becomes manageable or goes into remission. We are working towards cross-platform support Linux and Mac as well as adding support for Kaldi. Dragonfly is already cross-platform so just a few windows specific functions to be ported yet in Caster. Kaldi via daanzu's kaldi active grammar. https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar Talon may be free but is closed sourced.
- daanzu 7y ago
  I said this in another comment, but it can't be emphasized enough: I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could. For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed Kaldi Active Grammar, and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. Open source is what allows this.
  - lunixbochs 7y ago
    I understand and won't argue on your preference for the core Talon app. However, as all of my wav2letter code, models, tools, training methodology, and general advice (e.g. I am very active on the github/facebookresearch/wav2letter issue tracker helping others) are open source, and wav2letter as used in Talon is built from the public repository and dynamically linked, I don't think the speech engine is the place to speak against Talon's source policy.
    - daanzu 7y ago
DantesKite 7y ago
You can solve most carpal tunnel issues by massaging your forearms with a ball (or your knuckles). I know it sounds crazy, but I solved a very intense bout of plantar fasciitis by massaging my calves. Took some time, but eventually the pain went away. And when I had carpal tunnel, I did the same (although not learning heavily on my wrists helped a lot too). You'll know if you're hitting the right spot because it'll hurt. A lot.
- lunixbochs 7y ago
  * I am not a doctor, I have simply had a lot of experience with treating my own injuries and speaking to others in similar situations This is partially good, but dangerously incomplete advice. The spectrum of RSI is far bigger than carpal tunnel, such as cubital tunnel, tennis elbow, tendinitis, and general neuropathy. It can be caused by many things including various muscle groups in your shoulders, neck, posture, past injuries, nerve adhesion, flexibility, lack of strength. Massage is a wonderful way to address nerve adhesion, flexibility, general tightness, but if you have a different issue it’s not really a panacea. My philosophy is that you should do these things in strict order if you have symptoms: 1. reduce use (take breaks, get hobbies that take you away from the screen at home) 2. vary conditions of use (standing desk, ergonomics, using more than one kind of keyboard/mouse) 3. work out (yoga, swimming are highly recommended) 4. physical treatment (see a doctor, physical therapist, deep masseuse, etc) 5. alternative input (voice, eye tracking, etc) If you do these in a different order or skip any of them you’re probably doing yourself a disservice. Also. Don’t ice. Don’t take anti inflammatories or other pain meds while you continue to work. Blood flow is important to recovery - heat can soothe your pain similarly to as ice, but without causing further damage. And pain meds during working hours will make you ignore the damage that is causing your pain, allowing you to cause more damage. Bad. I’ve also seen some negative studies on steroids and surgery, suggesting if you get those without changing your habits, your pain will recur within a year in 90% of cases. Do your own research, but you might as well change your habits first and only use surgery/drugs as an extreme last resort. (Unless you have the specific form of carpal tunnel that can be permanently fixed in a day with a minor surgery in exchange for lower grip strength?)
disease 7y ago
I've had some close calls with RSI and the most helpful things for me were: 1) Getting an ergo keyboard, in my case the Microsoft Sculpt 2) Remapping my keys to better match my workflow. Left and right parens are mapped to left and right shift - same for Ctrl-Braces and Alt for brackets. Mapping Caps lock to delete one word back was also a big one. Further, I have the number pad on the left to both make using the mouse require less movement in addition to remapping all the numpad keys to useful programming commands.
- lunixbochs 7y ago
  A minor tip for people fiddling with keyboards/layout. Be careful of regular pinky use for modifiers. Pinkies are weak and overuse can cause ulnar issues. Corollary, if you have ulnar (pinky side) issues in your forearm, reduce pinky use.
anotherevan 7y ago
A good accompaniment to this is a recent presentation at Linux.conf.au this year by Shervin Emami: Desktop Linux, without a keyboard, mouse or desk. https://youtu.be/3aQfwS5pyrg https://youtu.be/3aQfwS5pyrg
singularity2001 7y ago
That’s why I’m working on angle, a speakable dialect of python https://github.com/pannous/angle https://github.com/pannous/angle
jorgeer 7y ago
I had several onsets of RSI a few years back, and had to resort to voice coding as a last resort, after stretches, pauses and ergonomic everything did not do the job. It was pretty awful. But then, after having seen doctors and neurologists, and finally a physical therapist, I came across my salvation: - Exercising my hands. I very rarely see this mentioned for some reason. I exercised regularly, but only the bigger muscle groups, rarely grip strength and wrist strength. It felt counter-intuitive to exercise my already extremely painfully aching hands (when typing), but using grip weights and other methods to work out my hands and wrists, the pain went away quickly! If you are not diagnosed with carpal tunnel, and not already doing this, definitely try it, it saved my career.
- benterris 7y ago
  Could you please elaborate on the exercises you're finding helpful ? I have a mild RSI myself (outer part of the forearm, near the elbow) and have been trying some eccentric exercises for a few months now but I'm not seeing a big improvement.
  - ajmarcic 7y ago
    I list the low-hanging fruit first: Look up nerve flossing exercises on YouTube. Routinely doing these had the largest impact for me. You'll feel the ones that work on whatever nerve is inflamed. Try to improve your posture. My general mantra is "lift your head as much as possible. Pull your shoulders back". It gets easier eventually. If your head lies forward from the spine you have a hunch. You may notice a small lump of muscle behind your neck. That's bad. There are exercises to try and strengthen the opposite muscles. If you sleep on your arms try to stop as well. I recommend sleeping on your back. Fluffy couches and back rests sacrifice posture. Don't use a laptop in bed. Sleep, eat nutrient-dense foods, and run or swim. Avoid alcohol. If you do pushups or bench press, make sure to exercise your upper back equally to avoid imbalance.
  - untoreh 7y ago
    I use a powerball
- 7y ago
mikob 7y ago
I have a free Chrome Extension that let's you control the browser via voice: https://www.lipsurf.com https://www.lipsurf.com Rest is ultimately the best way to prevent hand injuries and since I spend most of my time in Chrome, this extension lets me do it hands free.

316 points by buchuki 7y ago

“I type fast enough that I have never had much use for snippets in my text editor. Perhaps if I’d used them more effectively, I would not have developed the RSI symptoms that I have!" Often we have discussions about advanced code completion on HN. Many developers feel they don’t need it, or that it gets in their way, for example. Reading stories like this convinces me even more that our editors (tools) need to be smarter. There is so much repetition is coding, it’s hard to believe we can’t do better. This tool is often mentioned: https://tabnine.com/ https://tabnine.com/

busterarm 7y ago
I've been typing just about 4-16 hours a day going on 30 years now without RSI. Fingers crossed.
- melling 7y ago
  Isn’t that sort of like saying, I smoked 2 packs of cigarettes for 30 years, fingers crossed? Some people are more susceptible to problems. This was on HN several years ago. http://www.looknohands.me/ http://www.looknohands.me/ You can be in your 20’s and have a problem.
  - busterarm 7y ago
    I also worked with someone in a remote capacity who insisted on using Dragon Naturally Speaking and his communication on technical topics was unintelligible. It clearly wasn't working, but he insisted and everyone who worked with him suffered for it. And now he's not working in tech anymore and is a field engineer doing repetitive labor with his hands...
  - magicalhippo 7y ago
    I actively try to switch things up, depending on if I feel something is starting to get uncomfortable. If my mouse hand starts to give signals, I'll switch to using keyboard more. Regardless I'll switch seating position and hand orientation throughout the day. So in that respect it's unlike smoking I guess.
keymone 7y ago
We should use better languages. Better understood and integrated with the editor and better at expressing every level of abstraction. coughpretty much any lispcough
- ulisesrmzroche 7y ago
  Or you could take a short break every so often What we really need are better programmers
  - keymone 7y ago
    Tools matter. First ascent of Eiger’s north face took few days and team of 4. Last I heard Ueli Steck did it few years ago solo in less than 2 hours. Maybe he was better but not by that much. His tools were unquestionably better.
    - shrimp_emoji 7y ago
      If better tools also make "worse" programmers (i.e., people saner and happier and uninitiated with the brain death rigors of struggling with awful/no tools), tools also matter more the better they get.
      - keymone 7y ago
        If a tool makes worse programmer, it’s not better I would say.
lunixbochs 7y ago
Deep TabNine looks so good it’s downright spooky: https://tabnine.com/blog/deep https://tabnine.com/blog/deep
- singularity2001 7y ago
  This will result in a whole new class of copy paste bugs. Great anyways
est 7y ago
I found that Kite is comparable with Tabnine, at least with Python. https://kite.com/integrations/kite-vs-tabnine/ https://kite.com/integrations/kite-vs-tabnine/
- Gaelan 7y ago
  Is Kite's behavior still as shady as it used to be?
- lunixbochs 7y ago
  Because this says “single token completion” as the tabnine example gif I’m guessing it isn’t comparing to deep tabnine at all, which is on an entirely different level
- 90litshi99 7y ago
  the memory usage comparison is accurate for tabnine, at least in my case. local-only tab nine wasn't practical. and using a cloud server for an autocomplete seems unnecessary.
short_sells_poo 7y ago
I like the idea of TabNine, unfortunately it doesn't seem to be super well supported :( There are no news on whether it is being actively developed and the current implementation is unusable in a corporate environment because it can't dial home through a proxy so it refuses to activate the license. It's a shame too because it was basically a "shut up and take my money" reaction from me. I'd pay for this product. I'd pay good money for this product.
- tcoff91 7y ago
  It has a completely local option which is the default.
  - short_sells_poo 7y ago
    Yes, but it still needs to talk to it's servers once to unlock the restricted version. It's all nice and dandy that after the one time activation it can function fully locally, but I never get to that point because the pointless "feature" of activation can't get through the corporate proxy. So to recap: 1. There is currently no paid license 2. The free version nevertheless needs to be activated online to unlock the full power (otherwise there are severe limitations) 3. It can't handle proxies, so you can't activate it at all in corporate environments
thrower123 7y ago
I am relatively sure that you could tell that my keyboard is primarily used for coding C# in Visual Studio just by looking at which keys are the most worn and which carry a significant layer of dust on them. The keys that complete an Intellisense selection (space, period, semi-colon, Enter) are nearly ground down to nubs, and the open-paren and open-bracket keys are worn smooth, while the corresponding close keys indicate near complete disuse. Similarly for F5 (Start Debugger) and F10 (Step Over) compared to the rest of the F-keys.

ijidak 7y ago

FYI, I thought my programming career was over due to RSI. Now, I only type while wearing long-sleeves. And of course, I still have to take regular breaks. I no longer suffer RSI symptoms. I'm guessing because it increases blood flow to the area and perhaps the warmth helps keep ligaments and muscles flexible and loose. Simple solution, but took a while to figure out. Hopefully this helps someone reading this.

meddlepal 7y ago
How do the long sleeves help? Cushion?
- geewee 7y ago
  Probably also warmth, increasing blood flow. Particularly important if you also have cold hands.
  - Wistar 7y ago
    Rapid Reboot makes compression (peristaltic?) massaging boots and the like for athletes. Maybe something similar but heated, massaging sleeves, can be developed to help with keyboard RSI? https://rapidreboot.com/?gclid=EAIaIQobChMImbn228fq5wIVjMVkCh0fOAySEAAYASAAEgJGZfD_BwE https://rapidreboot.com/?gclid=EAIaIQobChMImbn228fq5wIVjMVkC...
deleted 7y ago
[deleted]
reportgunner 7y ago
I remember reading some guy's article where he said wearing fingerless gloves helped him with RSI because he was actually sitting right below the AC fan the whole time.
- geewee 7y ago
  Probably this one: https://codewithoutrules.com/2016/11/18/rsi-solution/ https://codewithoutrules.com/2016/11/18/rsi-solution/

rijoja 7y ago

How well would this work in an open office setting, especially if several people are using it?

PaulRobinson 7y ago
I suspect this setup would not be ideal as background noise getting worse would be annoying for all concerned and even more challenging than an open plan office already is, and your colleagues might not appreciate this unless you're already surrounded by people talking all day (e.g. support or sales teams on calls). That aside, in terms of worrying about your mic picking up other people's voices and the voice dictation getting confused, most dedicated microphones these days (i.e. not ones that are built into your phone's headphones), are pretty good at background noise reduction. I've not used the one OP recommends - I'd never have considered a table based mic like that before - but the noise reduction on the Plantronics Blackwire 3215 headset I use is so good that if I move the mic boom a few inches up or down away from my mouth, people can't really hear me on calls. It's superb at getting rid of background noises, and if somebody else was in my home office using voice dictation it would not be picked up by my headset.
- rijoja 7y ago
  Very interesting. But what about people who simply just rather use their fingers? Would you not rather type than speak if possible?
  - PaulRobinson 7y ago
    I think I personally would, but then I would rather hear some of my colleagues talking rather than using their mechanical keyboards in an open plan office. I work mostly from home, so I'm interested in this as a programming method to see how or if it changes the way I approach it as work.
deleted 7y ago
[deleted]
lunixbochs 7y ago

pfraze 7y ago

I was curious what this might look like and found a couple videos on YouTube https://www.youtube.com/watch?v=fBhBqlQj00Q https://www.youtube.com/watch?v=fBhBqlQj00Q https://www.youtube.com/watch?v=hGPNs5C1Lp0 https://www.youtube.com/watch?v=hGPNs5C1Lp0

lunixbochs 7y ago
Some of the voice coding videos I’ve found are a little contrived, or seem practiced (Even some of mine were just copying code from one window to another or demoing one small feature). In this playlist, I do a lot of code from scratch, by competing in Advent of Code 2018 for the first week: https://www.youtube.com/playlist?list=PL2wTcyeSmhsZuHqbSlLAxkroWIuIpExwx&feature=share https://www.youtube.com/playlist?list=PL2wTcyeSmhsZuHqbSlLAx... As I start each day exactly when the problem is revealed, without any prior knowledge of the task, everything about these videos including debugging broken code is completely real.
gitgreen 7y ago
Here is a very early(2007) and unintentionally humorous attempt to script in Perl with voice recognition. https://www.youtube.com/watch?v=MzJ0CytAsec https://www.youtube.com/watch?v=MzJ0CytAsec
- lunixbochs 7y ago
  I know it’s funny, but I hate the prevalence of this video so much, because people reference it as a reason voice programming doesn’t work, and I think it discourages people who may have otherwise had success with voice programming. You should consider linking to the middle of Emily Shea’s PerlCon video where she first plays this video then shows that it’s actually possible to dictate the same code effectively.
  - gitgreen 7y ago
    I don't know which video you're referring to and a cursory search didn't turn it up. Perhaps you could link it?

dwiel 7y ago

I've been voice coding for about 5 years now. For those of you not on windows, I use talon voice on mac (linux version is in beta). It works quite well and I'm at least as productive writing code by voice than I ever was by hand. I was someone who would spend the time to get my emacs and then later vim configs highly optimized, but there is something liberating about not constraining yourself to key bindings. I used to type gcC to comment a python class in vim, now I say comment class. For commands you type frequently to get into muscle memory this isn't a huge gain, but for all the things you don't use regularly, it's so much easier for me to remember normal words than keyboard shortcuts.

lunixbochs 7y ago
At this point all of the projects mentioned in this thread (caster/talon/serenade) have some option for supporting the three main (win/lin/mac) platforms.
- gleb 7y ago
  How do you get/request access to Talon on Linux?
  - lunixbochs 7y ago
    You can follow the Talon website in my HN profile to the Donate link (Patreon) and sign up for the beta tier then join the Slack to get access. Or go straight to the Slack and make your case for not being able to support the beta on Patreon at this time.

lunixbochs 7y ago

This post starts out talking about expecting to spend around $1,000. There are at least two cross-platform projects where the biggest expense is a microphone instead of software. 1. My project, Talon. Windows/Linux/Mac support, and a first party local speech recognition engine that is pretty good and getting better. It’s free, but the engine is in a private beta (which is $15/mo to support development, optional if there’s a financial issue). 2. Serenade. They are VC backed. Currently free, unsure about their longer term plans. They use cloud based recognition.

pumanoir 7y ago
I have intermittent RSI and I've been using Talon for 1.5 years for programming (web) with intellij and spacemacs. I reckon I'm as productive as using my hands if I use only voice. When my hands don't hurt and i mix talon with my hands I feel i can do more than I could do with my hands alone. Thanks, lunixbochs. Talon is great.
- bobakanoosh 7y ago
  Can you give some examples of how you intermix the two? I'm curious in trying this out.
  - pumanoir 7y ago
    When using vim keys for example, to jump to the beginning of the line I press ^ then press 'dw' keys delete first word in a line. So I do the caret with my voice and the other press with my hand which is faster than doing it by hand alone.
    - lunixbochs 7y ago
      Interesting, is that something you’d consider using the noise input for once I finish that? (As noise will be lower latency, you’ll have about 40 noises to map, and it’s “out of band” with speech)

qixxiq 7y ago

It's still surprising to see Dragon Speech Recognition as the recommended (and only) choice here. Is anyone working on decent speech recognition for Mac/Linux or know good resources for that? The ideal output is a stream of what could have been said, as well as some alternatives, each with a confidence. Every alternative I've tried has not been as effective as the version of Dragon I used from 2011. I think the focus on accents and training is a big thing here -- I'm happy to spend a couple hours training it for better results.

tbenst 7y ago
The best alternative right now for voice coding is https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar. wav2letter is also under use by some, although requires more effort to setup
- lunixbochs 7y ago
  > wav2letter is also under use by some, although requires more effort to setup This isn’t quite accurate. To my knowledge the options are: you are either using it in Talon and it just works, or you want to use it outside Talon and you will need to write entirely new glue code to add support for it in your project of choice.
deleted 7y ago
[deleted]
lunixbochs 7y ago
The Talon beta ships with wav2letter and a really good many-accent English model that can handle both arbitrary commands and free form English. All of my trained models and some information is posted here: https://talonvoice.com/research/ https://talonvoice.com/research/
- woodson 7y ago
  Are you going to release the speech data you collect at https://speech.talonvoice.com/ https://speech.talonvoice.com/ or is it proprietary?

random3 7y ago

I had my second surgery just a few months ago. I can type again, but each time I had to use my left hand for months. Initially, it feels like your brain doesn't function properly anymore (not mentioning the psychological effort you have to make in order to be focused on work when you feel your hands are falling apart). Keyboard speed is directly related to how fast you can move your hands to support your thought flow. I tried Kinesis and even a split vertical keyboard (KeyboardIo) but none avoided the pain and numbness that came with typing. The other problem with thumb-cluster keyboards is that your IDE productivity goes to zero. I was faster with just my left hand on a regular keyboard than with both. I think this would be fixable with a good amount of time remapping shortcuts, etc. Now that my hand works again, I think I should start spending time getting used to my KeyboardIO and at least try to buy some time. The "voice coding" space is maybe not a mess, but far from great or even acceptable. However, there seem to be more recent efforts to make better tools. I would definitely check https://serenade.ai/ https://serenade.ai/ out. The main problem, I think is that "voice coding" is too much focused on editor typing which they can't do right as, when combined with code syntax, it becomes too complex. Instead, they should focus on higher level actions (which btw, Serenade does) along with a different approach to typing. I think Vim is a good example of where editing should be. IntelliJ refactoring is where voice coding should start. With all the AI buzz, it's unbelievable how bad voice recognition is. I'm not talking about "Siri set an alarm", but instead separating context from tone, not having to say things 2-3 times having good response latency, etc. Lastly, I wish there was simple voice assistance for code navigation - like go to definition, find usages, etc. This is much simpler to "parse" than code structure. Unfortunately, this is not even tackled by any tool as far as I've seen.

mwiethoff 7y ago
I’m one of the creators of Serenade—thanks for mentioning us! We totally agree about the need for higher-level layers of abstraction, and we’re working on some of the code navigation functionality you mentioned right now. If you have any other ideas or feedback, we’d love to chat more, I’m matt@serenade.ai.
- nshm 7y ago
  Among all demos above Serenade seems the only product that does the right thing. Why the hell I should say aloud "colon" or "quote" where tool can put them automatically.

AstralStorm 7y ago

This article could use a better title. Voice coding makes me think of audio compression for voice. This is programming using voice recognition instead.

jakemal 7y ago
Isn't that voice encoding?
- aidenn0 7y ago
  Speech coding actually (though a "vocoder" may be used as part of speech coding): https://en.wikipedia.org/wiki/Speech_coding https://en.wikipedia.org/wiki/Speech_coding
fizixer 7y ago
I agree. 'Voice coding' does not make any sense, no offense, even if it wasn't a technical term in a completely different discipline. It should be 'voice-activated coding', VAC, or 'voice-activated programming' (VAP) or something.
lunixbochs 7y ago
A small nitpick when talking about voice programming is there’s a superset of “voice control”. When your hands are hurt you don’t just want to type in your editor and terminal, there’s chat programs, application switching, web browsing, and other tasks it would be nice to offload from your hands too. So the clearest way to represent some of this may be to encode the meaning of “voice input/control capable of programming” in a title. We might need a new name for this kind of input to best represent it.
bradenb 7y ago
I think this is a bit nitpicky. I figured what it meant pretty immediately but even if I hadn't the actual article would have cleared it up quickly.

dotasm 7y ago

Wow DragonFly, that’s a name I haven’t heard in a while. I wonder if the speech to text space will change now that ML has gained popularity.

lunixbochs 7y ago
There’s recently Common Voice from Mozilla, which is a huge free English dataset (1500 hours and growing), and wer_are_we [1] has shown really impressive accuracy increases in published research the past few years. Exciting times. [1] https://github.com/syhw/wer_are_we https://github.com/syhw/wer_are_we

lbj 7y ago

Just a quick shout out to Microsoft (no affiliation) - Their Sculpt keyboard has taken all the discomfort out of 15hour coding sessions. If you're a professional developer, get one.

yreg 7y ago
I'd recommend avoiding 15-hour coding sessions over using an ergonomic keyboard.
- lbj 7y ago
  Probably better. But when the motivation grabs me, I cant stop.
lunixbochs 7y ago
The Sculpt and Kensington expert trackball (big trackball, not thumb trackball), while not fixing my issues completely, did improve things for me quite a bit. That said, barring genetic lottery, as the sibling said: enough 15 hour sessions are going to give you RSI no matter what you’re typing on. There’s no way to do that every day, and exercise, take breaks, and sleep enough. Balance is important.
- lbj 7y ago
  I agree. Worst streak was 18 months of those session nearly daily. Was a bit rough at the end.
war1025 7y ago
> taken all the discomfort out of 15hour coding sessions I can't imagine working 15 hours straight would be comfortable for me no matter what tools I was using.
jedberg 7y ago
I've been using the Microsoft Natural Ergo for 20+ years. It's the very first thing I ask for when I get a new job, or if they don't supply keyboards, the first thing I bring from home. I've never had any RSI type symptoms or even fatigue after long typing sessions. The sculpt seems to just be a fancier wireless version of the same thing (although I haven't tried it so I could be wrong).

cjbassi 7y ago

For those on Linux, I've been working on a Talon inspired voice coding program called Osprey that uses the Google Cloud speech to text API: https://github.com/osprey-voice/osprey https://github.com/osprey-voice/osprey. It's still very much a work in progress but it's already been working very well for me and I'm actually using it to type out this response right now.

nshm 7y ago
Why would you work with google when there are much more accurate open source speech recognizers based on Kaldi? With that specific usecase it is very easy to beat Google on accuracy.
- lunixbochs 7y ago
  I think google as the primary engine results in stuff like this: https://github.com/osprey-voice/osprey-starter-pack/blob/master/src/text/formatters.py#L8 https://github.com/osprey-voice/osprey-starter-pack/blob/mas... On the other hand it’s probably better at general (non command) English.
  - cjbassi 7y ago
    Yeah it uses a lot of machine learning and context based inference, which is great for dictating phrases but less so for commands.
- cjbassi 7y ago
  Actually I don't think I ended up testing kaldi because it seemed difficult to set up but I'll give it a try now that you mention it.
  - nshm 7y ago
    Ok, if you want to start with Kaldi it is probably easier to check kaldi-active-grammar mention above or https://github.com/alphacep/vosk-api https://github.com/alphacep/vosk-api

totalthrowaway 7y ago

Example (with Perl!): https://www.youtube.com/watch?v=Mz3JeYfBTcY https://www.youtube.com/watch?v=Mz3JeYfBTcY

azinman2 7y ago

You gotta be careful not to get RSI in your vocal chords. I almost did when I tried voice coding years ago. For me at least, it tends to shift to whenever “work” is being done. Same thing happened when I experimented with eye tracking.

est 7y ago

Off topic, but I really wish to find out any Alexa-like "smart speakers" capable of voice programming. For example: 1. I would like to command the speaker listen for a keyword like the Fizz Buzz Test[1] if I counted to certain number. 2. Ask the speaker to remind me of something when hearing certain topics during a conversation. Much like the "if" keyword in text based computer programming languages. 3. Program a poem into the speaker over the microphone, tutor my kids to memorize it, correct the wrong parts. Share the snippet to other parents. program simple home made riddles and tests over voice. 4. The ability to store certain list/map structure as global variables. e.g. asking the speaker, who is the second oldest son in this family? Who got up first this morning? 5. Voice memos and search engine. Stored and indexed securely offline on my home NAS. [1]: https://wiki.c2.com/?FizzBuzzTest https://wiki.c2.com/?FizzBuzzTest

lunixbochs 7y ago
I would probably start with the seeed studio ReSpeaker array hardware wise. Otherwise, can you write up a more specific use case list with specific commands and responses? This sounds fun and I can probably help you make this happen.
- est 7y ago
  > write up a more specific use case list with specific commands and responses "no swear words" rule for kids. The speaker need to maintain a global counter for kid_id/word counter for every time a swear word is heard. Will reset every week for rewards/penalty, etc. The parents (as admin) could add/remove the swear words list, and can configure how long the counter need to be rotated. A non-trivial example, I am an ESL speaker, trying to teach my kid to learn English. But had little patience with the grammar and pronunciation. (My Bad) I'd like to record a simple poem e.g. The Moon by Robert Stevenson, ask my kid to listen to it without need to stare at any screen, the aim is to recite it correctly, the programmable speaker would correct any missing words with stronger emphasizes next time, and count any badly spoken words with a counter, test it several rounds to see if my kid improves and recite the full text correctly. There are many family games that can be played around a programmable speaker, e.g. as the dungeon master. The story teller, the on-demand background music composer, etc.

russellbeattie 7y ago

Just want to point out a very accurate, yet completely inelegant solution to voice text input: Saying each letter and symbol individually. If you haven't tried it, you'll think I'm crazy, but it's amazing how fast computers can recognize individual letter names. You can just blurt them all out. I discovered this while entering a domain name by voice - just spell it out and poof, no problem, no corrections. Not sure I'd want to spend all day doing it that way, but rather than fighting with voice recognition for misunderstood homonyms, just fall back on individual keys.

lunixbochs 7y ago
You’re exactly right, and this is something I put a lot of work into! I designed a new phonetic alphabet - the “Talon alphabet” - (which isn’t proprietary to Talon, people have used it with Caster as well). The trick is one-syllable words to represent each letter in the alphabet, but they have high phoneme diversity (sub-syllable unique sounds) and care was taken to ensure they can be differentiated even when saying them “too fast” in a stream without gaps, and I tested them extensively to make sure I picked words that didn’t clog up your mouth too much. The result is I can type 58wpm for short bursts using the alphabet alone, which is enough to play ztype, write entire words that would be hard to dictate, and control vim directly without an additional command set. The base alphabet is: air bat cap drum each fine gust harp sit jury crunch look made near odd pit quench red sun trap urge vest whale plex yank zip Go ahead, see how fast you can read the alphabet in order. Then try reading it even faster, blurring the word boundaries together a bit. You can still tell the words apart! Some people change a couple of words if they don’t work well in their accent, but the base idea holds strong.
- Hasnep 7y ago
  How do you pronounce jury as one syllable word? I'm finding it difficult to imagine.
  - shagbag 7y ago
    You could pronounce it as jree. Like tree with a j.

mfrye0 7y ago

I went through a similar situation and ended up using https://voicecode.io https://voicecode.io for a few years. It worked well enough for me, but I always found it hard using it in a public setting. Ironically, I solved my RSI issue after coming across a book on HN. Everyone seemed to recommend the Mind Body Prescription, and after trying it, it did the trick. https://www.amazon.com/Mindbody-Prescription-Healing-Body-Pain/dp/0446675156 https://www.amazon.com/Mindbody-Prescription-Healing-Body-Pa... Edit: I've been wanting to try this project for several years now: https://www.ctrl-labs.com https://www.ctrl-labs.com. I think something like this is the solution moving forward vs voice coding.

LexiconCode 7y ago

Greetings Everyone, I help maintain the Caster project. The key difference from other solutions out there as we seek to support a completely open source voice coding stack. Open source is only way to go long term if you're going to being using a tool for most of your life. Fortunately for some it acts as a bridge until their RSI symptoms becomes manageable or goes into remission. We are working towards cross-platform support Linux and Mac as well as adding support for Kaldi. Dragonfly is already cross-platform so just a few windows specific functions to be ported yet in Caster. Kaldi via daanzu's kaldi active grammar. https://github.com/daanzu/kaldi-active-grammar https://github.com/daanzu/kaldi-active-grammar Talon may be free but is closed sourced.

daanzu 7y ago
I said this in another comment, but it can't be emphasized enough: I created Kaldi Active Grammar because I didn't trust relying on closed source software for something so crucial to my productivity, where a decision by an outside party determines whether I can function. As a bonus, open source means I can make it work better to fit my needs than closed source ever could. For what it's worth, my voice is quite abnormal, so most untrained speech recognition is terrible for me, and even performing the normal "training" for Dragon still resulted in very poor accuracy. However, apparently their training is quite limited, because once I developed Kaldi Active Grammar, and did my own direct training, the results were fantastic in comparison, with orders of magnitude better accuracy. Open source is what allows this.
- lunixbochs 7y ago
  I understand and won't argue on your preference for the core Talon app. However, as all of my wav2letter code, models, tools, training methodology, and general advice (e.g. I am very active on the github/facebookresearch/wav2letter issue tracker helping others) are open source, and wav2letter as used in Talon is built from the public repository and dynamically linked, I don't think the speech engine is the place to speak against Talon's source policy.
  - daanzu 7y ago

DantesKite 7y ago

You can solve most carpal tunnel issues by massaging your forearms with a ball (or your knuckles). I know it sounds crazy, but I solved a very intense bout of plantar fasciitis by massaging my calves. Took some time, but eventually the pain went away. And when I had carpal tunnel, I did the same (although not learning heavily on my wrists helped a lot too). You'll know if you're hitting the right spot because it'll hurt. A lot.

lunixbochs 7y ago
* I am not a doctor, I have simply had a lot of experience with treating my own injuries and speaking to others in similar situations This is partially good, but dangerously incomplete advice. The spectrum of RSI is far bigger than carpal tunnel, such as cubital tunnel, tennis elbow, tendinitis, and general neuropathy. It can be caused by many things including various muscle groups in your shoulders, neck, posture, past injuries, nerve adhesion, flexibility, lack of strength. Massage is a wonderful way to address nerve adhesion, flexibility, general tightness, but if you have a different issue it’s not really a panacea. My philosophy is that you should do these things in strict order if you have symptoms: 1. reduce use (take breaks, get hobbies that take you away from the screen at home) 2. vary conditions of use (standing desk, ergonomics, using more than one kind of keyboard/mouse) 3. work out (yoga, swimming are highly recommended) 4. physical treatment (see a doctor, physical therapist, deep masseuse, etc) 5. alternative input (voice, eye tracking, etc) If you do these in a different order or skip any of them you’re probably doing yourself a disservice. Also. Don’t ice. Don’t take anti inflammatories or other pain meds while you continue to work. Blood flow is important to recovery - heat can soothe your pain similarly to as ice, but without causing further damage. And pain meds during working hours will make you ignore the damage that is causing your pain, allowing you to cause more damage. Bad. I’ve also seen some negative studies on steroids and surgery, suggesting if you get those without changing your habits, your pain will recur within a year in 90% of cases. Do your own research, but you might as well change your habits first and only use surgery/drugs as an extreme last resort. (Unless you have the specific form of carpal tunnel that can be permanently fixed in a day with a minor surgery in exchange for lower grip strength?)

disease 7y ago

I've had some close calls with RSI and the most helpful things for me were: 1) Getting an ergo keyboard, in my case the Microsoft Sculpt 2) Remapping my keys to better match my workflow. Left and right parens are mapped to left and right shift - same for Ctrl-Braces and Alt for brackets. Mapping Caps lock to delete one word back was also a big one. Further, I have the number pad on the left to both make using the mouse require less movement in addition to remapping all the numpad keys to useful programming commands.

lunixbochs 7y ago
A minor tip for people fiddling with keyboards/layout. Be careful of regular pinky use for modifiers. Pinkies are weak and overuse can cause ulnar issues. Corollary, if you have ulnar (pinky side) issues in your forearm, reduce pinky use.

anotherevan 7y ago

A good accompaniment to this is a recent presentation at Linux.conf.au this year by Shervin Emami: Desktop Linux, without a keyboard, mouse or desk. https://youtu.be/3aQfwS5pyrg https://youtu.be/3aQfwS5pyrg

singularity2001 7y ago

That’s why I’m working on angle, a speakable dialect of python https://github.com/pannous/angle https://github.com/pannous/angle

jorgeer 7y ago

I had several onsets of RSI a few years back, and had to resort to voice coding as a last resort, after stretches, pauses and ergonomic everything did not do the job. It was pretty awful. But then, after having seen doctors and neurologists, and finally a physical therapist, I came across my salvation: - Exercising my hands. I very rarely see this mentioned for some reason. I exercised regularly, but only the bigger muscle groups, rarely grip strength and wrist strength. It felt counter-intuitive to exercise my already extremely painfully aching hands (when typing), but using grip weights and other methods to work out my hands and wrists, the pain went away quickly! If you are not diagnosed with carpal tunnel, and not already doing this, definitely try it, it saved my career.

benterris 7y ago
Could you please elaborate on the exercises you're finding helpful ? I have a mild RSI myself (outer part of the forearm, near the elbow) and have been trying some eccentric exercises for a few months now but I'm not seeing a big improvement.
- ajmarcic 7y ago
  I list the low-hanging fruit first: Look up nerve flossing exercises on YouTube. Routinely doing these had the largest impact for me. You'll feel the ones that work on whatever nerve is inflamed. Try to improve your posture. My general mantra is "lift your head as much as possible. Pull your shoulders back". It gets easier eventually. If your head lies forward from the spine you have a hunch. You may notice a small lump of muscle behind your neck. That's bad. There are exercises to try and strengthen the opposite muscles. If you sleep on your arms try to stop as well. I recommend sleeping on your back. Fluffy couches and back rests sacrifice posture. Don't use a laptop in bed. Sleep, eat nutrient-dense foods, and run or swim. Avoid alcohol. If you do pushups or bench press, make sure to exercise your upper back equally to avoid imbalance.
- untoreh 7y ago
  I use a powerball
7y ago

mikob 7y ago

I have a free Chrome Extension that let's you control the browser via voice: https://www.lipsurf.com https://www.lipsurf.com Rest is ultimately the best way to prevent hand injuries and since I spend most of my time in Chrome, this extension lets me do it hands free.