6 ms·
Hands-Free Coding (2020)
- themanmaran 3y agoThe demo is incredible. Thinking of adding Talon to my daily workflow simply for the speed improvement. It would be awesome to speak out commands that are slow to type. And honestly, with the custom python scripts support, this seems like a perfect use case for some GPT enabled programming. Something like "command: gpt add react button component".
- jasfi 3y agoI don't want to have to speak out anything, but it could be useful as an option. E.g. the author has an RSI.
- 55555 3y agoIf every developer at least pretends to embrace verbal programming, they'll have to get rid of open floor plans.
- willjp 3y agoGood riddance! ;)
- qwertox 3y agoI can't currently see myself dictating code or code intentions into an editor, but to a chat system, where I can say, "with the code I just copied over to you, ..." or "with the code which is selected in the editor, ..." and then discuss the problem verbally. Because it gets somewhat tiresome to write out all those sentences.
- ly3xqhl8g9 3y agoThe demo looks primitive, "go up second and go left third"? What is this, 2020? When now, in 2023, you can say "add parameters". Programming will only become higher and higher in abstraction. Why even say "parameters", just say "handle this use case".
- Projectiboga 3y agoThe article is dated 2020 in the title.
- ly3xqhl8g9 3y agoYes, that was the joke. Only 2-3 years have passed and yet we have advanced so far past that point, a sign of the accelerating things that are yet to come up to full abstraction over the machine interaction: just say what you want to achieve, let the machine handle the details, from what dependencies to use to the file structure of the repository.
- mbork_pl 3y ago> When someone says “I want a programming language in which I need only say what I wish done,” give him a lollipop. (Alan Perlis) ;-)
- ly3xqhl8g9 3y agoMr. Perlis died in 1990 unfortunately [1] [2], could just as well been in the prehistoric times. Of course, LLMs will not be able to give alone such a programming language, call it JUST, just say what you wish, but 5 or 6 more quantum leaps at the same level as the transformer architecture? I just can't see how it doesn't go that way. AlexNet [3] was just 10 years, 9 months and 17 days ago, September 2012. The same time from now it would be the year 2034, February. Can anyone really dare to say we won't have the JUST language by 2034? I'll raise you even one further: in 30-40, hopefully 20, years we will have a JUST anatomical compiler and we will say programatically to the cells of various bodies: just build an eye here [4]. [1] https://en.wikipedia.org/wiki/Alan_Perlis https://en.wikipedia.org/wiki/Alan_Perlis [2] The lolliquote is from 1982, Epigrams on Programming, epigram 93, http://www.cs.yale.edu/homes/perlis-alan/quotes.html http://www.cs.yale.edu/homes/perlis-alan/quotes.html [3] https://en.wikipedia.org/wiki/AlexNet https://en.wikipedia.org/wiki/AlexNet some prior work was 2011, Dan C. Cireșan et al., "High-Performance Neural Networks for Visual Object Classification", also quoted in the AlexNet paper. Both papers relied on "Convolutional Neural Networks designs introduced by Yann LeCun et al. (1989)". Curious if Mr. Perlis knew about Mr. LeCun's work. Here is Jürgen Schmidhuber giving a talk about the paper in October 2011, https://www.youtube.com/watch?v=rkCNbi26Hds https://www.youtube.com/watch?v=rkCNbi26Hds The 10-year-old YouTube comment "I was still under the impression neural networks were rarely used any longer." is particularly funny. [4] "The Collective Intelligence of Cells During Morphogenesis with Dr. Michael Levin", https://youtu.be/p4Fm7jLNrpg?t=3898 https://youtu.be/p4Fm7jLNrpg?t=3898 from 1:04:58 to 1:07:30, but the entire presentation is showing the future that is to come, in biology, in programming, in bio-programming.
- diginova 3y agoIt's awesome to realize that lot's of products and services are surrounded by the thought of how can most people, or how can the general public best use it etc. But in reality solving a problem for specific use cases can later on be extended to general purpose products and become a good business. Lot's of successful companies started of just by someone trying to solve his own, or some colleague/friend/relatives specific problem
- 55555 3y agoOkay, now I want to use voice commands to iteratively code using AI. You'd start out very high level, like, "Write a react component that does X thing." and then you'd modify it more granularly, like "Re-write the last 5 lines so it doesn't use variable X." I could probably do 90% of my coding without typing.
- cassianoleal 3y agoIntegrate that with the IDE and let the integration modify and save files, commit and push, etc., then make it speak back to you with the voice of Majel Barrett and you got a winner. - "Computer, run unit tests on this codebase." <processing noises> - "Unable to comply."
- avinassh 3y agoRelatedly, are there any accessibility tools to reduce eye usage while coding? I frequently get dry eyes and wonder in what other ways I can use them less. I follow the 20 20 20 exercises for eyes; it has been helpful. I don't want to shut them off completely, but I want to reduce them as much as possible. e.g. I am actively shutting down my monitor and using screenreaders for any long-form articles.
- jna_sh 3y agoI would highly recommend checking out https://www.cursorless.org/ https://www.cursorless.org/, an editor for voice built on Talon (what’s being used here by Josh) and a VSCode extension. If for nothing else than to watch the dev live code with it.
- qumpis 3y agoLooks awesome. Is anyone using it in their everyday workflow (I guess those who WFH)? This seems a bit better than the OP post, with some structure and shortcurs of navigating between code. I think writing comments and later structuring them could be useful in cases where you need to think quite a bit before writing code. Or jumping between search results, opening tabs (tiling on windows through voice doesn't sound bad). Pity there's no Jetbrains support.
- johndough 3y ago> If for nothing else than to watch the dev live code with it. YouTube channel https://www.youtube.com/@PokeyRuleJams/videos https://www.youtube.com/@PokeyRuleJams/videos
- willjp 3y agoTalon is such an exceptional piece of software. I’ve donated to the project for a few years now. I’m so grateful software like this exists, that it’s cross platform, and that it’s extremely customizable.
- rickydroll 3y agoAs someone with RSI who writes code using speech recognition, before copilot, something like Talon was a pretty good solution. Awkward, kinda rough on the throat, not something you would use in the long term but better than the other options. I've always thought that a editor based on the abstract syntax tree in the language was a far better solution but that's a hard to solve problem and needs to be redone with every language. I now use copilot almost exclusively when writing code. I've learned how to give it prompts so that writes code the way I would've written code. Occasionally it surprises me but when I dig into the solution, I realize that's a good one and start using it intentionally. That said, navigation still a pain (sometimes literally). I mostly use the arrow keys and page up/page down to get to where I want to go. It is usually not worth the hassle to build special macros for editor specific navigation. I believe having meat macros like I developed with Emacs is no longer a win. Maybe if I had abstract syntax tree driven navigation, it would be worthwhile.
- gymbeaux 3y agoIt's not beneficial to believe that you have "RSI". That's like saying you have "cancer" and are treating it with chemotherapy. What if you just had Thyroid Cancer, in which case you merely needed to take a special iodine pill. In other words, both cancer and "RSI" do not have a single treatment. RSI doesn't have to be this chronic, debilitating condition... but in my experience, you can't put too much faith in orthopedic docs/surgeons, either. I find orthobullets.com to be a very useful source of information on orthopedic conditions. Give it a look and see if your symptoms really do align... https://www.orthobullets.com/hand/6021/cubital-tunnel-syndrome https://www.orthobullets.com/hand/6021/cubital-tunnel-syndro...
- rickydroll 3y agoI've been living with this problem since 1994. Some of the symptoms match up with cubital tunnel syndrome in that my hands feel split along the middle finger ring finger line. At the same time, the palms hands feel like they are burning, and my shoulders are caused me low-level discomfort. If I overdo it, the pain migrates up into my neck. sometimes my forearms and fingers swell. When the finger swelling happens, I lose mobility in my fingers. I've also lost fine motor control and can no longer do things like write. One of the drugs they put me on gave me tremors that have not gone away so now I can no longer write or use a soldering iron. Sometimes tremors are bad enough typing is literally hit or miss. I miss juggling and cartooning. For the first 10 years I chased doctor after doctor hand surgeon orthopedic surgeon etc. looking for a cure. Nobody gave me consistent diagnosis and even some of the recommended surgical interventions for some of my symptoms were not recommended because I didn't present in a way that indicated the surgery would be successful. I've tried numerous things acupuncture, physical therapy, trigger point massage, and drugs like gabapentin and tricyclic antidepressants. The only thing that gave me any relief whatsoever was lidocaine injections in trigger points around my shoulders. The release lasted only 48 hours but it was so wonderful. I've given up that there's any medical intervention that will help and I just have to live with it the rest of my life.
- mbork_pl 3y agoRelated: https://youtu.be/8SkdfdXWYaI https://youtu.be/8SkdfdXWYaI (2013)
- gymbeaux 3y agoI'm not a doctor, but I'm skeptical that the author has/had Cubital Tunnel Syndrome. Similar to Carpal Tunnel Syndrome, it isn't a condition that significantly improves, much less resolves with rest. Both conditions warrant surgery to remove the nerve compression. I would guess that the author has relapsed by now, unless he made a lifestyle change not mentioned in the article (e.g. more ergonomic desktop, or started regularly exercising and stretching). I will add too that I have yet to find a study that supports the notion that Carpal Tunnel Syndrome is caused by computer use. Vibrating tools are the main culprit, along with genetics.
- woolion 3y agoEven if you were a doctor, making a diagnosis upon reading a post is not good practice. It is easy to find subjects who did nothing else but the suspected activity develop RSI, so it is a weird claim to suspect it might not exist. For computers in particular, the "emacs pinky" is common knowledge, unless you'd think people suffer from collective hallucination (maybe a side-effect of emacs?). It is definitely a condition that improves with rest, even if not totally and over a long period (months rather than days). There is also some level of evidence that suggest that psychological factors have an influence (there are a few obvious potential links that would explain it, but I'm not familiar enough with the research).
- eimrine 3y ago> For computers in particular, the "emacs pinky" is common knowledge, unless you'd think people suffer from collective hallucination (maybe a side-effect of emacs?) I really like the name, considering emacs as "Escame Meta Alt Control Shift" abbreviation. All of that keys typically pressed by pinky.
- gymbeaux 3y agoEh, read up on Cubital Tunnel Syndrome... it's illogical to expect "ergonomics" to resolve a muscular impingement of the ulnar nerve. "RSI" isn't a thing. It refers to an umbrella of conditions and therefore doesn't have a single "cure".
- deleted 3y ago
- DarthNebo 3y agomacOS has built in head tracking & control with expressions or additional hardware switches
- fouric 3y agoAs someone who may slowly be working his way toward RSI, I'd like to try to mitigate it before I get to the point of the author. To that end, I'd like to try to build a maximally ergonomic setup (for the back and neck in addition to the hands) that doesn't necessarily have to conform to the standard chair+desk setup (e.g. a custom setup where one lies on their back with their monitor suspended above their face and their hands lying to their sides with a split keyboard). In order to do that, I need the principles of ergonomics for the hands/wrists/arms, neck, and back - something that's been very challenging to find on the internet, as most of my search results are either trivial knowledge, blogspam, or just regurgitate the standard advice about various joints being at right angles without revealing the principles that drive that advice. Where can I find those principles?
- dnsco 3y agoMy understanding is that any position for too long becomes un-ergonomic, and really our bodies crave variability. I do my most productive coding on my couch in a weird slouching position, but I try not to do it for too long. I'll frequently switch back to my desk, and then try and take as many calls as possibly walking around the neighborhood. I can't focus on code at a standing desk or sitting on a yoga ball or whatever other people prescribe, but if you can throw that into your mix go for it, but sitting on a yoga ball all day is probably as deleterious as doing anything else all day. There are definitely different ways to use your body with different effects, but I think the problem is over use of any specific form.
- fouric 3y agoI appreciate your attempt to help, but what works for your body may not work for others - I'm looking for general ergonomic principles backed up by sound scientific evidence.
- radlad 3y agoHave you started here? https://scholar.google.com/scholar?q=anatomy+ergonomics https://scholar.google.com/scholar?q=anatomy+ergonomics
- evnix 3y agoI feel the nerve related situation is getting worse because of tiny modern keyboards which focus on aesthetics and sound. I have noticed split keyboard users in my office have better posture throughout the day whereas those tiny mechanical keyboard users droop their shoulders close and end up sliding down the chair putting their spine in a very awkward position promoting more severe medical conditions. It could be just that split keyboard users care more about their posture but from my observation it seems everyone starts with a good posture in the morning but tiny keyboard users end up with a very bad posture within a few hours. Would be interesting to know what others are seeing
- spit2wind 3y agoDeveloped RSI and overcame it through learning stenography with Plover. I've used it exclusively (only using the keyboard to login to the computer) for 2 years. I'm a full time developer and it works great. There's a learning curve for a few months before being able to switch to stenography completely. Practice was running drills with a metronome for 15 minutes or so a day, progressing to the next lesson after hitting my target of 70 wpm at 95% accuracy. Comments (generally on this topic) often make it seem harder than it is or raise scares about impracticality. Practice was boring. However, I found that 30 wpm is plenty sufficient to do programming on the job, as thinking takes far more time than putting characters on the screen. Very rarely, at least for me, am I writing continuously for more than a moment. Stenography is a far superior input method than 1:1 character-wise entry. Given my condition, I didn't really have a choice. If keyboard entry works for you, great. If you have RSI or are developing it, learning steno is 1000% worth it. Prejudice against invisible disabilities exists. People dismiss them, judge you, and take advantage of your situation. Stenography is a way out of that. Also, see an occupational therapist (a professional who specializes in repetitive, work-related injuries). Good luck to anyone going through RSI. You're not alone and you will get through it. Link to Plover to save you (potentially) painful typing: https://www.openstenoproject.org/ https://www.openstenoproject.org/
- rustyminnow 3y agoI recently started learning Plover! Did you have any trouble using it for coding? It feels largely geared towards natural langauge and seems like there could be challenges using it for code. Though I haven't practiced enough to try yet.
- hiatus 3y agoBefore settling on steno had you tried alternative keyboard layouts? Tarmak has relieved a lot of the stress for me but as time goes on I feel I may need a more drastic measure.
- revilotom 3y agoHey do you mind sharing which keyboard you have used for stenography? (Im looking for something that has super low activation force switches)