4 ms·

While I agree with 99% of what you said… > I’d say it’s a mouse-driven IDE with a YAML-like code. But in its current guise, I can see people itching to use the

by szvsw 2y ago

While I agree with 99% of what you said… > I’d say it’s a mouse-driven IDE with a YAML-like code. But in its current guise, I can see people itching to use the keyboard for speed once they know what text pieces they want. It’s useful to keep in mind that a “mouse-driven” IDE could have some real benefits from an accessibility perspective. I agree that this generally might miss the mark from a variety of goals, especially compared to graph based dataflow programming languages (see my other comment in this thread), and to really maximize accessibility as a goal you would need to actually design it for that from the ground up, but it’s still good to see experiments with other modalities and raise the question. At the end of the day the fraction of programmers who have difficulty typing is low… but that doesn’t mean it should be ignored, especially considering that one day almost all of us on here will have trouble typing, but likely still have the urge to code!

smaudet 2y ago
As I said in another comment, I think the part of visual languages their devs are blind to (pun intended) are the input method(s). We can all agree that keyboards are a) large b) necessitate being able to type (need hands) but we don't have that many high-frequency out-of-the-way input methods in computing, generally. I want to see an editor that can a) re-use existing code (libraries, concepts, etc) but that can b) use mouse, keyboard, touchscreen, hand gloves, <insert new input method here>. Drawing boxes on screens and sticking text inside them is just not impressive, and its clunky to boot. Any worth is whatever input method is provided, and TBH I wouldn't expect this to be any easier on e.g. mobile, probably worse, in fact.
- szvsw 2y ago
  Yeah, the high frequency input capabilities of hands/fingers on keyboard are pretty remarkable and hard to beat, especially in conjunction with navigation tools like vim and especially when you start considering things like stenography applied in a programming context (intellisense/copilot etc can in some ways be be considered a form of that already). I suspect natural language speech is too slow, but i do sometimes wonder if the only other part of our body capable of such high throughput (and the requisite fine motor control) is our mouth/tongue… I’ve always wondered what a programming language optimized for spoken input would look/sound like. Edit: mentioned this in another comment, but I think graph based visual programming languages would actually be pretty well suited to speech entry, especially with shorthands for common operations like “create node” := “kurr” or “connect” := “kuh”.
  - cess11 2y ago
    Typing speed seems like a rather junior concern. In practice most time tends to be spent thinking or cycling through edits and inspecting test results. I think putting your tests in a watch -n 2 on the second screen would be a much better improvement than fiddling with some reinvention of stenography.
    - smaudet 2y ago
      This is still not helped by other input methods, stenography helps with the navigation too... Ctrl clicking though symbols is all well and good but you often want or need to do much more complicated things than that. So, yes, input speed is still a problem for reading and understanding code, too.
    - szvsw 2y ago
      I agree that typing probably is not the limiting factor in most cases and for most people, and so the conversation might seem a little silly… But that’s precisely because typing is fast. Once you know what you are doing, you start typing and you are done relatively quickly, and probably spent a lot of the time typing thinking about what you would do next… but we are specifically talking about a scenario where typing is the limiting factor, or even entirely infeasible, for a variety of reasons.
  - conartist6 2y ago
    Check out https://www.cursorless.org https://www.cursorless.org
hnlmorg 2y ago
Accessibility is a good point. Though I can’t think of any disabilities where I couldn’t type but could drag and drop components, which is a much more precise as well as sustained physical motion. I suspect there are better ways for coding in those scenarios. Like perhaps the kind of word entry that Stephen Hawkin used. Or if you were to go totally touch UI (I’ve known one individual with motor disabilities use an old touch screen CRT with a stylus in their mouth) then you’d want those blocks a little taller but with something more akin to Miro’s infinite zoom. Rather than a text UI with mouse input. That said, every project has to start somewhere. So this might be that proof of concept needed to fully flesh out accessibility tweaks.
- szvsw 2y ago
  Yeah good points. You can easily imagine composing graph based/data flow programming languages with just your mouth, ie “create node:<type>:<name>; connect node:<name>:<outlet> to node:<name>:<inlet>” as a relatively easy way to synchronize a visual programming language’s traditional mouse-based construction engine to speech based construction with realtime visual updates etc.

by szvsw 2y ago

smaudet 2y ago

As I said in another comment, I think the part of visual languages their devs are blind to (pun intended) are the input method(s). We can all agree that keyboards are a) large b) necessitate being able to type (need hands) but we don't have that many high-frequency out-of-the-way input methods in computing, generally. I want to see an editor that can a) re-use existing code (libraries, concepts, etc) but that can b) use mouse, keyboard, touchscreen, hand gloves, <insert new input method here>. Drawing boxes on screens and sticking text inside them is just not impressive, and its clunky to boot. Any worth is whatever input method is provided, and TBH I wouldn't expect this to be any easier on e.g. mobile, probably worse, in fact.

szvsw 2y ago
Yeah, the high frequency input capabilities of hands/fingers on keyboard are pretty remarkable and hard to beat, especially in conjunction with navigation tools like vim and especially when you start considering things like stenography applied in a programming context (intellisense/copilot etc can in some ways be be considered a form of that already). I suspect natural language speech is too slow, but i do sometimes wonder if the only other part of our body capable of such high throughput (and the requisite fine motor control) is our mouth/tongue… I’ve always wondered what a programming language optimized for spoken input would look/sound like. Edit: mentioned this in another comment, but I think graph based visual programming languages would actually be pretty well suited to speech entry, especially with shorthands for common operations like “create node” := “kurr” or “connect” := “kuh”.
- cess11 2y ago
  Typing speed seems like a rather junior concern. In practice most time tends to be spent thinking or cycling through edits and inspecting test results. I think putting your tests in a watch -n 2 on the second screen would be a much better improvement than fiddling with some reinvention of stenography.
  - smaudet 2y ago
    This is still not helped by other input methods, stenography helps with the navigation too... Ctrl clicking though symbols is all well and good but you often want or need to do much more complicated things than that. So, yes, input speed is still a problem for reading and understanding code, too.
  - szvsw 2y ago
    I agree that typing probably is not the limiting factor in most cases and for most people, and so the conversation might seem a little silly… But that’s precisely because typing is fast. Once you know what you are doing, you start typing and you are done relatively quickly, and probably spent a lot of the time typing thinking about what you would do next… but we are specifically talking about a scenario where typing is the limiting factor, or even entirely infeasible, for a variety of reasons.
- conartist6 2y ago
  Check out https://www.cursorless.org https://www.cursorless.org

hnlmorg 2y ago

Accessibility is a good point. Though I can’t think of any disabilities where I couldn’t type but could drag and drop components, which is a much more precise as well as sustained physical motion. I suspect there are better ways for coding in those scenarios. Like perhaps the kind of word entry that Stephen Hawkin used. Or if you were to go totally touch UI (I’ve known one individual with motor disabilities use an old touch screen CRT with a stylus in their mouth) then you’d want those blocks a little taller but with something more akin to Miro’s infinite zoom. Rather than a text UI with mouse input. That said, every project has to start somewhere. So this might be that proof of concept needed to fully flesh out accessibility tweaks.

szvsw 2y ago
Yeah good points. You can easily imagine composing graph based/data flow programming languages with just your mouth, ie “create node:<type>:<name>; connect node:<name>:<outlet> to node:<name>:<inlet>” as a relatively easy way to synchronize a visual programming language’s traditional mouse-based construction engine to speech based construction with realtime visual updates etc.