3 ms·

I'm the creator of Talon. You're exactly right about context sensitivity, and you're wrong that I'm not designing for it. One of Talon's secret missions is to m

I'm the creator of Talon. You're exactly right about context sensitivity, and you're wrong that I'm not designing for it. One of Talon's secret missions is to make extreme context sensitivity possible. I've done a lot of things at a low technical level to make this possible eventually, such as: - You can change the entire voice grammar from anything to anything in a couple of milliseconds. - I'm building a scope system, kinda like syntax scopes in a text editor like Sublime, but for arbitrary systemwide stuff. So you'll be able to say "this command activates on scope 'code.ruby.method' and 'filename matches test_*.rb" (which takes advantage of the fast grammar swap feature). Getting really specific with commands and action implementations means heavy lifting can be done for you in a lot of really cool cases, and recognition accuracy goes way up. It's also hard to remember all this stuff, one fix is to use floating UI that shows you the most relevant context-specific commands. As far as "generally accessible", another goal of Talon is to preempt RSI and find ways to convince people who can type fine to use it anyway. Two ways are "make many things way faster/smoother than typing" and "lots of cool eye tracking features".

khitchdee 8y ago
I think the scope system is a step in a useful direction. The speed at which you change a grammar is not really that relevant because typically, you don't program in more than one language. Being able to do it is. When you start with a generic system that can be configured down to specialized use-cases, sometimes, your focus on the end user's perspective of your tool gets clouded by your own desire to engineer in lots of bells and whistles since you have a system that can do that. From the end user's perspective, the specifics matter first. Sounds like you're making it easy to create the equivalent of macros to abbreviate the input of common tasks. It would be interesting to see if you could make this system function without a screen so even a blind person could produce code.
- lunixbochs 8y ago
  I've definitely considered blind UI, it's a different sort of paradigm. It should be nice for other cases, like using a computer with just an earpiece. It turns out some of my approach will make fully blind voice UI much easier, even on existing apps / environments. You definitely don't understand the implications of highly dynamic grammars then. A couple instances where rapid grammar changes matter are: "I want to switch between my text editor, a terminal, and a browser, and have exactly the most specific and relevant commands available at all times with no delay", and "I want to have syntax-aware voice grammars that are very in tune with how my programming language and framework operate" You appear to be talking about this without any significant research, sources, or knowledge of my software (which you have multiple times made fairly confident statements about that are not remotely true), please stop generalizing about what I'm doing without some examples to back it up.
  - khitchdee 8y ago
    Re: grammar changes: I assumed each language was associated with its grammar and it was not more fine grained than that. This makes sense though in the context where you're simply trying to replicate the keyboard-window-mouse based tool design with a speech-window-eye-tracking one. I guess this makes it easy for a user on the former UI to switch over. You will note my comments were not about your software, but hands-free programming interfaces in general. They continue to be this way, you will note. I am addressing the subject of this topic at a high level. I also have a fairly good idea on where your software stands within that context. Don't need to do any research for that.
  - deleted 8y ago
    [deleted]
  - khitchdee 8y ago
    I guess the fact that you have a delay associated with the recognition of commands not loaded into the recognizers current vocabulary is an artifact associated with the design of the recognizer which was probably tuned towards continuous speech. In other words your dynamic grammars idea was a way to work around a limitation of the speech recognizer you were using. Part of the benefit of designing a language specifically for speech based coding is that the design of the associated recognizert could be tuned to that language. Compared to a brute force translated version of the keyboard input based language developer workflow, a speicifcally designed language would likely have a far narrower vocabulary. This would make it possible to tune a recognizer for better recognition accuracy and speed.
    - deleted 8y ago
      [deleted]

khitchdee 8y ago

I think the scope system is a step in a useful direction. The speed at which you change a grammar is not really that relevant because typically, you don't program in more than one language. Being able to do it is. When you start with a generic system that can be configured down to specialized use-cases, sometimes, your focus on the end user's perspective of your tool gets clouded by your own desire to engineer in lots of bells and whistles since you have a system that can do that. From the end user's perspective, the specifics matter first. Sounds like you're making it easy to create the equivalent of macros to abbreviate the input of common tasks. It would be interesting to see if you could make this system function without a screen so even a blind person could produce code.

lunixbochs 8y ago
I've definitely considered blind UI, it's a different sort of paradigm. It should be nice for other cases, like using a computer with just an earpiece. It turns out some of my approach will make fully blind voice UI much easier, even on existing apps / environments. You definitely don't understand the implications of highly dynamic grammars then. A couple instances where rapid grammar changes matter are: "I want to switch between my text editor, a terminal, and a browser, and have exactly the most specific and relevant commands available at all times with no delay", and "I want to have syntax-aware voice grammars that are very in tune with how my programming language and framework operate" You appear to be talking about this without any significant research, sources, or knowledge of my software (which you have multiple times made fairly confident statements about that are not remotely true), please stop generalizing about what I'm doing without some examples to back it up.
- khitchdee 8y ago
  Re: grammar changes: I assumed each language was associated with its grammar and it was not more fine grained than that. This makes sense though in the context where you're simply trying to replicate the keyboard-window-mouse based tool design with a speech-window-eye-tracking one. I guess this makes it easy for a user on the former UI to switch over. You will note my comments were not about your software, but hands-free programming interfaces in general. They continue to be this way, you will note. I am addressing the subject of this topic at a high level. I also have a fairly good idea on where your software stands within that context. Don't need to do any research for that.
- deleted 8y ago
  [deleted]
- khitchdee 8y ago
  I guess the fact that you have a delay associated with the recognition of commands not loaded into the recognizers current vocabulary is an artifact associated with the design of the recognizer which was probably tuned towards continuous speech. In other words your dynamic grammars idea was a way to work around a limitation of the speech recognizer you were using. Part of the benefit of designing a language specifically for speech based coding is that the design of the associated recognizert could be tuned to that language. Compared to a brute force translated version of the keyboard input based language developer workflow, a speicifcally designed language would likely have a far narrower vocabulary. This would make it possible to tune a recognizer for better recognition accuracy and speed.
  - deleted 8y ago
    [deleted]