4 ms·

I've been working on this exact thing for Chrome the last 3 years: https://www.lipsurf.com https://www.lipsurf.com Anyone can make an open source plugin for it

by mikob 6y ago

I've been working on this exact thing for Chrome the last 3 years: https://www.lipsurf.com https://www.lipsurf.com Anyone can make an open source plugin for it to do anything with voice (https://github.com/LipSurf/plugins https://github.com/LipSurf/plugins) I've wanted to port it to Firefox, but the HTML5 SpeechRecognition API (https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecognition https://developer.mozilla.org/en-US/docs/Web/API/SpeechRecog...) is still not available. Why not just make the API available and leave this in addon territory for all developers?

TedDoesntTalk 6y ago
> Available as an extension for Firefox on desktop / laptop According to article, Firefox is offering this through an addon... obviously not using the SpeechRecognition API, however.
kevingadd 6y ago
"On some browsers, like Chrome, using Speech Recognition on a web page involves a server-based recognition engine. Your audio is sent to a web service for recognition processing, so it won't work offline." Seems like the answer for why the API isn't in Firefox. It's also not standardized, and is prefixed, so...
aerovistae 6y ago
I built something like this 7 years ago, it's called Hands Free for Chrome, a now languishing project that I lost interest in a long time ago, unfortunately. It made the top 10 of HN back then though! My site's design is not nearly as nice as yours. https://www.handsfreechrome.com/ https://www.handsfreechrome.com/ I just didn't get enough users or support to really care about it. But I wish you the best. It was an exciting thing to build and using it always felt futuristic to me. This is just so fascinating though. It's like seeing what could have been if I had been a better developer and found the dedication to really stick to the project in the longterm. Edit: I see we had the exact same idea! Your "tag" is my "map." Love it. One big difference is that mine was just a free project. I'd be super interested to know how many users you've got. I never had more than ~1100. From looking through your website mine was a much less intensive project. (Oh, CWS says 4000+ for you....wow, wonder how many are paid.) Edit2: Looking over your update history is almost nostalgic. "Fixed issue with overlapping commands -- delaying commands that are partial matches of other commands." Had to do the exact same thing! Edit3: We have so many overlapping command names that I wonder if you took inspiration from my project, almost. Either that or it's just a case of convergent evolution. Edit4: Suggestion for dictation: a way to alternate between a special character and actually writing the word. Doesn't look like there's a way to do ^ vs carrot or & vs ampersand. Something like "Enter special character caret". Maybe you already have a plugin for this though, idk. Edit5: God, this is so well architected! plugins and contexts are just fantastic ideas for this domain. Click by voice using hidden search-for-text is also a perfect solution to that problem. I wonder if this could be made more intelligent, i.e. "Click Submit in the sidebar on the left"-- challenging though. Edit6: Wow, just noticed someone else built something called "Handsfree for Web" somewhere along the way and theirs is ALSO way better than what I had built. Geez. Starting to feel bad about my awful website.
- mikob 6y ago
  Never saw yours before, but I discovered "Handsfree for Web" a few months after I started - and thought he had ripped mine off. But I no longer think so. Yes, seems like many commands are the same. Shame that so much wheel reinvention is going on. One thing that makes LipSurf "special" is the deep integration with sites. I wanted to use Duolingo, Reddit, HN and some others more with voice - so they get special plugins. Doing Duolingo with voice is a game changer for language learning - and if it weren't for usecases like that I would have likely lost interest like you long ago.
  - aerovistae 6y ago
    Has this replaced your full time job?
  - TheSpiceIsLife 6y ago
    I want hands free for CAD. Imagine being able to vocalise and build a model. I did have a HN user who said they be happy to collaborate with me to build it but I dropped the ball and have since killed that email address.
    - franga2000 6y ago
      If you manage to get into the GPT-3 beta, I'd love to work on that with you :D. For simple models, English -> OpenSCAD sounds like it's doable given the things I've seen on Twitter and for normal modeling, GPT-3 would probably make an excellent intent recognizer for voice commands.
      - TheSpiceIsLife 6y ago
        How would I go about looking in to this? I'm a metal fabricator by trade but also technically minded, AutoHotKey scripts for a few things and can very basic Python and C# if I need to. I tend to learn in a very solutions oriented. And would be keen to collaborate / learn / skill share.
zaynb 6y ago
Hi @mikob, could I please ask what you used to create the SVG animation for your site?
andrenatalbr 6y ago
The Web Speech Api is available in Firefox Nightly behind a prefix. You can enable it this way: https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition

by mikob 6y ago

TedDoesntTalk 6y ago

> Available as an extension for Firefox on desktop / laptop According to article, Firefox is offering this through an addon... obviously not using the SpeechRecognition API, however.

kevingadd 6y ago

"On some browsers, like Chrome, using Speech Recognition on a web page involves a server-based recognition engine. Your audio is sent to a web service for recognition processing, so it won't work offline." Seems like the answer for why the API isn't in Firefox. It's also not standardized, and is prefixed, so...

aerovistae 6y ago

I built something like this 7 years ago, it's called Hands Free for Chrome, a now languishing project that I lost interest in a long time ago, unfortunately. It made the top 10 of HN back then though! My site's design is not nearly as nice as yours. https://www.handsfreechrome.com/ https://www.handsfreechrome.com/ I just didn't get enough users or support to really care about it. But I wish you the best. It was an exciting thing to build and using it always felt futuristic to me. This is just so fascinating though. It's like seeing what could have been if I had been a better developer and found the dedication to really stick to the project in the longterm. Edit: I see we had the exact same idea! Your "tag" is my "map." Love it. One big difference is that mine was just a free project. I'd be super interested to know how many users you've got. I never had more than ~1100. From looking through your website mine was a much less intensive project. (Oh, CWS says 4000+ for you....wow, wonder how many are paid.) Edit2: Looking over your update history is almost nostalgic. "Fixed issue with overlapping commands -- delaying commands that are partial matches of other commands." Had to do the exact same thing! Edit3: We have so many overlapping command names that I wonder if you took inspiration from my project, almost. Either that or it's just a case of convergent evolution. Edit4: Suggestion for dictation: a way to alternate between a special character and actually writing the word. Doesn't look like there's a way to do ^ vs carrot or & vs ampersand. Something like "Enter special character caret". Maybe you already have a plugin for this though, idk. Edit5: God, this is so well architected! plugins and contexts are just fantastic ideas for this domain. Click by voice using hidden search-for-text is also a perfect solution to that problem. I wonder if this could be made more intelligent, i.e. "Click Submit in the sidebar on the left"-- challenging though. Edit6: Wow, just noticed someone else built something called "Handsfree for Web" somewhere along the way and theirs is ALSO way better than what I had built. Geez. Starting to feel bad about my awful website.

mikob 6y ago
Never saw yours before, but I discovered "Handsfree for Web" a few months after I started - and thought he had ripped mine off. But I no longer think so. Yes, seems like many commands are the same. Shame that so much wheel reinvention is going on. One thing that makes LipSurf "special" is the deep integration with sites. I wanted to use Duolingo, Reddit, HN and some others more with voice - so they get special plugins. Doing Duolingo with voice is a game changer for language learning - and if it weren't for usecases like that I would have likely lost interest like you long ago.
- aerovistae 6y ago
  Has this replaced your full time job?
- TheSpiceIsLife 6y ago
  I want hands free for CAD. Imagine being able to vocalise and build a model. I did have a HN user who said they be happy to collaborate with me to build it but I dropped the ball and have since killed that email address.
  - franga2000 6y ago
    If you manage to get into the GPT-3 beta, I'd love to work on that with you :D. For simple models, English -> OpenSCAD sounds like it's doable given the things I've seen on Twitter and for normal modeling, GPT-3 would probably make an excellent intent recognizer for voice commands.
    - TheSpiceIsLife 6y ago
      How would I go about looking in to this? I'm a metal fabricator by trade but also technically minded, AutoHotKey scripts for a few things and can very basic Python and C# if I need to. I tend to learn in a very solutions oriented. And would be keen to collaborate / learn / skill share.

zaynb 6y ago

Hi @mikob, could I please ask what you used to create the SVG animation for your site?

andrenatalbr 6y ago

The Web Speech Api is available in Firefox Nightly behind a prefix. You can enable it this way: https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition https://wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition