5 ms·

Seamless Streaming looks really promising! We just had a new employee start a few months back with profound hearing loss and our company had no idea what to do

Seamless Streaming looks really promising! We just had a new employee start a few months back with profound hearing loss and our company had no idea what to do with him from an accessibility standpoint. They threw out solutions like Dragon, not realizing those solutions are not real-time. He ended up rolling his own solution by standing up Whisper in one of our clusters and writing a basic front end and API to take his laptop’s mic input and chunk it every few seconds to send to the model and get back text in pseudo-realtime. We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs. I can’t wait to see what he does with these new models!

kylixz 3y ago
I recommend checking out: https://talonvoice.com/ https://talonvoice.com/
- FloatArtifact 3y ago
  It's not open source nor does the author intend to open the stack.
cgb223 3y ago
Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware and let them roll an accessibility solution We need more support for employees like this!
- cced 3y ago
  Second this! Also, what about Apple’s latest M3 series chips? Are this in the same realm as Alienware in terms of AI compute?
  - jackson1442 3y ago
    I think generally the consensus of Apple Silicon is that they're great _for a laptop_, but still aren't going to beat a dedicated graphics card + high-end CPU like i9/Ryzen 9. Biggest thing going for apple is the performance/watt though which is critical for a laptop.
    - cjbprime 3y ago
      I think this is missing the main reason to use Apple Silicon, which is that your dedicated graphics card probably has 24GB or less of RAM, whereas e.g. an M2 Ultra Mac Studio can have 192GB of RAM with a far superior memory bandwidth to anything on x86. This is important because even a "small" LLM like Llama2 13B would require quantization to fit in the 24GB RAM that the dedicated graphics card will give you, whereas the Mac could run Llama2 70B without quantization (at FP16).
      - aftbit 3y ago
        Whisper doesn't need that much RAM though.
  - willy_k 3y ago
    They definitely are in terms of energy efficiency
  - nodja 3y ago
    They're better than most consumers x86 CPUs but worse than using a GPU. Where they shine is when the ML model can't fit the GPU's VRAM since you have better options for ram size with macs.
- romwell 3y ago
  >Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware So gracious, to give a software developer some hardware to run the software they need to work, that costs a whopping nothing more than what other people in the industry get on the average. >and let them roll an accessibility solution "You're such a good employer! You let your employee build their own accessibility ramp to the back entrance in their own time, and even got them a mortar spatula to do so!" We need more support for employees like this! >We need more support for employees like this! And less support for employers like this.
  - Solvency 3y ago
    Not sure why you're being downvoted. Literally the equivalent of building your own ramp.
    - freedomben 3y ago
      I didn't downvote, but I considered doing so because nowhere that I saw in GP does it say in his own time, and that's a critical piece of the equation. Hallucinating that datum means they got the argument wrong, and worse they were harshly critical of the company based on that wrongly assumed information. It reminds me of the Homer Simpson quote, "I don’t mind being called a liar when I’m lying, or about to lie, or just finished lying, but NOT WHEN I’M TELLING THE TRUTH!" I would be equally critical if it was warranted, but when it isn't it's deeply unfair to the accused. If the person wanted to build their own ramp, and the employer let them do it on the clock, that's a completely different scenario than the employee having to come in during their off-hours to build the ramp just so they can go to work.
      - navbaker 3y ago
        Yeah, it wasn’t on his own time. He had a full budget and this was right in line with stuff he had already done research in anyway, so he just went for it.
        3y ago
qkeast 3y ago
Awesome! I love hearing about places making the effort to be inclusive. As someone who’s profoundly deaf myself, another less technical approach is to install Rogue Amoeba’s Loopback, and use it to pipe audio from a given app into a tool like Google Meet or Otter.ai using the Loopback device as the audio source. This effectively provides real time captions for anything running on your existing machine.
- tuukkah 3y ago
  Clever use of Google Meet as a tool! Also, Google Pixel phones now provide realtime captions to any speech playing on the phone (Accessibility > Live Caption). You can also choose a "preferred language" and the captions will be automatically translated to that language from other languages.
- jallmann 3y ago
  Google Chrome [1] also has captioning built-in [2], so this could also work from a plain page that hooks into the loopback device. Pretty sure it's using the same TTS backend that Google Meet uses. The nice thing about Chrome feature is you can move the caption box around and keep it in the foreground while doing other things, although styling options seem limited (the text might be a little small for some). [1] on desktop, not sure about mobile [2] via chrome://settings/accessibility -> Live Caption
- romwell 3y ago
  >Awesome! I love hearing about places making the effort to be inclusive. The extent of the effort being getting their employee a slightly-more-expensive-than-average tool that would enable them to do their job better regardless of the disability? Such inclusive, much pat-yourself-on-the-back, wow. "We gave our woodworking shop employee a quality saw so that they'd make their own accessibility ramps!"
  - callalex 3y ago
    What would you have them do instead?
pawelduda 3y ago
That's very nice of you
- romwell 3y ago
  >He ended up rolling his own solution >That's very nice of you ...doesn't compute. What exactly was nice here?
  - diab0lic 3y ago
    > We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs. Probably this.
    - romwell 3y ago
      As opposed to what, an IBM 286 that everyone else gets? An "Alienware" is not a Ferrari, those things cost about the same as any other professional-level workstations.
lovich 3y ago
Y’all should turn that into a product, or at least open source it and get the positive PR + helping others
- FloatArtifact 3y ago
  > Y’all should turn that into a product, or at least open source it and get the positive PR + helping others There you go. https://github.com/dictation-toolbox/dragonfly https://github.com/dictation-toolbox/dragonfly
deleted 3y ago
[deleted]
aftbit 3y ago
Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor.
- FloatArtifact 3y ago
  > Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor. Simply voice to text is not what's needed for dictating commands. Unless I can load commands of on the fly and decode utterances that may be useful. The client would need to be able to send its commands to the server on the fly.
FloatArtifact 3y ago
Problem with whisper is its not really optimized for command recognition versus general dictation. - Whisper processes 30 second audio chunks. So if you process 5 seconds of audio you have to pad it out with 25 seconds of silence. Hence a loss of efficiency with wasted CPU / GPU cycles on 25 seconds per chunk in the case above. - Whisper most likely can't handle hundreds of commands much less than a thousand performantly. - Whisper doesn't handle short commands very well with a degree of accuracy post processing commands from free dictation utterances. Command dictation should be weighted higher than general dictation when decoding. I work with a little under 1500 of commands dragon naturally speaking. DNS is hot garbage as a program despite it has the best accuracy to date with the feature of commands and dictation in one utterance. You get to pay $750 for the privilege m I've yet to see a free and open source speech recognition engine that can handle both dictation and commands with a high degree of accuracy. Please please let me know if there's alternatives out there. I would definitely pay to support an open source project like this that focuses on command and dictation. Most solutions out there that are open source nowadays focus so much on iot command recognition with intents. That's not well suited for controlling your computer with grammars containing voice commands.
- novok 3y ago
  Is 30s the input size set by the model, or programs that wrap the model? Is it how it's trained?
  - bakkoting 3y ago
    It's a property of the model itself. > Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder. https://openai.com/research/whisper https://openai.com/research/whisper
sagz 3y ago
Do they need realtime transcription? Computer: webcaptioner.com Android: Live Transcribe (g.co/livetranscribe) iOS: Live Caption with the 'mic' icon enabled. Web conferencing: Meet, Zoom, Teams all support realtime CC, which is pretty good.

kylixz 3y ago

I recommend checking out: https://talonvoice.com/ https://talonvoice.com/

FloatArtifact 3y ago
It's not open source nor does the author intend to open the stack.

cgb223 3y ago

Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware and let them roll an accessibility solution We need more support for employees like this!

cced 3y ago
Second this! Also, what about Apple’s latest M3 series chips? Are this in the same realm as Alienware in terms of AI compute?
- jackson1442 3y ago
  I think generally the consensus of Apple Silicon is that they're great _for a laptop_, but still aren't going to beat a dedicated graphics card + high-end CPU like i9/Ryzen 9. Biggest thing going for apple is the performance/watt though which is critical for a laptop.
  - cjbprime 3y ago
    I think this is missing the main reason to use Apple Silicon, which is that your dedicated graphics card probably has 24GB or less of RAM, whereas e.g. an M2 Ultra Mac Studio can have 192GB of RAM with a far superior memory bandwidth to anything on x86. This is important because even a "small" LLM like Llama2 13B would require quantization to fit in the 24GB RAM that the dedicated graphics card will give you, whereas the Mac could run Llama2 70B without quantization (at FP16).
    - aftbit 3y ago
      Whisper doesn't need that much RAM though.
- willy_k 3y ago
  They definitely are in terms of energy efficiency
- nodja 3y ago
  They're better than most consumers x86 CPUs but worse than using a GPU. Where they shine is when the ML model can't fit the GPU's VRAM since you have better options for ram size with macs.
romwell 3y ago
>Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware So gracious, to give a software developer some hardware to run the software they need to work, that costs a whopping nothing more than what other people in the industry get on the average. >and let them roll an accessibility solution "You're such a good employer! You let your employee build their own accessibility ramp to the back entrance in their own time, and even got them a mortar spatula to do so!" We need more support for employees like this! >We need more support for employees like this! And less support for employers like this.
- Solvency 3y ago
  Not sure why you're being downvoted. Literally the equivalent of building your own ramp.
  - freedomben 3y ago
    I didn't downvote, but I considered doing so because nowhere that I saw in GP does it say in his own time, and that's a critical piece of the equation. Hallucinating that datum means they got the argument wrong, and worse they were harshly critical of the company based on that wrongly assumed information. It reminds me of the Homer Simpson quote, "I don’t mind being called a liar when I’m lying, or about to lie, or just finished lying, but NOT WHEN I’M TELLING THE TRUTH!" I would be equally critical if it was warranted, but when it isn't it's deeply unfair to the accused. If the person wanted to build their own ramp, and the employer let them do it on the clock, that's a completely different scenario than the employee having to come in during their off-hours to build the ramp just so they can go to work.
    - navbaker 3y ago
      Yeah, it wasn’t on his own time. He had a full budget and this was right in line with stuff he had already done research in anyway, so he just went for it.
      - 3y ago

qkeast 3y ago

Awesome! I love hearing about places making the effort to be inclusive. As someone who’s profoundly deaf myself, another less technical approach is to install Rogue Amoeba’s Loopback, and use it to pipe audio from a given app into a tool like Google Meet or Otter.ai using the Loopback device as the audio source. This effectively provides real time captions for anything running on your existing machine.

tuukkah 3y ago
Clever use of Google Meet as a tool! Also, Google Pixel phones now provide realtime captions to any speech playing on the phone (Accessibility > Live Caption). You can also choose a "preferred language" and the captions will be automatically translated to that language from other languages.
jallmann 3y ago
Google Chrome [1] also has captioning built-in [2], so this could also work from a plain page that hooks into the loopback device. Pretty sure it's using the same TTS backend that Google Meet uses. The nice thing about Chrome feature is you can move the caption box around and keep it in the foreground while doing other things, although styling options seem limited (the text might be a little small for some). [1] on desktop, not sure about mobile [2] via chrome://settings/accessibility -> Live Caption
romwell 3y ago
>Awesome! I love hearing about places making the effort to be inclusive. The extent of the effort being getting their employee a slightly-more-expensive-than-average tool that would enable them to do their job better regardless of the disability? Such inclusive, much pat-yourself-on-the-back, wow. "We gave our woodworking shop employee a quality saw so that they'd make their own accessibility ramps!"
- callalex 3y ago
  What would you have them do instead?

pawelduda 3y ago

That's very nice of you

romwell 3y ago
>He ended up rolling his own solution >That's very nice of you ...doesn't compute. What exactly was nice here?
- diab0lic 3y ago
  > We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs. Probably this.
  - romwell 3y ago
    As opposed to what, an IBM 286 that everyone else gets? An "Alienware" is not a Ferrari, those things cost about the same as any other professional-level workstations.

lovich 3y ago

Y’all should turn that into a product, or at least open source it and get the positive PR + helping others

FloatArtifact 3y ago
> Y’all should turn that into a product, or at least open source it and get the positive PR + helping others There you go. https://github.com/dictation-toolbox/dragonfly https://github.com/dictation-toolbox/dragonfly

deleted 3y ago

[deleted]

aftbit 3y ago

Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor.

FloatArtifact 3y ago
> Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor. Simply voice to text is not what's needed for dictating commands. Unless I can load commands of on the fly and decode utterances that may be useful. The client would need to be able to send its commands to the server on the fly.

FloatArtifact 3y ago

Problem with whisper is its not really optimized for command recognition versus general dictation. - Whisper processes 30 second audio chunks. So if you process 5 seconds of audio you have to pad it out with 25 seconds of silence. Hence a loss of efficiency with wasted CPU / GPU cycles on 25 seconds per chunk in the case above. - Whisper most likely can't handle hundreds of commands much less than a thousand performantly. - Whisper doesn't handle short commands very well with a degree of accuracy post processing commands from free dictation utterances. Command dictation should be weighted higher than general dictation when decoding. I work with a little under 1500 of commands dragon naturally speaking. DNS is hot garbage as a program despite it has the best accuracy to date with the feature of commands and dictation in one utterance. You get to pay $750 for the privilege m I've yet to see a free and open source speech recognition engine that can handle both dictation and commands with a high degree of accuracy. Please please let me know if there's alternatives out there. I would definitely pay to support an open source project like this that focuses on command and dictation. Most solutions out there that are open source nowadays focus so much on iot command recognition with intents. That's not well suited for controlling your computer with grammars containing voice commands.

novok 3y ago
Is 30s the input size set by the model, or programs that wrap the model? Is it how it's trained?
- bakkoting 3y ago
  It's a property of the model itself. > Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder. https://openai.com/research/whisper https://openai.com/research/whisper

sagz 3y ago

Do they need realtime transcription? Computer: webcaptioner.com Android: Live Transcribe (g.co/livetranscribe) iOS: Live Caption with the 'mic' icon enabled. Web conferencing: Meet, Zoom, Teams all support realtime CC, which is pretty good.