4 ms·

Do you know of any workflows like this using entirely open source software? EDIT: Seems like caster itself has instructions for an open source recognition engi

Do you know of any workflows like this using entirely open source software? EDIT: Seems like caster itself has instructions for an open source recognition engine to pair with it. Not sure how accurate it'll be but I'm going to give it a shot! I hate the idea of relying on closed source software to be able to continue in my profession. If this works I'm definitely going to be donating to the FOSS options

whimsicalism 6y ago
My understanding is that Talon is built off of wav2letter's inference framework for ASR.
- lunixbochs 6y ago
  I do not use the wav2letter@anywhere inference frontend - I trained the acoustic model using the facebook upstream code, but the decoder is almost entirely new, and on Windows I use Pytorch for inference. Talon ships with a libw2l.so/dylib on Linux/Mac built from my open source repos.
  - whimsicalism 6y ago
    Interesting. What model are you using for your acoustic - the streaming convnet? I didn't know that there was a pytorch implementation of the w2l architectures..
  - nshm 6y ago
    Feels like wav2letter will not be actively developed anymore. Understandable since it is hard to compete with Pytorch with a custom NN toolkit. Any plans to move to Pytorch/Tensorflow?
    - lunixbochs 6y ago
      I don’t think that’s right at all. They moved development to the Flashlight repo. It seems very actively developed to me. Last commit 3 hours ago. wav2vec 2.0 blog post also went up last month (September) and the current state of the art iirc is a Google model based on Facebook’s Wav2vec 2.0 work. For my own use I’ve already built Pytorch and CoreML frontends, with a shared model format (can convert models to/from wav2letter format and my custom format), and I have the ability to create new models in these frameworks from wav2letter architecture files. I still run my training in the wav2letter framework, but for compatible training in Pytorch I would mostly just need criterion implementations. I assume warpCTC is fine for the CTC models. There’s also a third party Pytorch ASG criterion package but I haven’t tried it yet.
ianhorn 6y ago
Yeah there is the Kaldi backend for caster, which I've tried on my Mac machine, since Dragon isn't a thing on Mac. Unfortunately it's not nearly as good :-( I'd like to try to record my usage of Dragon that I could fine-tune my own model, but it's harder to get around to hobby coding projects like that now that coding is more of a pain in the ass. The unfortunate reality in my case is that using Dragon is the least frustrating way to keep working. I don't think closed paid ASR models will stop being noticeably better better than three open ones until the state-of-the-art error rate is basically zero. I do like that caster is open source where talon isn't, but preferences like that are pretty low priority when say your career is on the line.
- lunixbochs 6y ago
  Fwiw talon has a setting to record everything locally, annotated with the recognized words. In the next release it will also work when using dragon.
- cybwraith 6y ago
  > I do like that caster is open source where talon isn't, but preferences like that are pretty low priority when say your career is on the line. For sure if its on the line right now you use what is available and works best right now, but the reality of niche software is the companies/people backing it tend to close down or move on as they realize the market is too small to sustain them. Or Microsoft updates its OS in a way that forces them to expend considerable time making it functional again and they just don't have the bandwidth for it so you're stuck for some amount of time or indefinitely. If you have no other choice, its definitely better than nothing and you want to use what enables you to do your job the best. But in terms of where I'd be willing to throw my financial support? It'd need to be something for the better of the coding community as whole. For me, this means an open source tool or set of tools, not a proprietary system.

whimsicalism 6y ago

My understanding is that Talon is built off of wav2letter's inference framework for ASR.

lunixbochs 6y ago
I do not use the wav2letter@anywhere inference frontend - I trained the acoustic model using the facebook upstream code, but the decoder is almost entirely new, and on Windows I use Pytorch for inference. Talon ships with a libw2l.so/dylib on Linux/Mac built from my open source repos.
- whimsicalism 6y ago
  Interesting. What model are you using for your acoustic - the streaming convnet? I didn't know that there was a pytorch implementation of the w2l architectures..
- nshm 6y ago
  Feels like wav2letter will not be actively developed anymore. Understandable since it is hard to compete with Pytorch with a custom NN toolkit. Any plans to move to Pytorch/Tensorflow?
  - lunixbochs 6y ago
    I don’t think that’s right at all. They moved development to the Flashlight repo. It seems very actively developed to me. Last commit 3 hours ago. wav2vec 2.0 blog post also went up last month (September) and the current state of the art iirc is a Google model based on Facebook’s Wav2vec 2.0 work. For my own use I’ve already built Pytorch and CoreML frontends, with a shared model format (can convert models to/from wav2letter format and my custom format), and I have the ability to create new models in these frameworks from wav2letter architecture files. I still run my training in the wav2letter framework, but for compatible training in Pytorch I would mostly just need criterion implementations. I assume warpCTC is fine for the CTC models. There’s also a third party Pytorch ASG criterion package but I haven’t tried it yet.

ianhorn 6y ago

Yeah there is the Kaldi backend for caster, which I've tried on my Mac machine, since Dragon isn't a thing on Mac. Unfortunately it's not nearly as good :-( I'd like to try to record my usage of Dragon that I could fine-tune my own model, but it's harder to get around to hobby coding projects like that now that coding is more of a pain in the ass. The unfortunate reality in my case is that using Dragon is the least frustrating way to keep working. I don't think closed paid ASR models will stop being noticeably better better than three open ones until the state-of-the-art error rate is basically zero. I do like that caster is open source where talon isn't, but preferences like that are pretty low priority when say your career is on the line.

lunixbochs 6y ago
Fwiw talon has a setting to record everything locally, annotated with the recognized words. In the next release it will also work when using dragon.
cybwraith 6y ago
> I do like that caster is open source where talon isn't, but preferences like that are pretty low priority when say your career is on the line. For sure if its on the line right now you use what is available and works best right now, but the reality of niche software is the companies/people backing it tend to close down or move on as they realize the market is too small to sustain them. Or Microsoft updates its OS in a way that forces them to expend considerable time making it functional again and they just don't have the bandwidth for it so you're stuck for some amount of time or indefinitely. If you have no other choice, its definitely better than nothing and you want to use what enables you to do your job the best. But in terms of where I'd be willing to throw my financial support? It'd need to be something for the better of the coding community as whole. For me, this means an open source tool or set of tools, not a proprietary system.