2 ms·

Going to be honest but I don't think that is physically possible (i.e. I don't believe that, no offense). Unless you're using a very small beam and model.

lunixbochs 7y ago
Ok, with a few caveats: 1. I don't have the quad core CPU we hit <0.01x on, I only have a dual core laptop CPU. (The quad core numbers were with an i7 7700k or so, they came from the other person I'm working on this with). 2. I'm not running a parallel decode. That's in a branch from my collaborator I haven't merged/built myself yet since I'm only on a dual core. 3. Screen capture has a CPU hit and seems to have slightly increased my RTF during recording. Here's your comment read with 0.05x-0.10x on a dual core CPU: https://youtu.be/jIgUKwR-LaA https://youtu.be/jIgUKwR-LaA Is that enough to convince you that with a stronger CPU and parallel decode we can hit 0.01x?
- Nimitz14 7y ago
  Haha nice one. Okay yeah I believe it (can imagine 0.02, which is close enough). Although I have to mention judging by when the text appears it's above 0.5, but I'm guessing you only output the text after a pause or something.
  - lunixbochs 7y ago
    Sure, that's probably the VAD, which can be tuned (the VAD decides if your phrase is done or not yet). That's part of why I included activity monitor watching w2l, so you could see it wasn't using any CPU while I talked. Would probably be more convincing if I printed something at the moment the VAD cut off the user so you could visually judge from the boundary of what we considered their speech. This video can be partially reproduced with the code and models from here: https://github.com/facebookresearch/wav2letter/issues/327 https://github.com/facebookresearch/wav2letter/issues/327 but most of our decoder optimizations post-date when I compiled that demo. Some of the later code is here: https://github.com/talonvoice/wav2letter/tree/w2lapi_static https://github.com/talonvoice/wav2letter/tree/w2lapi_static https://github.com/ckamm/wav2letter/tree/wip-parallel https://github.com/ckamm/wav2letter/tree/wip-parallel Here is the exact timing loop code I used in that demo: https://bochs.info/p/nhz5cv https://bochs.info/p/nhz5cv
    - Nimitz14 7y ago
      Are your acoustic models 400MB?

lunixbochs 7y ago

Ok, with a few caveats: 1. I don't have the quad core CPU we hit <0.01x on, I only have a dual core laptop CPU. (The quad core numbers were with an i7 7700k or so, they came from the other person I'm working on this with). 2. I'm not running a parallel decode. That's in a branch from my collaborator I haven't merged/built myself yet since I'm only on a dual core. 3. Screen capture has a CPU hit and seems to have slightly increased my RTF during recording. Here's your comment read with 0.05x-0.10x on a dual core CPU: https://youtu.be/jIgUKwR-LaA https://youtu.be/jIgUKwR-LaA Is that enough to convince you that with a stronger CPU and parallel decode we can hit 0.01x?

Nimitz14 7y ago
Haha nice one. Okay yeah I believe it (can imagine 0.02, which is close enough). Although I have to mention judging by when the text appears it's above 0.5, but I'm guessing you only output the text after a pause or something.
- lunixbochs 7y ago
  Sure, that's probably the VAD, which can be tuned (the VAD decides if your phrase is done or not yet). That's part of why I included activity monitor watching w2l, so you could see it wasn't using any CPU while I talked. Would probably be more convincing if I printed something at the moment the VAD cut off the user so you could visually judge from the boundary of what we considered their speech. This video can be partially reproduced with the code and models from here: https://github.com/facebookresearch/wav2letter/issues/327 https://github.com/facebookresearch/wav2letter/issues/327 but most of our decoder optimizations post-date when I compiled that demo. Some of the later code is here: https://github.com/talonvoice/wav2letter/tree/w2lapi_static https://github.com/talonvoice/wav2letter/tree/w2lapi_static https://github.com/ckamm/wav2letter/tree/wip-parallel https://github.com/ckamm/wav2letter/tree/wip-parallel Here is the exact timing loop code I used in that demo: https://bochs.info/p/nhz5cv https://bochs.info/p/nhz5cv
  - Nimitz14 7y ago
    Are your acoustic models 400MB?