5 ms·

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People

by kmfrk 1y ago

Whisper is genuinely amazing - with the right nudging. It's the one AI thing that has genuinely turned my life upside-down in an unambiguously good way. People should check out Subtitle Edit (and throw the dev some money) which is a great interface for experimenting with Whisper transcription. It's basically Aegisub 2.0, if you're old, like me. HOWTO: Drop a video or audio file to the right window, then go to Video > Audio to text (Whisper). I get the best results with Faster-Whisper-XXL. Use large-v2 if you can (v3 has some regressions), and you've got an easy transcription and translation workflow. The results aren't perfect, but Subtitle Edit is for cleaning up imperfect transcripts with features like Tools > Fix common errors. EDIT: Oh, and if you're on the current gen of Nvidia card, you might have to add "--compute_type float32" to make the transcription run correctly. I think the error is about an empty file, output or something like that. EDIT2: And if you get another error, possibly about whisper.exe, iirc I had to reinstall the Torch libs from a specific index like something along these lines (depending on whether you use pip or uv): pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 uv pip install --system torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 If you get the errors and the above fixes work, please type your error message in a reply with what worked to help those who come after. Or at least the web crawlers for those searching for help. https://www.nikse.dk/subtitleedit https://www.nikse.dk/subtitleedit https://www.nikse.dk/donate https://www.nikse.dk/donate https://github.com/SubtitleEdit/subtitleedit/releases https://github.com/SubtitleEdit/subtitleedit/releases

tossit444 1y ago
Aegisub is still actively developed (forked), and imo, both software can't really be compared to one another. They can complement each other, but SE is much better for actual transcription. Aegisub still does the heavy lifting for typesetting and the like.
pawelduda 1y ago
Can you give an example why it made your life that much better?
- shrx 1y ago
  As a hard of hearing person, I can now download any video from the internet (e.g. youtube) and generate subtitles on the fly, not having to struggle to understand badly recorded or unintelligible speech.
  - dylan604 1y ago
    IF the dialog is badly recorded or unintelligible speech, how would a transcription process get it correct?
    - gregoryl 1y ago
      Because it can use the full set of information of the audio - people with hearing difficulties cannot. Also interesting, people with perfectly functional hearing, but whom have "software" bugs (i.e. I find it extremely hard to process voices with significant background nose) can also benefit :)
      - spauldo 1y ago
        I have that issue as well - I can hear faint noises OK but if there's background noise I can't understand what people say. But I'm pretty sure there's a physical issue at the root of it in my case. The problem showed up after several practice sessions with a band whose guitarist insisted on always playing at full volume.
        dylan604 1y ago
        > I have that issue as well You say issue, I say feature. It's a great way to just ignore boring babbling at parties or other social engagements where you're just not that engaged. Sort of like selective hearing in relationships, but used on a wider audience
        enneff 1y ago
        I don’t mean to speak for OP, but it strikes me as rude to make light of someone’s disability in this way. I’d guess it has caused them a lot of frustration.
notatallshaw 1y ago
> uv pip install --system torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 https://download.pytorch.org/whl/cu118 uv has a feature to get the correct version of torch based on your available cuda (and some non-cuda) drivers (though I suggest using a venv not the system Python): > uv pip install torch torchvision torchaudio --torch-backend=auto More details: https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection https://docs.astral.sh/uv/guides/integration/pytorch/#automa... This also means you can safely mix torch requirements with non-torch requirements as it will only pull the torch related things from the torch index and everything else from PyPI.
- xrd 1y ago
  I love uv and really feel like I only need to know "uv add" and "uv sync" to be effective using it with python. That's an incredible feat. But, when I hear about these kinds of extras, it makes me even more excited. Getting cuda and torch to work together is something I have struggled countless times. The team at Astral should be nominated for a Nobel Peace Prize.
  - eigenvalue 1y ago
    They’ve definitely saved me many hours of wasted time between uv and ruff.
  - danudey 1y ago
    > "uv add" One life-changing thing I've been using `uv` for: System python version is 3.12: $ python3 --version Python 3.12.3 A script that requires a library we don't have, and won't work on our local python: $ cat test.py #!/usr/bin/env python3 import sys from rich import print if sys.version_info < (3, 13): print("This script will not work on Python 3.12") else: print(f"Hello world, this is python {sys.version}") It fails: $ python3 test.py Traceback (most recent call last): File "/tmp/tmp/test.py", line 10, in <module> from rich import print ModuleNotFoundError: No module named 'rich' Tell `uv` what our requirements are $ uv add --script=test.py --python '3.13' rich Updated `test.py` `uv` updates the script: $ cat test.py #!/usr/bin/env python3 # /// script # requires-python = ">=3.13" # dependencies = [ # "rich", # ] # /// import sys from rich import print if sys.version_info < (3, 13): print("This script will not work on Python 3.12") else: print(f"Hello world, this is python {sys.version}") `uv` runs the script, after installing packages and fetching Python 3.13 $ uv run test.py Downloading cpython-3.13.5-linux-x86_64-gnu (download) (33.8MiB) Downloading cpython-3.13.5-linux-x86_64-gnu (download) Installed 4 packages in 7ms Hello world, this is python 3.13.5 (main, Jun 12 2025, 12:40:22) [Clang 20.1.4 ] And if we run it with Python 3.12, we can see that errors: $ uv run --python 3.12 test.py warning: The requested interpreter resolved to Python 3.12.3, which is incompatible with the script's Python requirement: `>=3.13` Installed 4 packages in 7ms This script will not work on Python 3.12 Works for any Python you're likely to want: $ uv python list cpython-3.14.0b2-linux-x86_64-gnu <download available> cpython-3.14.0b2+freethreaded-linux-x86_64-gnu <download available> cpython-3.13.5-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/bin/python3.13 cpython-3.13.5+freethreaded-linux-x86_64-gnu <download available> cpython-3.12.11-linux-x86_64-gnu <download available> cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12 cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3 -> python3.12 cpython-3.11.13-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/bin/python3.11 cpython-3.10.18-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/bin/python3.10 cpython-3.9.23-linux-x86_64-gnu <download available> cpython-3.8.20-linux-x86_64-gnu <download available> pypy-3.11.11-linux-x86_64-gnu <download available> pypy-3.10.16-linux-x86_64-gnu <download available> pypy-3.9.19-linux-x86_64-gnu <download available> pypy-3.8.16-linux-x86_64-gnu <download available> graalpy-3.11.0-linux-x86_64-gnu <download available> graalpy-3.10.0-linux-x86_64-gnu <download available> graalpy-3.8.5-linux-x86_64-gnu <download available>
deleted 1y ago
[deleted]
taminka 1y ago
whisper is great, i wonder why youtube's auto generated subs are still so bad? even the smallest whisper is way better than google's solution? is it licensing issue? harder to deploy at scale?
- briansm 1y ago
  I believe youtube still uses 40 mel-scale vectors as feature data, whisper uses 80 (which provides finer spectral detail but is computationally more intensive to process naturally, but modern hardware allows for that)
- ec109685 1y ago
  You’d think they’d use the better model for at least videos that have a large view counts (they already do that when deciding compression optimizations).
jokethrowaway 1y ago
whisper is definitely nice, but it's a bit too slow. Having subtitles and transcription for everything is great - but Nemo Parakeet (pretty much whisper by nvidia) completely changed how I interact with the computer. It enables dictation that actually works and it's as fast as you can think. I also have a set of scripts which just wait for voice commands and do things. I can pipe the results to an LLM, run commands, synthesize a voice with F5-TTS back and it's like having a local Jarvis. The main limitation is being english only.
- threecheese 1y ago
  Would you share the scripts?
  - ec109685 1y ago
    Or at least more details. Very cool!
- forgingahead 1y ago
  Yeah, mind sharing any of the scripts? I looked at the docs briefly, looks like we need to install ALL of nemo to get access to Parakeet? Seems ultra heavy.
  - rhdunn 1y ago
    You only need the ASR bits -- this is where I got to when I previously looked into running Parakeet: # NeMo does not run on 3.13+ python3.12 -m venv .venv source .venv/bin/activate git clone https://github.com/NVIDIA/NeMo.git nemo cd nemo pip install torch torchaudio torchvision --index-url https://download.pytorch.org/whl/cu128 pip install .[asr] deactivate Then run a transcribe.py script in that venv: import os import sys import nemo.collections.asr as nemo_asr model_path = sys.argv[1] audio_path = sys.argv[2] # Load from a local path... asr_model = nemo_asr.models.EncDecRNNTBPEModel.restore_from(restore_path=model_path) # Or download from huggingface ('org/model')... asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained(model_name=model_path) output = asr_moel.transcribe([audio_path]) print(output[0]) With that I was able to run the model, but I ran out of memory on my lower-spec laptop. I haven't yet got around to running it on my workstation. You'll need to modify the python script to process the response and output it in a format you can use.
throwoutway 1y ago
I found this online demo of it: https://www.nikse.dk/subtitleedit/online https://www.nikse.dk/subtitleedit/online
codedokode 1y ago
Kdeenlive also supports auto-generating subtitles which need some editing, but it is faster than create them from scratch. Actually I would be happy even with a simple voice detector so that I don't have to set the timings manually.
hart_russell 1y ago
Is there a way to use it to generate a srt subtitle file given a video file?
- prurigro 1y ago
  It generates a few formats by default including srt
guluarte 1y ago
you can install suing winget or chocolately winget install --id=Nikse.SubtitleEdit -e
Morizero 1y ago
You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?
- kmfrk 1y ago
  Proper diarization still remains a white whale for me, unfortunately. Last I looked into it, the main options required API access to external services, which put me off. I think it was pyannotate.audio[1]. [1]: https://github.com/pyannote/pyannote-audio https://github.com/pyannote/pyannote-audio
  - peterleiser 1y ago
    I used diarization in https://github.com/jhj0517/Whisper-WebUI https://github.com/jhj0517/Whisper-WebUI last night and once it downloads the model from HuggingFace it runs offline (it claims).
- jduckles 1y ago
  WhipserX's diarization is great imo: whisperx input.mp3 --language en --diarize --output_format vtt --model large-v2 Works a treat for Zoom interviews. Diarization is sometimes a bit off, but generally its correct.
  - Morizero 1y ago
    > input.mp3 Thanks but I'm looking for live diarization.
- peterleiser 1y ago
  Check out https://github.com/jhj0517/Whisper-WebUI https://github.com/jhj0517/Whisper-WebUI I ran it last night using docker and it worked extremely well. You need a HuggingFace read-only API token for the Diarization. I found that the web UI ignored the token, but worked fine when I added it to docker compose as an environment variable.
BrunoJo 1y ago
Subtitle Edit is great if you have the hardware to run it. If you don't have GPUs available or don't want to manage the servers I built a simple to use and affordable API that you can use: https://lemonfox.ai/ https://lemonfox.ai/
kanemcgrath 1y ago
Subtitle edit is great, and their subtitle library libse was exactly what I needed for a project I did.

by kmfrk 1y ago

tossit444 1y ago

Aegisub is still actively developed (forked), and imo, both software can't really be compared to one another. They can complement each other, but SE is much better for actual transcription. Aegisub still does the heavy lifting for typesetting and the like.

pawelduda 1y ago

Can you give an example why it made your life that much better?

shrx 1y ago
As a hard of hearing person, I can now download any video from the internet (e.g. youtube) and generate subtitles on the fly, not having to struggle to understand badly recorded or unintelligible speech.
- dylan604 1y ago
  IF the dialog is badly recorded or unintelligible speech, how would a transcription process get it correct?
  - gregoryl 1y ago
    Because it can use the full set of information of the audio - people with hearing difficulties cannot. Also interesting, people with perfectly functional hearing, but whom have "software" bugs (i.e. I find it extremely hard to process voices with significant background nose) can also benefit :)
    - spauldo 1y ago
      I have that issue as well - I can hear faint noises OK but if there's background noise I can't understand what people say. But I'm pretty sure there's a physical issue at the root of it in my case. The problem showed up after several practice sessions with a band whose guitarist insisted on always playing at full volume.
      - dylan604 1y ago
        > I have that issue as well You say issue, I say feature. It's a great way to just ignore boring babbling at parties or other social engagements where you're just not that engaged. Sort of like selective hearing in relationships, but used on a wider audience
        enneff 1y ago
        I don’t mean to speak for OP, but it strikes me as rude to make light of someone’s disability in this way. I’d guess it has caused them a lot of frustration.

notatallshaw 1y ago

> uv pip install --system torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 https://download.pytorch.org/whl/cu118 uv has a feature to get the correct version of torch based on your available cuda (and some non-cuda) drivers (though I suggest using a venv not the system Python): > uv pip install torch torchvision torchaudio --torch-backend=auto More details: https://docs.astral.sh/uv/guides/integration/pytorch/#automatic-backend-selection https://docs.astral.sh/uv/guides/integration/pytorch/#automa... This also means you can safely mix torch requirements with non-torch requirements as it will only pull the torch related things from the torch index and everything else from PyPI.

xrd 1y ago
I love uv and really feel like I only need to know "uv add" and "uv sync" to be effective using it with python. That's an incredible feat. But, when I hear about these kinds of extras, it makes me even more excited. Getting cuda and torch to work together is something I have struggled countless times. The team at Astral should be nominated for a Nobel Peace Prize.
- eigenvalue 1y ago
  They’ve definitely saved me many hours of wasted time between uv and ruff.
- danudey 1y ago
  > "uv add" One life-changing thing I've been using `uv` for: System python version is 3.12: $ python3 --version Python 3.12.3 A script that requires a library we don't have, and won't work on our local python: $ cat test.py #!/usr/bin/env python3 import sys from rich import print if sys.version_info < (3, 13): print("This script will not work on Python 3.12") else: print(f"Hello world, this is python {sys.version}") It fails: $ python3 test.py Traceback (most recent call last): File "/tmp/tmp/test.py", line 10, in <module> from rich import print ModuleNotFoundError: No module named 'rich' Tell `uv` what our requirements are $ uv add --script=test.py --python '3.13' rich Updated `test.py` `uv` updates the script: $ cat test.py #!/usr/bin/env python3 # /// script # requires-python = ">=3.13" # dependencies = [ # "rich", # ] # /// import sys from rich import print if sys.version_info < (3, 13): print("This script will not work on Python 3.12") else: print(f"Hello world, this is python {sys.version}") `uv` runs the script, after installing packages and fetching Python 3.13 $ uv run test.py Downloading cpython-3.13.5-linux-x86_64-gnu (download) (33.8MiB) Downloading cpython-3.13.5-linux-x86_64-gnu (download) Installed 4 packages in 7ms Hello world, this is python 3.13.5 (main, Jun 12 2025, 12:40:22) [Clang 20.1.4 ] And if we run it with Python 3.12, we can see that errors: $ uv run --python 3.12 test.py warning: The requested interpreter resolved to Python 3.12.3, which is incompatible with the script's Python requirement: `>=3.13` Installed 4 packages in 7ms This script will not work on Python 3.12 Works for any Python you're likely to want: $ uv python list cpython-3.14.0b2-linux-x86_64-gnu <download available> cpython-3.14.0b2+freethreaded-linux-x86_64-gnu <download available> cpython-3.13.5-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.13.5-linux-x86_64-gnu/bin/python3.13 cpython-3.13.5+freethreaded-linux-x86_64-gnu <download available> cpython-3.12.11-linux-x86_64-gnu <download available> cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3.12 cpython-3.12.3-linux-x86_64-gnu /usr/bin/python3 -> python3.12 cpython-3.11.13-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/bin/python3.11 cpython-3.10.18-linux-x86_64-gnu /home/dan/.local/share/uv/python/cpython-3.10.18-linux-x86_64-gnu/bin/python3.10 cpython-3.9.23-linux-x86_64-gnu <download available> cpython-3.8.20-linux-x86_64-gnu <download available> pypy-3.11.11-linux-x86_64-gnu <download available> pypy-3.10.16-linux-x86_64-gnu <download available> pypy-3.9.19-linux-x86_64-gnu <download available> pypy-3.8.16-linux-x86_64-gnu <download available> graalpy-3.11.0-linux-x86_64-gnu <download available> graalpy-3.10.0-linux-x86_64-gnu <download available> graalpy-3.8.5-linux-x86_64-gnu <download available>

deleted 1y ago

[deleted]

taminka 1y ago

whisper is great, i wonder why youtube's auto generated subs are still so bad? even the smallest whisper is way better than google's solution? is it licensing issue? harder to deploy at scale?

briansm 1y ago
I believe youtube still uses 40 mel-scale vectors as feature data, whisper uses 80 (which provides finer spectral detail but is computationally more intensive to process naturally, but modern hardware allows for that)
ec109685 1y ago
You’d think they’d use the better model for at least videos that have a large view counts (they already do that when deciding compression optimizations).

jokethrowaway 1y ago

whisper is definitely nice, but it's a bit too slow. Having subtitles and transcription for everything is great - but Nemo Parakeet (pretty much whisper by nvidia) completely changed how I interact with the computer. It enables dictation that actually works and it's as fast as you can think. I also have a set of scripts which just wait for voice commands and do things. I can pipe the results to an LLM, run commands, synthesize a voice with F5-TTS back and it's like having a local Jarvis. The main limitation is being english only.

threecheese 1y ago
Would you share the scripts?
- ec109685 1y ago
  Or at least more details. Very cool!
forgingahead 1y ago
Yeah, mind sharing any of the scripts? I looked at the docs briefly, looks like we need to install ALL of nemo to get access to Parakeet? Seems ultra heavy.
- rhdunn 1y ago
  You only need the ASR bits -- this is where I got to when I previously looked into running Parakeet: # NeMo does not run on 3.13+ python3.12 -m venv .venv source .venv/bin/activate git clone https://github.com/NVIDIA/NeMo.git nemo cd nemo pip install torch torchaudio torchvision --index-url https://download.pytorch.org/whl/cu128 pip install .[asr] deactivate Then run a transcribe.py script in that venv: import os import sys import nemo.collections.asr as nemo_asr model_path = sys.argv[1] audio_path = sys.argv[2] # Load from a local path... asr_model = nemo_asr.models.EncDecRNNTBPEModel.restore_from(restore_path=model_path) # Or download from huggingface ('org/model')... asr_model = nemo_asr.models.EncDecRNNTBPEModel.from_pretrained(model_name=model_path) output = asr_moel.transcribe([audio_path]) print(output[0]) With that I was able to run the model, but I ran out of memory on my lower-spec laptop. I haven't yet got around to running it on my workstation. You'll need to modify the python script to process the response and output it in a format you can use.

throwoutway 1y ago

I found this online demo of it: https://www.nikse.dk/subtitleedit/online https://www.nikse.dk/subtitleedit/online

codedokode 1y ago

Kdeenlive also supports auto-generating subtitles which need some editing, but it is faster than create them from scratch. Actually I would be happy even with a simple voice detector so that I don't have to set the timings manually.

hart_russell 1y ago

Is there a way to use it to generate a srt subtitle file given a video file?

prurigro 1y ago
It generates a few formats by default including srt

guluarte 1y ago

you can install suing winget or chocolately winget install --id=Nikse.SubtitleEdit -e

Morizero 1y ago

You don't happen to know a whisper solution that combines diarization with live audio transcription, do you?

kmfrk 1y ago
Proper diarization still remains a white whale for me, unfortunately. Last I looked into it, the main options required API access to external services, which put me off. I think it was pyannotate.audio[1]. [1]: https://github.com/pyannote/pyannote-audio https://github.com/pyannote/pyannote-audio
- peterleiser 1y ago
  I used diarization in https://github.com/jhj0517/Whisper-WebUI https://github.com/jhj0517/Whisper-WebUI last night and once it downloads the model from HuggingFace it runs offline (it claims).
jduckles 1y ago
WhipserX's diarization is great imo: whisperx input.mp3 --language en --diarize --output_format vtt --model large-v2 Works a treat for Zoom interviews. Diarization is sometimes a bit off, but generally its correct.
- Morizero 1y ago
  > input.mp3 Thanks but I'm looking for live diarization.
peterleiser 1y ago
Check out https://github.com/jhj0517/Whisper-WebUI https://github.com/jhj0517/Whisper-WebUI I ran it last night using docker and it worked extremely well. You need a HuggingFace read-only API token for the Diarization. I found that the web UI ignored the token, but worked fine when I added it to docker compose as an environment variable.

BrunoJo 1y ago

Subtitle Edit is great if you have the hardware to run it. If you don't have GPUs available or don't want to manage the servers I built a simple to use and affordable API that you can use: https://lemonfox.ai/ https://lemonfox.ai/

kanemcgrath 1y ago

Subtitle edit is great, and their subtitle library libse was exactly what I needed for a project I did.