3 ms·
Show HN: CPU-only transcription for YouTube, TikTok, X, Instagram videos
- spudlyo 5mo agoSo, this project consists of a ~175 line README and a ~500 line Python program that glues yt-dlp and Kroko together. Neat. I guess if it encourages you to install and figure out how to use ffmpeg, yt-dlp, kroko, numpy, and onnx that's a good thing. Sometimes just knowing a thing is possible is a huge benefit.
- mrkn1 5mo agothank you. You nailed the actual value, that's right. The real win is just knowing you can do this on a laptop CPU, offline, no GPU or cloud bill. There are tiny done-for-you details, like rescaling token timestamps back to real time after the atempo speedup so --timestamps doesn't lie to you, but they are minor.
- mscdex 5mo agoWhy the choice of Kroko over something like parakeet-tdt-0.6b-v3, which is also faster than realtime on CPU?
- deleted 5mo ago[deleted]
- nshm 5mo agoKroko models are more accurate and their size is just a hundred megabytes compared to parakeet (2.5 gigabytes in default fp32)
- mscdex 5mo agoDo you have a link to results confirming this? Kroko does not seem to be on the Open ASR Leaderboard. Parakeet has an average WER of 6.32 across several common datasets.
- mrkn1 5mo agoKroko's website says benchmarks aren't formalized yet. FWIW, this url says 5% WER for English [0]. though it doesn't specify the dataset, so not directly comparable to Parakeet's 6.32 on the Open ASR Leaderboard Best way to judge is to try it on your own audio [0] https://huggingface.co/hudaiapa88/sherpa-stt-onnx https://huggingface.co/hudaiapa88/sherpa-stt-onnx
- iririririr 5mo agoI see the value as a centralized anti-content-blocker. This repo is now a good way to centralize hacks around the sure-to-come blockers those platforms will add to prevent download. Just like uBlockOrigin was a way to centralize all the "just run this greasemonkey script" comments, I can see this getting a huge following for people who really value transcriptions.
- mrkn1 5mo agoI appreciate the perspective! higher ceiling than I'd put on it, but if it gets there awesome. PRs welcome!
- xnx 5mo ago[dead]
- charcircuit 5mo agoMost of these platforms already have transcriptions built in.
- mrkn1 5mo agoYoutube has transcripts on most videos, not all. The others don't expose them. If you mean the "transcript APIs" for TikTok/IG/X, they are all transcribing audio like yapsnap does. If you have a way to pull native ones, let me know, genuinely curious.
- charcircuit 5mo agoYouTube's is transcribing the audio too. The other do expose them as subtitles as the video is playing.
- mrkn1 5mo agoYes fair point, asr cached and exposed. I meant to draw the line more on fetchable or not.
- niraj-agarwal 5mo agoHad Claude test it out on 3 videos. Worked at 5-8x realtime. The beauty of it is that it works on all videos, not just the one with transcripts. Combine it with YouTube search and LLM takeaways from transcripts, and you have super-efficient content consumption. There are SaaS products that charge 1 cent per video for those with transcripts. There is a viable product in here somewhere, methinks.
- mrkn1 5mo agothanks for running it Niraj. I see something similar on my machine, which still surprises me every time lol
- ranger_danger 5mo agoHow can we transcribe other languages besides English?
- mrkn1 5mo agoJust download the model for your preferred language, all hosted on the Kroko-ASR collection here: https://huggingface.co/Banafo/Kroko-ASR/tree/main https://huggingface.co/Banafo/Kroko-ASR/tree/main Right now you have Dutch, French, Portuguese, Spanish, German, Italian, Swedish, Swiss German, Hebrew, and Turkish. Grab the one that matches your audio, point yapsnap at it with --model (or set KROKO_MODEL), and you're set!
- ranger_danger 5mo agoWas hoping for CJK languages but I don't see any there. Thanks anyway
- canadiantim 5mo agoNice. Can it do speaker diarization?
- HDBaseT 5mo agoWouldn't it still be more efficient to do GPU transcriptions anyways? is this something we could actually put the effectively useless NPUs to use in modern laptops?
- KingMob 5mo agoPossibly, but you may want to use the GPUs for other things, or have under-utilized CPU-only servers lying around.
- dharma1 5mo agoyes GPU is significantly faster, but cpu only lets you do it anywhere - wasm in the browser, any server etc. NPUs - definitely a good use case for at least part of it, there are ports of whisper that use coreML/ANE with less power and 3x speed of CPU only
- dmos62 5mo agoNow make it distinguish speakers and we really have something. As far as I know, that's significantly harder though.
- piotrrojek 5mo agoIf someone is interested, this is my supershort zsh/bash scripts that I keep in .zshrc for doing the same thing using plain whisper.cpp, ffmpeg and yt-dlp (`brew install whisper-cpp yt-dlp` for Mac); I output it in vtt format (subtitles) though, but it's easy enough to change it to txt. yt_to_srt() { local url="$1" local output_base="$2" local language="${3:-en}" yt-dlp -x --audio-format wav --postprocessor-args "-ar 16000" -o "$output_base.wav" "$url" whisper-cli --language "$language" --model "$WHISPER_MODEL" --split-on-word --max-len 65 --output-vtt --output-file "$output_base" --file "$output_base.wav" rm "$output_base.wav" } file_to_srt() { local filepath="$1" local language="${2:-en}" local filename=$(basename "$filepath") local filename_no_ext="${filename%.*}" local output_base="$filename_no_ext" local temp_wav="$output_base.wav" ffmpeg -i "$filepath" -vn -acodec pcm_s16le -ar 16000 -ac 1 "$temp_wav" whisper-cli --language "$language" --model "$WHISPER_MODEL" --split-on-word --max-len 65 --output-vtt --output-file "$output_base" --file "$temp_wav" rm "$temp_wav" } plus additional bootstrap script for large-v3-turbo model from my chez-moi dotfiles: #!/bin/bash # Download whisper.cpp models from Hugging Face (runs once per machine). set -euo pipefail MODELS_DIR="$HOME/whisper-models" BASE_URL="https://huggingface.co/ggerganov/whisper.cpp/resolve/main" MODELS=("ggml-large-v3-turbo.bin" "ggml-tiny.bin") mkdir -p "$MODELS_DIR" for model in "${MODELS[@]}"; do if [ ! -f "$MODELS_DIR/$model" ]; then echo "Downloading $model..." curl -L --progress-bar -o "$MODELS_DIR/$model" "$BASE_URL/$model" else echo "$model already exists, skipping." fi done echo "Whisper models ready at $MODELS_DIR"
- ramon156 5mo agoyt-dlp can download auto-subtitles and regular subtitles, why not do that and fall back to whisper?
- piotrrojek 5mo agoTo be frank I didn't know there's such an option :-)
- ranger_danger 5mo agoIn my experience Whisper is several orders of magnitude slower though.
- chris_explicare 5mo ago[flagged]
- throw98226 5mo agoWorks extremely well. Command to install on Debian 13: sudo apt update && sudo apt install -y ffmpeg python3-pip python3-venv && git clone https://github.com/kouhxp/yapsnap.git https://github.com/kouhxp/yapsnap.git && cd yapsnap && python3 -m venv ~/yapsnap-venv && source ~/yapsnap-venv/bin/activate && pip install --upgrade pip && pip install . On a 32GB ThinkPad X13, a 21 minute YouTube video was processed by yapsnap under 2 minutes. Very well done!
- mrkn1 5mo agothank you!
- delis-thumbs-7e 5mo agoAm I a bit thick, but first we created this amazing way to transfer any text very cheaply and fast over network, then we (well, I think it was Meta and Google) decided that no, everything must be a video, then we added subtitles and AI-transcriptions to those videos and now we just dowload transcriptions of those videos presumably to feel LLM to make summaries of them in order to… Read. Them. I think I’m gonna go read a book.
- mrkn1 5mo agoGood point! I haven't found a faster way to consume info than reading. But depends on the type of learner you are (visual, auditory, hands-on/interactive, etc)
- 7777777phil 5mo agoTis is very simple and very cool! Just installed it on my Hetzner box where I run a remote controlled local agent so now I can basically chat/email a video link to get a summary and/or ask questions. The only issue was YouTube's PO Token requirement (web/mweb clients refuse to serve formats from datacenter IPs without a valid Proof-of-Origin token.) So I had to find a client that still work without PO Token first. Thanks for sharing!
- mrkn1 5mo agothank you! good use case, what hetzner box specs have you chosen?
- 7777777phil 5mo agoCPX22
- jorritpr 5mo agoVery cool, I'm also working on a captioning/subtitling project for the lecture recordings for the university I work at. My biggest challenge is finding a proper language model that is fast enough and accurate enough since I have to caption about 600 hours of video per week and I preferably want to run all of this on a tiny server (2 cores 4 GB memory). This tool could easily do that with the kroko model but I'll have to test if the accuracy is good enough. Also in my own scripts I'm using ffmpeg to download just the audio of the videos that I want to caption, which saves a lot of bandwith and speeds up the whole process. As far as I can see this tool doesn't do that, that would be a nice functionality to add, plus an option to turn the output into a working .srt file.
- mrkn1 5mo agothanks! making a note of the feature request
- ranger_danger 5mo agoHow is this so much faster than even GPU-based whisper?
- mrkn1 5mo agosmall, ONNX-optimized models designed specifically for low-latency CPU streaming, so it avoids overhead of large transformer arch and GPU memory transfers
- majorchord 5mo agoI thought ONNX models were only for text-to-speech? How does one tell them apart if I find some files online?
- mrkn1 5mo ago[dead]
- photonair 5mo ago[flagged]
- mrkn1 5mo agoAdded Diarization / Speaker Separation that is fast and CPU only. Thank you all for the great feedback and support. PRs welcome! yapsnap "https://www.youtube.com/watch?v=NzKJ-xO-VhE" --diarize SPEAKER_00 [00:00]: Welcome to the show. SPEAKER_01 [00:03]: Glad to be here, thanks for having me. SPEAKER_00 [00:08]: Let's get started.
- alextmx 5mo ago[flagged]
- dubaiamir1111 5mo agodubaiamir1111
- freakynit 5mo agoSo just tested this on a bunch of videos. It's crazy accurate (not 100% obviously), fast and resource efficient. Not only was I testing the stt part, but also, timestamps and speaker identifications. Tested on 3 different videos, local and online both. Timestamps were precise down to sub 500-ms level even on longer 20+ minute videos. Speaker identifications worked equally well. My old M1 Air didn't hang a single bit while the transcription was going on. --- 1. Here's one from a single speaker video (https://www.youtube.com/watch?v=-X6YzlY_8tM https://www.youtube.com/watch?v=-X6YzlY_8tM): https://pastebin.com/vPVPNnne https://pastebin.com/vPVPNnne 2. A shorter with up to 4 different speakers and mixed, complex scene/narration changes (https://www.youtube.com/watch?v=4tASl0auPOg https://www.youtube.com/watch?v=4tASl0auPOg): https://pastebin.com/iHZZD8Qe https://pastebin.com/iHZZD8Qe -- My zsh shorthand: `alias transcribe="yapsnap --timestamps --diarize "`