3 ms·

If you're going as far as using yt-dlp, why not run the audio through Whisper?

andai 2y ago
Interesting, I haven't used Whisper, is it cost effective? Seems to be about 36 cents per (hour long) video? How long does processing take?
- kajecounterhack 2y ago
  You can run it locally, and it's really fast. But since YouTube transcription is really good, I don't see why you'd use Whisper and get a worse transcription (unless maybe it's on videos that Google did not transcribe for whatever reason).
  - gs17 2y ago
    > But since YouTube transcription is really good Are you sure you're looking at automatic transcripts? YouTube transcripts are bizarrely low quality if they're not provided by the creators (I've actually used my Google Pixel's live transcription to make better captions occasionally). I just checked a video my girlfriend uploaded a week ago and the auto-transcript was still pretty messy. I've used Whisper for the same task and it's significantly better.
    - ofou 2y ago
      Agreed. However, you can get great YT transcriptions using GPT-4o mini to clean them up.
    - jokethrowaway 2y ago
      That's crazy, months ago I compared whisper v2 transcripts with YouTube transcripts generated on my video and found them to be identical, down to the timestamps. I know people who upload a video on YouTube unlisted just to get transcripts generation for free and then delete the video.
- HPsquared 2y ago
  36 cents an hour is how much it costs to hire an entire GPU like an A4000. I can assure you Whisper runs much, much faster than 1x!
  - jokethrowaway 2y ago
    A few derivative projects are faster than 1x, insanely-fast-whisper being the fastest I've tried. whisper v3 large on release day was around 1x on a 4090
davidzweig 2y ago
The security against downloading audio from YouTube has been upped recently with 'PO tokens'. Whisper is only a few tenths of a cent per hour transcribed if transcribing on your gpu though, at about 30x real-time on a 3080 etc. with batching.
- swyx 2y ago
  > The security against downloading audio from YouTube has been upped recently with 'PO tokens'. do you have a source? more generally is there a community or news source for youtube "api" news like this?
  - davidzweig 2y ago
    I haven't been following closely the last few weeks, but you can check the issues in this repo, for example: https://github.com/distubejs/ytdl-core https://github.com/distubejs/ytdl-core
- HPsquared 2y ago
  Tbh I've not had trouble with this for personal use.

Interesting, I haven't used Whisper, is it cost effective? Seems to be about 36 cents per (hour long) video? How long does processing take?

kajecounterhack 2y ago
You can run it locally, and it's really fast. But since YouTube transcription is really good, I don't see why you'd use Whisper and get a worse transcription (unless maybe it's on videos that Google did not transcribe for whatever reason).
- gs17 2y ago
  > But since YouTube transcription is really good Are you sure you're looking at automatic transcripts? YouTube transcripts are bizarrely low quality if they're not provided by the creators (I've actually used my Google Pixel's live transcription to make better captions occasionally). I just checked a video my girlfriend uploaded a week ago and the auto-transcript was still pretty messy. I've used Whisper for the same task and it's significantly better.
  - ofou 2y ago
    Agreed. However, you can get great YT transcriptions using GPT-4o mini to clean them up.
  - jokethrowaway 2y ago
    That's crazy, months ago I compared whisper v2 transcripts with YouTube transcripts generated on my video and found them to be identical, down to the timestamps. I know people who upload a video on YouTube unlisted just to get transcripts generation for free and then delete the video.
HPsquared 2y ago
36 cents an hour is how much it costs to hire an entire GPU like an A4000. I can assure you Whisper runs much, much faster than 1x!
- jokethrowaway 2y ago
  A few derivative projects are faster than 1x, insanely-fast-whisper being the fastest I've tried. whisper v3 large on release day was around 1x on a 4090

davidzweig 2y ago

The security against downloading audio from YouTube has been upped recently with 'PO tokens'. Whisper is only a few tenths of a cent per hour transcribed if transcribing on your gpu though, at about 30x real-time on a 3080 etc. with batching.

swyx 2y ago
> The security against downloading audio from YouTube has been upped recently with 'PO tokens'. do you have a source? more generally is there a community or news source for youtube "api" news like this?
- davidzweig 2y ago
  I haven't been following closely the last few weeks, but you can check the issues in this repo, for example: https://github.com/distubejs/ytdl-core https://github.com/distubejs/ytdl-core
HPsquared 2y ago
Tbh I've not had trouble with this for personal use.