7 ms·
With transcribing a talk by Andrej, you already picked the most challenging case possible, speed-wise. His natural talking speed is already >=1.5x that of a nor
by w-m 1y ago
With transcribing a talk by Andrej, you already picked the most challenging case possible, speed-wise. His natural talking speed is already >=1.5x that of a normal human. One of the people you absolutely have to set your YouTube speed back down to 1x when listening to follow what's going on.
In the idea of making more of an OpenAI minute, don't send it any silence.
E.g.
ffmpeg -i video-audio.m4a \
-af "silenceremove=start_periods=1:start_duration=0:start_threshold=-50dB:\
stop_periods=-1:stop_duration=0.02:stop_threshold=-50dB,\
apad=pad_dur=0.02" \
-c:a aac -b:a 128k output_minpause.m4a -y
will cut the talk down from 39m31s to 31m34s, by replacing any silence (with a -50dB threshold) longer than 20ms by a 20ms pause. And to keep with the spirit of your post, I measured only that the input file got shorter, I didn't look at all at the quality of the transcription by feeding it the shorter version.
- georgemandis 1y agoOooh fun! I had a feeling there was more ffmpeg wizardry I could be leaning into here. I'll have to try this later—thanks for the idea!
- w-m 1y agoIn the meantime I realized that the apad part is nonsensical - it pads the end of the stream, not at each silence-removed cut. I wanted to get angry at o3 for proposing this, but then I had a look at the silenceremove= documentation myself: https://ffmpeg.org/ffmpeg-filters.html#silenceremove https://ffmpeg.org/ffmpeg-filters.html#silenceremove Good god. You couldn't make that any more convoluted and hard-to-grasp if you wanted to. You gotta love ffmpeg! I now think this might be a good solution: ffmpeg -i video-audio.m4a \ -af "silenceremove=start_periods=1:stop_periods=-1:stop_duration=0.15:stop_threshold=-40dB:detection=rms" \ -c:a aac -b:a 128k output.m4a -y
- snickerdoodle12 1y agoI love ffmpeg but the documentation is often close to incomprehensible.
- squigz 1y agoOut of curiosity, how might you improve those docs? They seem fairly reasonable to me
- w-m 1y agoThe documentation reads like it was written by a programmer who documented the different parameters to their implementation of a specific algorithm. Now when you as the user come along and want to use silenceremove, you'll have to carefully read through this, and build your own mental model of that algorithm, and then you'll be able to set these parameters accordingly. That takes a lot of time and energy, in this case multiple read-throughs and I'd say > 5 minutes. Good documentation should do this work for you. It should explain somewhat atomic concepts to you, that you can immediately adapt, and compose. Where it already works is for the "detection" and "window" parameters, which are straightforward. But the actions of trimming in the start/middle/end, and how to configure how long the silence lasts before trimming, whether to ignore short bursts of noise, whether to skip every nth silence period, these are all ideas and concepts that get mushed together in 10 parameters which are called start/stop-duration/threshold/silence/mode/periods. If you want to apply this filter, it takes a long time to build mental models for these 10 parameters. You do have some example calls, which is great, but which doesn't help if you need to adjust any of these - then you probably need to understand them all. Some stuff I stumbled over when reading it: "To remove silence from the middle of a file, specify a stop_periods that is negative. This value is then treated as a positive value [...]" - what? Why is this parameter so heavily overloaded? "start_duration: Specify the amount of time that non-silence must be detected before it stops trimming audio" - parameter is named start_something, but it's about stopping? Why? "start_periods: [...] Normally, [...] start_periods will be 1 [...]. Default value is 0." "start_mode: Specify mode of detection of silence end at start": start_mode end at start? It's very clunky. Every parameter has multiple modes of operation. Why is it start and stop for beginning and end, and why is "do stuff in the middle" part of the end? Why is there no global mode? You could nitpick this stuff to death. In the end, naming things is famously one of the two hard problems in computer science (the others being cache invalidation and off-by-one errors). And writing good documentation is also very, very hard work. Just exposing the internals of the algorithm is often not great UX, because then every user has to learn how the thing works internally before they can start using it (hey, looking at you, git). So while it's easy to point out where these docs fail, it would be a lot of work to rewrite this documentation from the top down, explaining the concepts first. Or even rewriting the interface to make this more approachable, and the parameters less overloaded. But since it's hard work, and not sexy to programmers, it won't get done, and many people will come after, having to spend time on reading and re-reading this current mess.
- dylan604 1y agoif you did it in 2 passes, you could find the cut points using silence detect, use a bunch of -ss/-t/-i based on those segments, apad each segment with a -filter_complex chain the ends in concating. it would be a wonderfully gnarly command for very little benefit. but it could be done
- pragmatic 1y agoNo not really? The talk where he babbles about OSes and everyone is somehow impressed?
- behnamoh 1y ago> His natural talking speed is already >=1.5x that of a normal human. One of the people you absolutely have to set your YouTube speed back down to 1x when listening to follow what's going on. I wonder if there's a way to automatically detect how "fast" a person talks in an audio file. I know it's subjective and different people talk at different paces in an audio, but it'd be cool to kinda know when OP's trick fails (they mention x4 ruined the output; maybe for karpathy that would happen at x2).
- echelon 1y ago> I wonder if there's a way to automatically detect how "fast" a person talks in an audio file. Stupid heuristic: take a segment of video, transcribe text, count number of words per utterance duration. If you need speaker diarization, handle speaker utterance durations independently. You can further slice, such as syllable count, etc.
- nand4011 1y agohttps://www.science.org/doi/10.1126/sciadv.aaw2594 https://www.science.org/doi/10.1126/sciadv.aaw2594 Apparently human language conveys information at around 39 bits/s. You could use a similar technique as that paper to determine the information rate of a speaker and then correct it to 39 bits/s by changing the speed of the video.
- varispeed 1y agoIt's a shame platforms don't generally support speeds greater than 2x. One of my "superpowers" or a curse is that I cannot stand normal speaking pace. When I watch lectures, I always go for maximum speed and that still is too slow for me. I wish platforms have included 4x but done properly (with minimal artefacts).
- lofaszvanitt 1y agoRobot in a human body identified :D.
- mrmuagi 1y ago
- deleted 1y ago[deleted]
- brunoborges 1y agoThe interesting thing here is that OpenAI likely has a layer that trims down videos exactly how you suggest, so they can still charge by the full length while costing less for them to actually process the content.
- cbsmith 1y agoThat's an amusing perspective. I really struggle with watching any video at double speed, but I've never had trouble listening to any of his talks at 1x. To me, he seems to speak at a perfectly reasonable pace.
- swyx 1y ago> I didn't look at all at the quality of the transcription by feeding it the shorter version. guys how hard is it to toss both versions into like diffchecker or something haha youre just comparing text
- TimorousBestie 1y agoWhy use diffchecker when there’s a perfectly good LLM you could ask right there? lol
- serf 1y agobecause a lot of LLMs will just eat tokens to call a diffchecker. really it becomes a question of whether or not the friction of invoking the command or the cost of tokens is greater. as I get older and more rsi'd the tokens seem cheaper.
- trashchomper 1y agoAssuming sarcasm but if not, because deterministic vs. nondeterministic output?
- TimorousBestie 1y agoNot sarcasm, just a little joke. I thought the emote at the end would prevent it from being taken seriously. . .
- Der_Einzige 1y agoMake it semi deterministic with structured/constrained generation!
- QuantumGood 1y agoI wish there was a 2.25x YouTube option for "normal" humans. I already use every shortcut, and listen at 2x 90% of the time. But Andrej I can't take faster than 1.25x
- zamadatix 1y agoYouTube ran an experiment with up to 4x playback on mobile (???) but it went away in February. I get a lot of the experiments they do being experiments but why just allowing the slider to go farther is such a back and forth hoopla is beyond me. It's one of the oft touted features of 3rd party apps and extensions with nearly 0 UI impact to those who don't want to use it (just don't slide the slider past 2x if you don't want past 2x). https://www.theverge.com/news/603581/youtube-premium-experimental-test-4x-speed-now-available https://www.theverge.com/news/603581/youtube-premium-experim...
- K2L8M11N2 1y agoAs a premium subscriber I currently have 4x available on Android and they recently (in the last month) added it to web too
- zamadatix 1y agoAs a premium subscriber I no longer have 4x available on iOS and I never had it on web. I also no longer see it as an experiment on https://www.youtube.com/new https://www.youtube.com/new I hope they make up their mind on it soon instead of this endless A/B testing.
- zelphirkalt 1y agoProbably, because they are "A/B testing" things, that do not really show much effect or depend on more circumstances, than they care to eliminate and then overinterpret the results. Like almost all corporate A/B testing.
- ars 1y agoInstall this: https://mybrowseraddon.com/video-speed-control.html https://mybrowseraddon.com/video-speed-control.html I listen to a lot of videos on 3 or even 4x.
- nickjj 1y agoAndrej's talk seemed normal to listen at 2x but I've also listened to everything at 2x for a long time. Unfortunately a byproduct of listening to everything at 2x is I've had a number of folks say they have to watch my videos at 0.75x but even when I play back my own videos it feels painfully slow unless it's 2x. For reference I've always found John Carmack's pacing perfect / natural and watchable at 2x too. A recent video of mine is https://www.youtube.com/watch?v=pL-qft1ykek https://www.youtube.com/watch?v=pL-qft1ykek. It was posted on HN by someone else the other day so I'm not trying to do any self promotion here, it's just an example of a recent video I put up and am generally curious if anyone finds that too fast or it's normal. It's a regular unscripted video where I have a rough idea of what I want to cover and then turn on the mic, start recording and let it pan out organically. If I had to guess I'd say the last ~250-300 videos were recorded this way.
- noahjk 1y agoTo me you talk at what I would consider "1.2x" of podcast speed (which to me is a decent average measure of spoken word speed - I usually do 1.5x on all podcasts). You're definitely still in the normal distribution for tech YouTubers, in my experience - in fact it feels like a lot of tech YouTube talks like they've had a bit too much adderall, but you don't come off that way. Naturally people may choose to slow down tutorials, because the person giving the tutorial can never truly understand what someone learning would or wouldn't understand. So overall I think your speed is totally fine! Also, very timely video, I was interested in the exact topic, so I'm happy I found this.
- eru 1y ago> "[I]n fact it feels like a lot of tech YouTube talks like they've had a bit too much adderall, [...]" Funnily enough, if you actually have ADHD, then stimulants like adderall or even nicotine, will calm you down. > Naturally people may choose to slow down tutorials, [...] For me it also depends on what mood I'm in and whether I'm doing anything else at the same time. If I'm fully concentrating on a video, 2x is often fine. If I'm doing some physical task at the same time, I need it slower than that. If I'm doing a mental task at the same, I can forget about getting anything out of the video. At least, if the mental task involves any words. So eg I could probably still follow along a technical discussion at roughly 1x speed while playing Tetris, but not while coding.
- jwrallie 1y agoFrom my own experience with whisper.cpp, normalizing the audio and removing silence not only shortens the process time significantly, but also increases a lot the quality of the transcription, as silence can mean hallucinations. You can do that graphically with Audacity too, if you do not want to deal with the command line. You also do not need any special hardware to run whisper.cpp, with the small model literally any computer should be able to do it if you can wait a bit (less than the audio length). One half interesting / half depressing observation I made is that at my workplace any meeting recording I tried to transcribe in this way had its length reduced to almost 2/3 when cutting off the silence. Makes you think about the efficiency (or lack of it) of holding long(ish) meetings.
- d1sxeyes 1y ago1/3 of the meeting is silence? That’s a good thing. It’s allowing people time to think over what they’re hearing, there are pauses to allow people to contribute or participate. What do you think a better percentage of silent time would be?
- jwrallie 1y agoGood point, somehow if I think of a 30 minutes meeting, 10 minutes of silence sounds great, but seeing a 1 hour block disappear from a 3 hour recording makes me want to use that “free” hour to do something else. Well, I don’t think silence is not the real problem with a 3 hour meeting!
- literalAardvark 1y agoIf people could speak continuously for an entire meeting then that meeting would be better off as an email. Meetings are for bouncing half formed ideas around and coagulating that into something greater. There MUST be time to think
- sudhirj 1y agoIf a human meeting had lot of silence (assuming it's between words and not before / after), I would consider it a very efficient meeting where there was just enough information exchanged with adequate absorption, processing and response time.
- vayup 1y agoGemini charges by tokens rather than minutes. I used VAD to trim silence hoping token count will go down. I noticed the token count wasn't much different (Eg: 30 seconds of background noise had the same count as 2s of background noise). Either Gemini API trims silence under the hood, or the nature of tokenization is dependent on speech content rather than the length. Not sure which. In either case, I bet OpenAI is doing the same optimization under the hood and keeping the savings for themselves.
- CSMastermind 1y ago> to set your YouTube speed back down to 1x Is it common for people to watch Youtube sped up? I've heard of people doing this for podcasts and audiobooks and never understood it all that much there. Just feels like 'skimming' a real book instead of actually reading it.
- Feathercrown 1y agoSome people talk slower than your natural listening speed. It's less like skimming and more like if some books used 36pt font and you normalized the size back down to a comfortable information-dense size.
- Eezee 1y agoThat's completely different. Imagine you are reading a book and the words only get revealed to you at 1 word a second. You would get annoyed if your natural reading speed was higher than that. Same with a video. A lot of people speak considerably slower than you could process the information they are conveying, so you speed it up. You still get the same content and are not skipping parts as you would when skimming a book.
- keithxm23 1y agoOften, I'll come across speakers who just speak slowly and listening at 1.5x or 2x barely feels sped-up. Additionally, the brain tends to adjust to a faster talking speed very quickly. If I'm watching an average-paced person talk and speed them up by 2x, the first couple minutes of listening might be difficult and will require more intent-listening. However, the brain starts processing it as the new normal and it does not feel sped-up anymore. To the extent that if I go back to 1x, it feels like the speaker is way too slow.
- 83 1y ago>>Just feels like 'skimming' a real book instead of actually reading it. That's the goal for me lately. I primarily use Youtube for technical assistance (where are the screws to adjust this carburetor?, how do I remove this brake hub?, etc). There used to be short 1 to 2m videos on this kind of stuff but nowadays I have to suffer through a 10-15 minute video with multiple ad breaks. So now I always watch youtube at 2x speed while rapidly jumping the slider forward to find relevant portions.
- niutech 1y agoOr the OP could just use NotebookLM for free, which has text & video summarization built-in, without need for any trimming.