3 ms·

I see from comments I'm far from the only one using AI to summarise videos before deciding whether to watch them. Reminds of the meme "why spend 10 minutes doi

by nomilk 3y ago

I see from comments I'm far from the only one using AI to summarise videos before deciding whether to watch them. Reminds of the meme "why spend 10 minutes doing something when you can spend a week automating it". i.e. "why spend an hour watching a talk when you can spend 5 hours summarising it with AI and debating the summary's accuracy". This sounds silly but potential gains from learning AI summarisation tooling/flows are large, hence why it warrants discussion. Learning how to summarise effectively might save hours per week and improve decisions about which sources deserve our limited time/attention.

theGnuMe 3y ago
What ai tool do you use to summarize?
63 3y ago
> and debating the summary's accuracy Perhaps consider simply reading the description for an accurate summary. From the description: > Abstract: In this talk I’ll highlight several exciting trends in the field of AI and machine learning. Through a combination of improved algorithms and major efficiency improvements in ML-specialized hardware, we are now able to build much more capable, general purpose machine learning systems than ever before. As one example of this, I’ll give an overview of the Gemini family of multimodal models and their capabilities. These new models and approaches have dramatic implications for applying ML to many problems in the world, and I’ll highlight some of these applications in science, engineering, and health. This talk will present work done by many people at Google.
- nomilk 3y ago
  In this case the video description contains a useful Abstract. AI summaries can offer additional value though, going into more/less detail (as desired), and allowing you ask follow up questions to drill into anything potentially of interest.
- swyx 3y ago
  sure its an accurate summary, but is it at a granularity or specificity that you want? LLM summaries lets you move around the latent space of summaries and you probably dont agree with the one chosen for youtube descriprtions.
bmitc 3y ago
I feel like I'm missing some boat, but I'm not sure what boat it is. These "AI" systems seem very superficial to me, and they give me the same feeling as VR does. When I see VR be some terrible approximation of reality, it just makes me feel like I'm wasting my time in it, when I could go experience the real thing. Same with AI "augmentation" tooling. Why don't I just read a book instead of getting some unpredictable (or predictably unpredictable) synposis? It's not like there's too much specific information there. These tools are just exploding the amount of unspecific information. Who has ever said: "hey, I have too much information for building this system or learning this topic"? Basically no one. It's just going to move everything to the middle of the Bell curve, leaving the wings to die in obscurity.
- nomilk 3y ago
  If you know a book’s worth reading, going ahead and reading it works well. But for a lot of books/talks there’s competition for time - eg my bookshelf has 20 half read books (this is after triaging out the ones that aren’t worthy of my time) - any tooling that can help better determine where to invest tens or hundreds or hours of my time is a win. Regarding accuracy, I think we’re at a tipping point where ease of use and accuracy is starting to make it worth the effort. For example Bard seems to know about youtube videos (just a couple of months ago you’d have to download it -> audio to text -> feed into a LLM). So the combination of greater accuracy and much easier to use make it worth considering.
  - alex_smart 3y ago
    IMHO, the good old method of skimming through the table of contents, reading the preface and perhaps the first couple of chapters is going to be a much higher fidelity indicator of whether a book is worth your time than reading an AI generated summary.
  - ceruleanseas 3y ago
    LLM accuracy is so bad, especially in summarization, that I now have to fact check google search results because they’ve been repeatedly wrong about things like the hours restaurants are open.
ren_engineer 3y ago
I've A/B tested this with webinars and the tools I've tried tend to miss some really valuable/interesting stuff even when I give it the full transcript. Same goes for when I try to use ChatGPT or other tools for full interactive analysis, even when I basically hand it what I'm looking for as if I hadn't watched the video it will leave out the critical information
tomrod 3y ago
What is your workflow for this, if you don't mind me asking?
- nomilk 3y ago
  My approach wasn't fancy, just asked bard (aka gemini). I was drawn to bard/gemini for this since the source video is on youtube, so figured google would better support its related service (although that was an arbitrary hunch) https://imgur.com/a/psb64IP https://imgur.com/a/psb64IP
- sammyatman 3y ago
  Try out www.askyoutube.ai!
- jerpint 3y ago
  If you’re interested I did a YouTube video and short blog post about it https://www.jerpint.io/blog/yougptube/ https://www.jerpint.io/blog/yougptube/ https://www.youtube.com/watch?v=WtMrp2hp94E https://www.youtube.com/watch?v=WtMrp2hp94E
  - kendalf89 3y ago
    This is pretty cool. Would it be possible to just stream the audio directly into Whisper, maybe using something like vlc, at x2 play speed to get the summary faster?
    - jerpint 3y ago
      Probably, the openAI api got a lot better since I made that post, though if you stream audio at 2x speed you have to expect a drop in quality since on average most clips whisper is trained on are not at 2x
name_nick_sex_m 3y ago
What tool do you use to summarize video?
- kirill5pol 3y ago
  (Little self-plug) I made a tool that’s pretty relevant https://www.platoedu.org/videos/oSCRZkSQ1CE/watch https://www.platoedu.org/videos/oSCRZkSQ1CE/watch It's not really giving summaries but gives topic/section timestamps and highlights what was discussed. (Main focus is actually making mini-courses off of YouTube videos but I found the section summaries really useful for figuring out which parts to watch)
- nomilk 3y ago
  (since it's a youtube video) I used bard/gemini: https://imgur.com/a/psb64IP https://imgur.com/a/psb64IP I have no idea if it's the best (or even a good) tool. Other commenters suggest some other tools (for both text summaries and condensed video summaries - a sort of 'highlights reel'): https://news.ycombinator.com/item?id=39435930 https://news.ycombinator.com/item?id=39435930 https://news.ycombinator.com/item?id=39435964 https://news.ycombinator.com/item?id=39435964
sammyatman 3y ago
This is exactly why I built https://www.askyoutube.ai https://www.askyoutube.ai. It helps you figure out if a video has the answer you want before you spend time watching it. It does this by aggregating information from multiple videos in one-go. I don't think it completely replaces watching videos in some cases but it definitely helps you skip the fluff and even guides you to the right point in the video.
- esafak 3y ago
  Do you transcribe the videos or use the captions, because GPT4 can already do the latter?
  - sammyatman 3y ago
    It can be either depending on the mode, I don't think GPT4 can already do the latter though.
castles 3y ago
> Learning how to summarise effectively might save hours per week and improve decisions about which sources deserve our limited time/attention. If you like summaries, you'll probably love de-summaries (WIP): https://socontextual.com/ https://socontextual.com/
munificent 3y ago
1. Author spends a week producing a video when writing an article would have taken a day. 2. Viewer spends hours summarizing the video to an article so they don't have to watch it. P R O G R E S S
kirill5pol 3y ago
I made a tool that might be interesting for people here! https://www.platoedu.org/videos/oSCRZkSQ1CE/watch https://www.platoedu.org/videos/oSCRZkSQ1CE/watch It's not really giving summaries but gives topic/section timestamps and highlights what was discussed (for example: The Transformer Model (21:06 - 24:48) - Introduction of the Transformer model as a more efficient alternative to recurrent models for language processing) The main focus is actually creating Anki-like spaced repetition questions/flashcards for videos and lectures you watch to retain knowledge, but I found the section information quite helpful for finding which parts of the video contain the info relating to topics/concepts

by nomilk 3y ago

theGnuMe 3y ago

What ai tool do you use to summarize?

63 3y ago

> and debating the summary's accuracy Perhaps consider simply reading the description for an accurate summary. From the description: > Abstract: In this talk I’ll highlight several exciting trends in the field of AI and machine learning. Through a combination of improved algorithms and major efficiency improvements in ML-specialized hardware, we are now able to build much more capable, general purpose machine learning systems than ever before. As one example of this, I’ll give an overview of the Gemini family of multimodal models and their capabilities. These new models and approaches have dramatic implications for applying ML to many problems in the world, and I’ll highlight some of these applications in science, engineering, and health. This talk will present work done by many people at Google.

nomilk 3y ago
In this case the video description contains a useful Abstract. AI summaries can offer additional value though, going into more/less detail (as desired), and allowing you ask follow up questions to drill into anything potentially of interest.
swyx 3y ago
sure its an accurate summary, but is it at a granularity or specificity that you want? LLM summaries lets you move around the latent space of summaries and you probably dont agree with the one chosen for youtube descriprtions.

bmitc 3y ago

I feel like I'm missing some boat, but I'm not sure what boat it is. These "AI" systems seem very superficial to me, and they give me the same feeling as VR does. When I see VR be some terrible approximation of reality, it just makes me feel like I'm wasting my time in it, when I could go experience the real thing. Same with AI "augmentation" tooling. Why don't I just read a book instead of getting some unpredictable (or predictably unpredictable) synposis? It's not like there's too much specific information there. These tools are just exploding the amount of unspecific information. Who has ever said: "hey, I have too much information for building this system or learning this topic"? Basically no one. It's just going to move everything to the middle of the Bell curve, leaving the wings to die in obscurity.

nomilk 3y ago
If you know a book’s worth reading, going ahead and reading it works well. But for a lot of books/talks there’s competition for time - eg my bookshelf has 20 half read books (this is after triaging out the ones that aren’t worthy of my time) - any tooling that can help better determine where to invest tens or hundreds or hours of my time is a win. Regarding accuracy, I think we’re at a tipping point where ease of use and accuracy is starting to make it worth the effort. For example Bard seems to know about youtube videos (just a couple of months ago you’d have to download it -> audio to text -> feed into a LLM). So the combination of greater accuracy and much easier to use make it worth considering.
- alex_smart 3y ago
  IMHO, the good old method of skimming through the table of contents, reading the preface and perhaps the first couple of chapters is going to be a much higher fidelity indicator of whether a book is worth your time than reading an AI generated summary.
- ceruleanseas 3y ago
  LLM accuracy is so bad, especially in summarization, that I now have to fact check google search results because they’ve been repeatedly wrong about things like the hours restaurants are open.

ren_engineer 3y ago

I've A/B tested this with webinars and the tools I've tried tend to miss some really valuable/interesting stuff even when I give it the full transcript. Same goes for when I try to use ChatGPT or other tools for full interactive analysis, even when I basically hand it what I'm looking for as if I hadn't watched the video it will leave out the critical information

tomrod 3y ago

What is your workflow for this, if you don't mind me asking?

nomilk 3y ago
My approach wasn't fancy, just asked bard (aka gemini). I was drawn to bard/gemini for this since the source video is on youtube, so figured google would better support its related service (although that was an arbitrary hunch) https://imgur.com/a/psb64IP https://imgur.com/a/psb64IP
sammyatman 3y ago
Try out www.askyoutube.ai!
jerpint 3y ago
If you’re interested I did a YouTube video and short blog post about it https://www.jerpint.io/blog/yougptube/ https://www.jerpint.io/blog/yougptube/ https://www.youtube.com/watch?v=WtMrp2hp94E https://www.youtube.com/watch?v=WtMrp2hp94E
- kendalf89 3y ago
  This is pretty cool. Would it be possible to just stream the audio directly into Whisper, maybe using something like vlc, at x2 play speed to get the summary faster?
  - jerpint 3y ago
    Probably, the openAI api got a lot better since I made that post, though if you stream audio at 2x speed you have to expect a drop in quality since on average most clips whisper is trained on are not at 2x

name_nick_sex_m 3y ago

What tool do you use to summarize video?

kirill5pol 3y ago
(Little self-plug) I made a tool that’s pretty relevant https://www.platoedu.org/videos/oSCRZkSQ1CE/watch https://www.platoedu.org/videos/oSCRZkSQ1CE/watch It's not really giving summaries but gives topic/section timestamps and highlights what was discussed. (Main focus is actually making mini-courses off of YouTube videos but I found the section summaries really useful for figuring out which parts to watch)
nomilk 3y ago
(since it's a youtube video) I used bard/gemini: https://imgur.com/a/psb64IP https://imgur.com/a/psb64IP I have no idea if it's the best (or even a good) tool. Other commenters suggest some other tools (for both text summaries and condensed video summaries - a sort of 'highlights reel'): https://news.ycombinator.com/item?id=39435930 https://news.ycombinator.com/item?id=39435930 https://news.ycombinator.com/item?id=39435964 https://news.ycombinator.com/item?id=39435964

sammyatman 3y ago

This is exactly why I built https://www.askyoutube.ai https://www.askyoutube.ai. It helps you figure out if a video has the answer you want before you spend time watching it. It does this by aggregating information from multiple videos in one-go. I don't think it completely replaces watching videos in some cases but it definitely helps you skip the fluff and even guides you to the right point in the video.

esafak 3y ago
Do you transcribe the videos or use the captions, because GPT4 can already do the latter?
- sammyatman 3y ago
  It can be either depending on the mode, I don't think GPT4 can already do the latter though.

castles 3y ago

> Learning how to summarise effectively might save hours per week and improve decisions about which sources deserve our limited time/attention. If you like summaries, you'll probably love de-summaries (WIP): https://socontextual.com/ https://socontextual.com/

munificent 3y ago

1. Author spends a week producing a video when writing an article would have taken a day. 2. Viewer spends hours summarizing the video to an article so they don't have to watch it. P R O G R E S S

kirill5pol 3y ago

I made a tool that might be interesting for people here! https://www.platoedu.org/videos/oSCRZkSQ1CE/watch https://www.platoedu.org/videos/oSCRZkSQ1CE/watch It's not really giving summaries but gives topic/section timestamps and highlights what was discussed (for example: The Transformer Model (21:06 - 24:48) - Introduction of the Transformer model as a more efficient alternative to recurrent models for language processing) The main focus is actually creating Anki-like spaced repetition questions/flashcards for videos and lectures you watch to retain knowledge, but I found the section information quite helpful for finding which parts of the video contain the info relating to topics/concepts