12 ms·

Now I can just print that video

276 points by pforret 3y ago

paledot 3y ago
The copyright notice on the output is a poor choice, since you almost certainly do not own the copyright to any of the content. You've gone to impressive lengths to ensure that the result is true to the source material, which means that there is no claim to this being a transformative work. (Very cool and useful project, though.)
pforret 3y ago
Using yt-dlp, ffmpeg and various AI services to print videos (e.g. cooking IG reels)
Gys 3y ago
Could have been a Show HN
polygamous_bat 3y ago
You can also use software to detect “cuts” in the video, which can be used to improve the frame-extraction over just getting six evenly spaced frames from the video.
- patates 3y ago
  Not the post author but I tried this with ffmpeg and failed. Do you (does anyone) want to share some pointers?
  - murrain 3y ago
    PySceneDetect (https://www.scenedetect.com/ https://www.scenedetect.com/) might be useful.
  - markolson_ 3y ago
    I used something like this a few years ago in a project sort of similar to this one. There's a bunch of parsing and processing to do with that, and the "0.3" value is ... fiddly, but it worked pretty well: ffprobe -show_frames -of compact=p=0 -f lavfi "movie=THE_VIDEO_FILE,select=gt(scene\,0.3)" -pretty`
    - pforret 3y ago
      I played with that too before. `ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)'" -vsync vfr frame-%2d.jpg` (from the repo pforret/filmpace) For this project, I want to find an A.I. solution for finding the most 'interesting' frames. Not even sure how to measure interestingness yet, might be the presence of text, the presence of a human ...
- anewhnaccount2 3y ago
  This is a task called "video summarization". See https://paperswithcode.com/task/video-summarization https://paperswithcode.com/task/video-summarization . I guess the whole project is something like summarizing from video + subtitles + text to pictures + text.
- deleted 3y ago
  [deleted]
binarymax 3y ago
No need to spend hours trying to get the text extraction just right - pass the raw extraction into GPT and ask for it to give you the recipe.
- kevincox 3y ago
  I was thinking the same thing. Extraction and basic formatting of information from human language is something that LLMs excel at. Especially if the result is being shown to a human so small mistakes can be tolerated.
  - Closi 3y ago
    It could also dramatically increase quality, look at the below from the example PDF from the page: > Garlic balsamic chicken, you'll be making over and over. By sear your chicken. I resin wine. Grab yourself a nice large bowl, extra virgin and olive oil. Balsamic glaze. Tomato paste. Honey, fresh lemon juice, garlic, Oregano fresh thyme, coat my chicken with this beautiful balsamic, no balsamic, left behind. Don't you dare waste the good thing. Right? Going in the oven at four twenty five degrees, about thirty ish minutes. Look yes. Fresh thyme, fresh parsley. This is so good. I can't wait. Win our winner. Oh If we run this through ChatGPT with some basic prompt engineering this becomes: > Start by searing your chicken in a pan. In a large bowl, combine extra virgin olive oil and balsamic glaze. Add tomato paste, honey, fresh lemon juice, garlic, oregano, and fresh thyme. Coat the chicken thoroughly with this balsamic mixture, ensuring no glaze is left behind. Preheat your oven to 425 degrees Fahrenheit. Place the coated chicken in the oven and bake for about 30 minutes. Once cooked, garnish with fresh thyme and fresh parsley. Serve and enjoy your delicious garlic balsamic chicken. (Note: The phrase "I resin wine" in the audio transcription seems unclear and is possibly a mishearing. I have omitted it as it does not appear to fit the context of the recipe.)
    - beepbooptheory 3y ago
      It's a little confused about it though.. It's not clear in the gpt version that "balsamic glaze" is what you are making by mixing the ingredients together, and makes it sound rather like some other ingredient you are mixing in. Granted, we have to decipher that a little bit in the first one too, but its not nearly as bad.
zoomablemind 3y ago
If the main challenge was 'not having the smartphone in the kitchen', then one possible solution could have been getting another screen dedicated to the kitchen. A tablet, a laptop, a small TV+Google Cast or such combination. It seems to be a proper media for 'printing' a video. Of course, choosing challenges and finding solutions is what drives fun.
- goda90 3y ago
  I device I think would be great for the kitchen is a large wrist-mounted, waterproof e-ink screen, curved to wrap around the wrist, with two large scroll buttons. The recipe could be loaded up via a linked smartphone or something, but then you have a device that you can touch with food covered hands and then wash it right alongside your hands later. Big screen so you don't have to squint or scroll frequently like you would on a smartwatch. E-ink so it works well despite bright kitchen lights and has low power consumption.
  - dheera 3y ago
    Honestly large 10-13 inch e-Ink tablets already work well for this as long as you're opening a PDF that stays put. Much like a physical book that doesn't move. Live web pages suck because they pop up annoyances every 5 seconds that you have to deal with while your hands are messy, and the scrolling jumps around against your will.
- chankstein38 3y ago
  To me the main problem this solves is having to rewatch the video over and over for each step. Most of the time it's like "Step 2: do thing" then quickly cuts to step 3 well before I could've finished step 2. So having it laid out like this is actually a decent format to receive recipes in.
  - anewhnaccount2 3y ago
    Exactly. Fiddling with your phone over and over again while your hands are wet/covered in flour etc. A paper sheet you can pin to the fridge or just get dirty is a reasonable solution imo.
jsharf 3y ago
Recommend passing the speech-to-text narration through a round of GPT4 API to correct for any transcription errors (use some prompt giving context that it's speech to text)
cloudking 3y ago
It's a very cool technical feat, but not something I would personally pay for. I'll just spend the 1-2 minutes to watch the video for free. Not trying to discourage you, just giving honest feedback. Launching the early landing page is a good idea to validate further.
xnzakg 3y ago
Wonder if Kagi's universal summarizer would work on recipe videos. It seems to do a decent job on YouTube videos, but those usually have cc built in.
Hugsun 3y ago
Great work! It's potentially useful and also hilarious.
benob 3y ago
For some reason I though the goal was to print (with a 3d printer) a 3d projection of the 4d content of the video. That would be cool...
- deleted 3y ago
  [deleted]
- a1o 3y ago
  I thought it would just print the dessert in a way I could eat. It would be much easier. :P
quartz 3y ago
Definitely would use this. Instructional video instead of step-by-step text is a personal pet peeve. I know it's a lot easier to just record a video to show something like "how to replace the battery on a cordless vacuum" or "removing a sink basin nut" but it's often such a painful experience for consumption (watch a moment, pause, scrub back and watch again, pause, continue, pause, all with potentially gloved hands often in tight working spaces).
- dheera 3y ago
  Another big annoyance is websites with recipes that do any of the following indcredibly bad UX patterns: - Big white page not showing any text or images until the entire page and its assets are downloaded, which means if you accidentally click something and go back you have to wait another several seconds for everything to load again - Pop up GDPR popup while hands are covered in flour and eggs - Pop up "would you like to subscribe to the newsletter" while hands are covered in sticky sauce - Pop up "buy this shit for 10% off" with a microscopic X button while something on high heat on the stove - Not specifying image height and width in CSS so that when user is looking at a piece of text and images above it load, the scrolling position jumps For these reasons alone I've largely stopped looking at the internet for recipes and turned to physical books, which are much better behaved.
  - BizarroLand 3y ago
    Don't forget the lazy-loading pages that don't properly set their flex box positions so that when you go to click on something a new link pops in at the place you were just about to click that takes you to a different page. I'm tempted to say that this is a dark pattern because when it happens to me is is almost always a "subscribe", "purchase", or "login" button.
    - robocat 3y ago
      > dark pattern I wonder how often dark patterns are the result of goal directed A/B testing? If the goal is set to "did the user click on an advert" - and the A/B changes are fuzzed CSS - then the results would be deviously dark.
atticora 3y ago
I saw a YouTube video by a guy who specializes in building D&D characters. He spends twenty minutes going into detail on each one, and then makes the pitch for subscribing to his Patreon account with something like "members get all the details in a convenient list so that you don't have to keep going back to this video." So he's using the same bit of friction that this article is trying to solve, to fill his rice bowl. It's a bit of a shame that fixing this problem for me will cause one for him.
- slingnow 3y ago
  Maybe if your business model includes putting things in an inconvenient format that could best be replaced by a bulleted list, you should rethink your business model.
  - sakjur 3y ago
    If that's truly what you're doing, sure. If you can reduce a movie to its screenplay without losing value, that's probably what you should be doing. A recipe is a great complement to a cooking video, but reducing the value of a cooking video's value to the recipe is oversimplified. I wish we'd go the other way, where free text content is complemented by paid-for audio or video commentary. But it has to be a very dull video for a bullet list to be a good replacement (and a bad bullet list to be able to capture what a good video production can convey).
    - xg15 3y ago
      Yeah no. There are probably amazing and insightful cooking videos out there, but the moment you intentionally withold the recipe unless people pay, you're using a disadvantage of the medium for rent-seeking.
    - TeMPOraL 3y ago
      > A recipe is a great complement to a cooking video, but reducing the value of a cooking video's value to the recipe is oversimplified. There is a simple test to make here: is the complimentary recipe released to the viewer, so that they can read it conveniently and at their own pace? If the recipe is truly a complement to otherwise great cooking video, then releasing it is a no-brainer. If it's withheld, then one has to wonder why, and what the video publisher is afraid of.
gsa 3y ago
This is pretty cool but I'd like to see a well-formatted recipe, not a transcript. I prefer the markdown format for recipes so I worked on something like this earlier this year [0]. It fetches Youtube subs (with no audio processing like the video itself like this project) and returns a markdown with ingredients and steps. [0] https://github.com/gaganpreet/summarise-youtube-recipes https://github.com/gaganpreet/summarise-youtube-recipes
avgcorrection 3y ago
I could also need a service for trimming all of the fat from how-to articles. > We’ve all been there: we used the florb for too many glorbs and now it needs to be replaced. [...] > This is an experience that everyone at the staff of howto.biz.uk has had! [...] > But how do you replace a used-up florb? In this article we are going to show you how. [...] > [scan the next five paragraphs]
- NikxDa 3y ago
  Same with recipes, where the author frequently feels the need to reiterate their grand-grand-grandparents life history in 10 paragraphs before getting to the ingredients and step-by-step instructions. It's sad to see what SEO has become, really...
1970-01-01 3y ago
Filtering a video for true content is the real app. Print is simply the format you've chosen to express it.
tgsovlerkhgsel 3y ago
Based on the example shown on the page, the output doesn't seem very good. If that's one of the better examples the software produced, I don't think this will be useful in practice.
- pforret 3y ago
  This is one of the first results. The third, if I remember correctly. I got this running yesterday (Sunday), and I wanted to write the blog post first to test if there was any interest in this topic. Apparently, yes. Now I only have to do the remaining 80% ;-)
barrkel 3y ago
Great, a way to turn videos into something I can scan. Actually something I'd consider using.
hermannj314 3y ago
Do video formats support structured meta data to be embedded in them? If I make a video of me cooking, can I embed the recipe in the video, etc. Not just visually, but i.e. at 10s, I digitally insert the data "Add 1 cup red peppers". It isn't necessary a caption of something said or shown, just extra data. Could a video creator leave substantially more metadata in their videos? I always assumed the pop-up metadata was externally stored and timestamp synced. Is there a way to embed it?
- pforret 3y ago
  That sounds a bit like subtitles, or Timed Text. There are simple formats (just a text and the moment it should appear) but some formats support changing the position, color, font… most of the times this would be embedded in an extra sidecar file like an .srt or a .sub
- thomastjeffery 3y ago
  It would be better all-around to just have that data in a separate file with timestamps.
RBerenguel 3y ago
I kind of wrote something for this a few years ago: https://github.com/rberenguel/glancer https://github.com/rberenguel/glancer [edited a fat-fingered copy-paste] The use-case is technical videos (like from conferences) I’m interested, but not enough to invest 20-60 minutes. Haven’t used it in a few months so the yt-dlp commands may need updating.
- dan-g 3y ago
  Sadly getting a 404 here–maybe this is a private repository?
  - dcuthbertson 3y ago
    I think RBerenguel intended https://github.com/rberenguel/glancer https://github.com/rberenguel/glancer
    - RBerenguel 3y ago
      Thanks, I fatfingered the copy paste on my phone :/
ada1981 3y ago
I think you could send all of that to GPT4 and ask it to read it and provide you with a step by step instruction : recipie and it would do so easily. I didn’t see how that print out would be super useful, it’s not the complete step by step is it?
jusquan 3y ago
This is great, thank you for sharing! I wonder what the reverse would look like. More and more nowadays, I find myself first looking on YouTube for tutorials and walkthroughs, even if they wind up being more verbose than their written counterparts.
IgorPartola 3y ago
I actually wonder if in the limit of video encoding we could just get a diffusion model that can in real time render realistic video based on a script. Then downloading a movie is just downloading a few megabytes of a prompt and you get a movie playing based off it locally.
- parthianshotgun 3y ago
  Wouldn't it be non-deterministic? (Legit question, I'm new to this)
  - lucubratory 3y ago
    It depends on the specifics, but in general if your settings are correct and the hardware isn't doing something whacky/buggy then it should be deterministic. The math itself is deterministic, the only randomness comes from temperature (intentional randomness introduced in software) and some bugginess with GPUs that weren't designed for this workload.
- TeMPOraL 3y ago
  Maybe. The only problem I see is economical. Sure, sending over a sequence of prompts, instead of sequence of frames, is going to be a huge storage and bandwidth saver. However, you're going to pay for it dearly, in compute, whenever you want to watch such a live-generated video. In almost all cases, it's vastly better to use more storage than to use more compute, for the same reason that, if you need to keep something to stay above ground level, you're better off placing it on a table or bolting it on a wall, instead of attaching it to a jet engine pointing downwards, firing for TWR=1.
  - IgorPartola 3y ago
    That’s very true and I did make that comment mostly in jest. I think the idea of art generated like that has a distinctly dystopian feel. I think one thing that might push this idea closer to reality is dedicated hardware. It would also allow you to do things like enjoy movies with your favorite actors. “Netflix, show me Dune but with Dustin Hoffman as Paul”.
TrevorJ 3y ago
As someone who's learning was significantly accelerated by the "written tutorial" phase of the internet this would be a really great little tool. I find video tutorials to be far more cumbersome than text+ images.
adr1an 3y ago
Cool! I had the same project idea recently. You may be interested in this for the step of speech2text: https://github.com/SYSTRAN/faster-whisper https://github.com/SYSTRAN/faster-whisper
ForOldHack 3y ago
Ha! Print that video? Yes, but can you FIND THE PRINTER? ---- I humbly apologize, I thought this was some joke, or errant stupidity. Its not. This person has put some very serious thought into not only getting it to work, but to make it useful. Very useful. You have earned my Upvote, and recommendation. Thank you Mr Forret. Thank you.
mannyv 3y ago
If there are YouTube-generated captions you can get yt-dlp to download them when you download the video.
incahoots 3y ago
Oh wow....this will incredibly useful for the influx of recent home improvement videos I've been watching lately.
einpoklum 3y ago
Ok, so: * It does not print the video frames as a 3D object. * Despite what the graphic at the link suggests, it doesn't 3D-print food it extracts a recipe with images and text from a video, automatically.
lucubratory 3y ago
An evolution of this process would make it feasible to do retrieval-augmented generation using information from video content. I've thought about trying to do this to improve the (already impressive) abilities LLM's possess as a creative writing assistant/rubber ducky; a lot of good writing advice is on YouTube in the form of video essays, tutorials, lectures, etc.
roomey 3y ago
These tik tok videos are pretty short right? Why not just get a note book and write down the instructions. You could even do a little line drawing of the important bits. You could keep this "cook" book in your kitchen, and maybe pass it to one of your kids (just an example) when they move out or something.

276 points by pforret 3y ago

The copyright notice on the output is a poor choice, since you almost certainly do not own the copyright to any of the content. You've gone to impressive lengths to ensure that the result is true to the source material, which means that there is no claim to this being a transformative work. (Very cool and useful project, though.)

pforret 3y ago

Using yt-dlp, ffmpeg and various AI services to print videos (e.g. cooking IG reels)

Gys 3y ago

Could have been a Show HN

polygamous_bat 3y ago

You can also use software to detect “cuts” in the video, which can be used to improve the frame-extraction over just getting six evenly spaced frames from the video.

patates 3y ago
Not the post author but I tried this with ffmpeg and failed. Do you (does anyone) want to share some pointers?
- murrain 3y ago
  PySceneDetect (https://www.scenedetect.com/ https://www.scenedetect.com/) might be useful.
- markolson_ 3y ago
  I used something like this a few years ago in a project sort of similar to this one. There's a bunch of parsing and processing to do with that, and the "0.3" value is ... fiddly, but it worked pretty well: ffprobe -show_frames -of compact=p=0 -f lavfi "movie=THE_VIDEO_FILE,select=gt(scene\,0.3)" -pretty`
  - pforret 3y ago
    I played with that too before. `ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)'" -vsync vfr frame-%2d.jpg` (from the repo pforret/filmpace) For this project, I want to find an A.I. solution for finding the most 'interesting' frames. Not even sure how to measure interestingness yet, might be the presence of text, the presence of a human ...
anewhnaccount2 3y ago
This is a task called "video summarization". See https://paperswithcode.com/task/video-summarization https://paperswithcode.com/task/video-summarization . I guess the whole project is something like summarizing from video + subtitles + text to pictures + text.
deleted 3y ago
[deleted]

binarymax 3y ago

No need to spend hours trying to get the text extraction just right - pass the raw extraction into GPT and ask for it to give you the recipe.

kevincox 3y ago
I was thinking the same thing. Extraction and basic formatting of information from human language is something that LLMs excel at. Especially if the result is being shown to a human so small mistakes can be tolerated.
- Closi 3y ago
  It could also dramatically increase quality, look at the below from the example PDF from the page: > Garlic balsamic chicken, you'll be making over and over. By sear your chicken. I resin wine. Grab yourself a nice large bowl, extra virgin and olive oil. Balsamic glaze. Tomato paste. Honey, fresh lemon juice, garlic, Oregano fresh thyme, coat my chicken with this beautiful balsamic, no balsamic, left behind. Don't you dare waste the good thing. Right? Going in the oven at four twenty five degrees, about thirty ish minutes. Look yes. Fresh thyme, fresh parsley. This is so good. I can't wait. Win our winner. Oh If we run this through ChatGPT with some basic prompt engineering this becomes: > Start by searing your chicken in a pan. In a large bowl, combine extra virgin olive oil and balsamic glaze. Add tomato paste, honey, fresh lemon juice, garlic, oregano, and fresh thyme. Coat the chicken thoroughly with this balsamic mixture, ensuring no glaze is left behind. Preheat your oven to 425 degrees Fahrenheit. Place the coated chicken in the oven and bake for about 30 minutes. Once cooked, garnish with fresh thyme and fresh parsley. Serve and enjoy your delicious garlic balsamic chicken. (Note: The phrase "I resin wine" in the audio transcription seems unclear and is possibly a mishearing. I have omitted it as it does not appear to fit the context of the recipe.)
  - beepbooptheory 3y ago
    It's a little confused about it though.. It's not clear in the gpt version that "balsamic glaze" is what you are making by mixing the ingredients together, and makes it sound rather like some other ingredient you are mixing in. Granted, we have to decipher that a little bit in the first one too, but its not nearly as bad.

zoomablemind 3y ago

If the main challenge was 'not having the smartphone in the kitchen', then one possible solution could have been getting another screen dedicated to the kitchen. A tablet, a laptop, a small TV+Google Cast or such combination. It seems to be a proper media for 'printing' a video. Of course, choosing challenges and finding solutions is what drives fun.

goda90 3y ago
I device I think would be great for the kitchen is a large wrist-mounted, waterproof e-ink screen, curved to wrap around the wrist, with two large scroll buttons. The recipe could be loaded up via a linked smartphone or something, but then you have a device that you can touch with food covered hands and then wash it right alongside your hands later. Big screen so you don't have to squint or scroll frequently like you would on a smartwatch. E-ink so it works well despite bright kitchen lights and has low power consumption.
- dheera 3y ago
  Honestly large 10-13 inch e-Ink tablets already work well for this as long as you're opening a PDF that stays put. Much like a physical book that doesn't move. Live web pages suck because they pop up annoyances every 5 seconds that you have to deal with while your hands are messy, and the scrolling jumps around against your will.
chankstein38 3y ago
To me the main problem this solves is having to rewatch the video over and over for each step. Most of the time it's like "Step 2: do thing" then quickly cuts to step 3 well before I could've finished step 2. So having it laid out like this is actually a decent format to receive recipes in.
- anewhnaccount2 3y ago
  Exactly. Fiddling with your phone over and over again while your hands are wet/covered in flour etc. A paper sheet you can pin to the fridge or just get dirty is a reasonable solution imo.

jsharf 3y ago

Recommend passing the speech-to-text narration through a round of GPT4 API to correct for any transcription errors (use some prompt giving context that it's speech to text)

cloudking 3y ago

It's a very cool technical feat, but not something I would personally pay for. I'll just spend the 1-2 minutes to watch the video for free. Not trying to discourage you, just giving honest feedback. Launching the early landing page is a good idea to validate further.

xnzakg 3y ago

Wonder if Kagi's universal summarizer would work on recipe videos. It seems to do a decent job on YouTube videos, but those usually have cc built in.

Hugsun 3y ago

Great work! It's potentially useful and also hilarious.

benob 3y ago

For some reason I though the goal was to print (with a 3d printer) a 3d projection of the 4d content of the video. That would be cool...

deleted 3y ago
[deleted]
a1o 3y ago
I thought it would just print the dessert in a way I could eat. It would be much easier. :P

quartz 3y ago

Definitely would use this. Instructional video instead of step-by-step text is a personal pet peeve. I know it's a lot easier to just record a video to show something like "how to replace the battery on a cordless vacuum" or "removing a sink basin nut" but it's often such a painful experience for consumption (watch a moment, pause, scrub back and watch again, pause, continue, pause, all with potentially gloved hands often in tight working spaces).

dheera 3y ago
Another big annoyance is websites with recipes that do any of the following indcredibly bad UX patterns: - Big white page not showing any text or images until the entire page and its assets are downloaded, which means if you accidentally click something and go back you have to wait another several seconds for everything to load again - Pop up GDPR popup while hands are covered in flour and eggs - Pop up "would you like to subscribe to the newsletter" while hands are covered in sticky sauce - Pop up "buy this shit for 10% off" with a microscopic X button while something on high heat on the stove - Not specifying image height and width in CSS so that when user is looking at a piece of text and images above it load, the scrolling position jumps For these reasons alone I've largely stopped looking at the internet for recipes and turned to physical books, which are much better behaved.
- BizarroLand 3y ago
  Don't forget the lazy-loading pages that don't properly set their flex box positions so that when you go to click on something a new link pops in at the place you were just about to click that takes you to a different page. I'm tempted to say that this is a dark pattern because when it happens to me is is almost always a "subscribe", "purchase", or "login" button.
  - robocat 3y ago
    > dark pattern I wonder how often dark patterns are the result of goal directed A/B testing? If the goal is set to "did the user click on an advert" - and the A/B changes are fuzzed CSS - then the results would be deviously dark.

atticora 3y ago

I saw a YouTube video by a guy who specializes in building D&D characters. He spends twenty minutes going into detail on each one, and then makes the pitch for subscribing to his Patreon account with something like "members get all the details in a convenient list so that you don't have to keep going back to this video." So he's using the same bit of friction that this article is trying to solve, to fill his rice bowl. It's a bit of a shame that fixing this problem for me will cause one for him.

slingnow 3y ago
Maybe if your business model includes putting things in an inconvenient format that could best be replaced by a bulleted list, you should rethink your business model.
- sakjur 3y ago
  If that's truly what you're doing, sure. If you can reduce a movie to its screenplay without losing value, that's probably what you should be doing. A recipe is a great complement to a cooking video, but reducing the value of a cooking video's value to the recipe is oversimplified. I wish we'd go the other way, where free text content is complemented by paid-for audio or video commentary. But it has to be a very dull video for a bullet list to be a good replacement (and a bad bullet list to be able to capture what a good video production can convey).
  - xg15 3y ago
    Yeah no. There are probably amazing and insightful cooking videos out there, but the moment you intentionally withold the recipe unless people pay, you're using a disadvantage of the medium for rent-seeking.
  - TeMPOraL 3y ago
    > A recipe is a great complement to a cooking video, but reducing the value of a cooking video's value to the recipe is oversimplified. There is a simple test to make here: is the complimentary recipe released to the viewer, so that they can read it conveniently and at their own pace? If the recipe is truly a complement to otherwise great cooking video, then releasing it is a no-brainer. If it's withheld, then one has to wonder why, and what the video publisher is afraid of.

gsa 3y ago

This is pretty cool but I'd like to see a well-formatted recipe, not a transcript. I prefer the markdown format for recipes so I worked on something like this earlier this year [0]. It fetches Youtube subs (with no audio processing like the video itself like this project) and returns a markdown with ingredients and steps. [0] https://github.com/gaganpreet/summarise-youtube-recipes https://github.com/gaganpreet/summarise-youtube-recipes

avgcorrection 3y ago

I could also need a service for trimming all of the fat from how-to articles. > We’ve all been there: we used the florb for too many glorbs and now it needs to be replaced. [...] > This is an experience that everyone at the staff of howto.biz.uk has had! [...] > But how do you replace a used-up florb? In this article we are going to show you how. [...] > [scan the next five paragraphs]

NikxDa 3y ago
Same with recipes, where the author frequently feels the need to reiterate their grand-grand-grandparents life history in 10 paragraphs before getting to the ingredients and step-by-step instructions. It's sad to see what SEO has become, really...

1970-01-01 3y ago

Filtering a video for true content is the real app. Print is simply the format you've chosen to express it.

tgsovlerkhgsel 3y ago

Based on the example shown on the page, the output doesn't seem very good. If that's one of the better examples the software produced, I don't think this will be useful in practice.

pforret 3y ago
This is one of the first results. The third, if I remember correctly. I got this running yesterday (Sunday), and I wanted to write the blog post first to test if there was any interest in this topic. Apparently, yes. Now I only have to do the remaining 80% ;-)

barrkel 3y ago

Great, a way to turn videos into something I can scan. Actually something I'd consider using.

hermannj314 3y ago

Do video formats support structured meta data to be embedded in them? If I make a video of me cooking, can I embed the recipe in the video, etc. Not just visually, but i.e. at 10s, I digitally insert the data "Add 1 cup red peppers". It isn't necessary a caption of something said or shown, just extra data. Could a video creator leave substantially more metadata in their videos? I always assumed the pop-up metadata was externally stored and timestamp synced. Is there a way to embed it?

pforret 3y ago
That sounds a bit like subtitles, or Timed Text. There are simple formats (just a text and the moment it should appear) but some formats support changing the position, color, font… most of the times this would be embedded in an extra sidecar file like an .srt or a .sub
thomastjeffery 3y ago
It would be better all-around to just have that data in a separate file with timestamps.

RBerenguel 3y ago

I kind of wrote something for this a few years ago: https://github.com/rberenguel/glancer https://github.com/rberenguel/glancer [edited a fat-fingered copy-paste] The use-case is technical videos (like from conferences) I’m interested, but not enough to invest 20-60 minutes. Haven’t used it in a few months so the yt-dlp commands may need updating.

dan-g 3y ago
Sadly getting a 404 here–maybe this is a private repository?
- dcuthbertson 3y ago
  I think RBerenguel intended https://github.com/rberenguel/glancer https://github.com/rberenguel/glancer
  - RBerenguel 3y ago
    Thanks, I fatfingered the copy paste on my phone :/

ada1981 3y ago

I think you could send all of that to GPT4 and ask it to read it and provide you with a step by step instruction : recipie and it would do so easily. I didn’t see how that print out would be super useful, it’s not the complete step by step is it?

jusquan 3y ago

This is great, thank you for sharing! I wonder what the reverse would look like. More and more nowadays, I find myself first looking on YouTube for tutorials and walkthroughs, even if they wind up being more verbose than their written counterparts.

IgorPartola 3y ago

I actually wonder if in the limit of video encoding we could just get a diffusion model that can in real time render realistic video based on a script. Then downloading a movie is just downloading a few megabytes of a prompt and you get a movie playing based off it locally.

parthianshotgun 3y ago
Wouldn't it be non-deterministic? (Legit question, I'm new to this)
- lucubratory 3y ago
  It depends on the specifics, but in general if your settings are correct and the hardware isn't doing something whacky/buggy then it should be deterministic. The math itself is deterministic, the only randomness comes from temperature (intentional randomness introduced in software) and some bugginess with GPUs that weren't designed for this workload.
TeMPOraL 3y ago
Maybe. The only problem I see is economical. Sure, sending over a sequence of prompts, instead of sequence of frames, is going to be a huge storage and bandwidth saver. However, you're going to pay for it dearly, in compute, whenever you want to watch such a live-generated video. In almost all cases, it's vastly better to use more storage than to use more compute, for the same reason that, if you need to keep something to stay above ground level, you're better off placing it on a table or bolting it on a wall, instead of attaching it to a jet engine pointing downwards, firing for TWR=1.
- IgorPartola 3y ago
  That’s very true and I did make that comment mostly in jest. I think the idea of art generated like that has a distinctly dystopian feel. I think one thing that might push this idea closer to reality is dedicated hardware. It would also allow you to do things like enjoy movies with your favorite actors. “Netflix, show me Dune but with Dustin Hoffman as Paul”.

TrevorJ 3y ago

As someone who's learning was significantly accelerated by the "written tutorial" phase of the internet this would be a really great little tool. I find video tutorials to be far more cumbersome than text+ images.

adr1an 3y ago

Cool! I had the same project idea recently. You may be interested in this for the step of speech2text: https://github.com/SYSTRAN/faster-whisper https://github.com/SYSTRAN/faster-whisper

ForOldHack 3y ago

Ha! Print that video? Yes, but can you FIND THE PRINTER? ---- I humbly apologize, I thought this was some joke, or errant stupidity. Its not. This person has put some very serious thought into not only getting it to work, but to make it useful. Very useful. You have earned my Upvote, and recommendation. Thank you Mr Forret. Thank you.

mannyv 3y ago

If there are YouTube-generated captions you can get yt-dlp to download them when you download the video.

incahoots 3y ago

Oh wow....this will incredibly useful for the influx of recent home improvement videos I've been watching lately.

einpoklum 3y ago

Ok, so: * It does not print the video frames as a 3D object. * Despite what the graphic at the link suggests, it doesn't 3D-print food it extracts a recipe with images and text from a video, automatically.

lucubratory 3y ago

An evolution of this process would make it feasible to do retrieval-augmented generation using information from video content. I've thought about trying to do this to improve the (already impressive) abilities LLM's possess as a creative writing assistant/rubber ducky; a lot of good writing advice is on YouTube in the form of video essays, tutorials, lectures, etc.

roomey 3y ago

These tik tok videos are pretty short right? Why not just get a note book and write down the instructions. You could even do a little line drawing of the important bits. You could keep this "cook" book in your kitchen, and maybe pass it to one of your kids (just an example) when they move out or something.