16 ms·

Show HN: I created automatic subtitling app to boost short videos

45 points by arommelaere 2y ago

I made https://videofa.st https://videofa.st This is a web app to add automatic subtitles/captions into your short videos, I created this app because I was wasting so much time to add subtitles manually into my own social short videos and the quality was poor, but now it's easy and fast! I also needed to pay my bills and so I thought: why not help others getting access to this time saver app? That's how I decided to make it becomes a Saas and to share it with you today! The link is videofa.st

robsh 2y ago
I’d really like to see something like this optimized for regular subtitles usefulness, rather than for attention grabbing. Two subtle changes from regular subtitles to make it more fluid. First the words appear as they are spoken and from near the mouth (if on screen). From that point they fly down to form the phrase or sentence at the bottom of the screen. It would help avoid comedic spoilers when the punchline is out before it’s spoken. Ideally the AI processing could just happen real-time during playback, by looking at the existing subtitles and then refining the timing of each word using the audio and video.
- vasco 2y ago
  Have you actually considered the experience of watching something with words constantly flying around the screen?
- assono 2y ago
  you could try Happy Scribe, we do localisation with AI and expert linguists. Check it out at happyscribe.com
kawsper 2y ago
Interesting product, have you considered adding your app as a plugin to tools like Davinci Resolve or Premiere Pro to fit within existing workflows?
- qingcharles 2y ago
  +1 for this.
- swyx 2y ago
  > I also needed to pay my bills and so I thought: why not help others getting access to this time saver app? That's how I decided to make it becomes a Saas and to share it with you today! this probably answers your qtn
  - kawsper 2y ago
    It could be a paid plugin. There's several successful paid plugins for both applications.
- tkzed49 2y ago
  I really doubt most people are using either of those for TikToks. _Maybe_ YouTube shorts if they also produce long form content.
- arommelaere 2y ago
  I'm not sure how hard it is and if it's feasible but I will add it to my notes! Thanks for this suggestion! Do you know if plugins on those tools can "connect to the internet and run the process via my videofa.st servers" even if they are plugins?
qingcharles 2y ago
I signed up. I often have to caption videos for TikTok/Reels etc. I normally use CapCut mobile (desktop version has weak subtitling). It is a massive PITA to make compelling engaging captions like this. Going back and adding emojis etc is another PITA. I'm willing to give it a test. The pricing is a little steep for simply captioning, especially as a lot of short videos are very speculative and most won't make any return on investment. One idea I would love: the tool to identify different speakers and change the color of the captions. I had to do this manually the other day on an interview and it was a real time sink.
- cloakedcode 2y ago
  I’m in a similar boat where I produce about one video a day and will definitely give this a try. But if I I want to caption them all that means $149/mo because the $15 plan has a cap of 20 videos/mo. At that price I’d much more likely pay for DaVinci Resolve Studio one time at $295 to use their subtitle generator (and that fits into my existing workflow). If there was a better pricing plan for someone who has yet to see a dime from videos, I’d consider this long term.
  - dowakin 2y ago
    You could try `https://github.com/jianfch/stable-ts https://github.com/jianfch/stable-ts`, it's an improvement for OpenAI Whisper. It does pretty good job.
  - emmanueloga_ 2y ago
    whisper.cpp [1] has a karaoke example that uses ffmpeg's drawtext filter to display rudimentary karaoke-like captions. It also supports diarisation. Perhaps it could be a starting point to create a better script that does what you need. -- 1: https://github.com/ggerganov/whisper.cpp/blob/master/README.md#karaoke-style-movie-generation-experimental https://github.com/ggerganov/whisper.cpp/blob/master/README....
2024throwaway 2y ago
Short video content is cancer. I wish you the best, looks like a good solution. Don’t worry about me, I just hate it.
- makeitshine 2y ago
  I normally agree, but there are times when I want a short video. For example, my wife makes crochet and knitting videos. She has series which are 30-60s long and cover one technique. The opposite are people who want to waste your time with a 30 second intro, a talk about whatever and then spend 30s showing you the same technique. This is not ops target, I understand.
  - unglaublich 2y ago
    Okay! BUT ((why)) this VERY Annoying!? Format :head_explode:
  - Timwi 2y ago
    There's no reason people can't make 30 seconds-long videos. Your crochet/knitting use-case sounds perfect! The problem is with how YouTube treats short videos as its own thing, expects them to be in vertical video format (because rotating your phone is asking too much) and completely messes up the UI (can't even seek properly). I think channels with short videos that teach a technique or a nugget of knowledge could be very popular and very effective (educationally speaking) if it weren't for the capitalist enshittification.
    - makeitshine 2y ago
      It's funny you mention vertical videos. I'm currently working on a toy project using MobileNet which classifies YouTube thumbnails as clickbait or not, as per my preferences, and I classify every vertical video as such. It needs more training data but it's up to ～78% accuracy at the moment. Next I'm loading it in a Chrome extension.
- mrtksn 2y ago
  These days long videos are short videos with 10x fillers to optimize the runtime for advertisers. A video about this psychological phenomenon? Why don’t we start with defining what is psychology, a history of it and and some surprising fact before telling you this thing rahat takes 20s to tell? Also, let’s hear from our sponsors first. The really long videos are just background noise people play when doing other stuff. Really low quality garbage. IMHO short videos == bad is unjustified meme. Some TikTokers have mastered it, they will have an intro that is shorter than the actual content, you can decide if you want to watch this in just a few moments and then deliver you the money shot. The long 3 min videos that keep building up are insufferable though. Short videos are not really for the people with low attention span, it’s just that it takes a few seconds to explain anything and it takes 10,000 to master it. 22 minutes video about why the Catholic Churches look the way they they look isn’t a step towards mastering architecture or theology, it’s just 21 minutes of fillers like history and examples to deliver the idea that Catholicism is very centered around the church, therefor they design it to look like an authority. A 20 second TikTok video will be available to convey the same information without wasting your time and if you’re actually interested and master this stuff you can go read books or join a program that teaches the subject. It’s no surprise that some of the best tech videos on YouTube are by Fireship. Their videos are somewhere between 1 and 4 minutes long, including the jokes.
attentiongroper 2y ago
From an ethical point of view, have you considered your “attention grabbing tool” evil ? We really need our attention to ourselves, please stop groping it. Yours looks particularly evil.
- thorncorona 2y ago
  lots of people prefer subtitles to audio. i am one of them. this seems like it will do fine.
  - nullandvoid 2y ago
    I think it's the unnecessary insertion of emoji's next to words, and flashing of the current word that OP is talking to. No one is is arguing subtitles are bad, it's the way this tries to increase engagement through abusing attention mechanisms, that I also agree with OP is likely damaging to us. It's part of the many reasons I dislike the current short form media craze - horrible tactics like this are used more often than not.
    - unglaublich 2y ago
      Exactly. And since the human mind has limited attention capacity, I feel it desensitizes people from paying attention to "normal life". People look like zombies.
  - Timwi 2y ago
    Nobody said you couldn't have subtitles. It sounds like you want to impose subtitles on everyone just because you want them.
deleted 2y ago
[deleted]
lionkor 2y ago
I love that, as a demo, you're just stealing content. Keep it up
- arommelaere 2y ago
  The content is from a creative common video
aestetix 2y ago
This service is awesome. Just fyi, I sent you a message via the contact form on the website :)
- arommelaere 2y ago
  Thank you for your interest in videofa.st! I've replied to your message ;)
ale42 2y ago
I guess some people like this kind of videos, personally I can't stand it. Flashing subtitles that just look like an interference. I want to watch the video, not the damn subtitles. At a bare minimum, they should be optional!
- arommelaere 2y ago
  Thanks for your opinion, would provide slow/soft animations would solve this problem?
  - ale42 2y ago
    Good question, I'd have to try. Usually I simply don't want to see subtitles, unless the audio is in a foreign language I don't speak, or if I want to quickly check a video (e.g. a tutorial) without having audio, like during a meeting.
    - jfoster 2y ago
      Subtitles are extremely useful if you want to watch videos without audio, though. I think the whole idea of short videos is that you might be in-between doing things and be looking for some quick entertainment. In such a scenario, you're likely in a public place.
      - ale42 2y ago
        Agreed, but why embedding them in the video stream? Missing a proper subtitle support where the videos are supposed to be played?
        jfoster 2y ago
        This product seems to be targeted toward short videos, like TikTok, Instagram reels, or YouTube Shorts. I'm not sure, but I don't think any of those offer built in subtitle support yet? I know YouTube does on regular videos, but I don't think I've seen it for Shorts.
fareesh 2y ago
Which whisper model do you use?
pipe2devnull 2y ago
Ignoring any comments about wether or not short videos are good or bad, the demo seems like the subtitles are appearing and disappearing way too quickly, it’s a bit hard to read but maybe that’s just the video
- arommelaere 2y ago
  Thanks for your feedback, current subtitles are sync with speech speed, so yes the demo video might be too fast, maybe I should make a slower video
tamimio 2y ago
You should add the pricing in the main page, not hiding it behind the registration..

45 points by arommelaere 2y ago

robsh 2y ago

I’d really like to see something like this optimized for regular subtitles usefulness, rather than for attention grabbing. Two subtle changes from regular subtitles to make it more fluid. First the words appear as they are spoken and from near the mouth (if on screen). From that point they fly down to form the phrase or sentence at the bottom of the screen. It would help avoid comedic spoilers when the punchline is out before it’s spoken. Ideally the AI processing could just happen real-time during playback, by looking at the existing subtitles and then refining the timing of each word using the audio and video.

vasco 2y ago
Have you actually considered the experience of watching something with words constantly flying around the screen?
assono 2y ago
you could try Happy Scribe, we do localisation with AI and expert linguists. Check it out at happyscribe.com

kawsper 2y ago

Interesting product, have you considered adding your app as a plugin to tools like Davinci Resolve or Premiere Pro to fit within existing workflows?

qingcharles 2y ago
+1 for this.
swyx 2y ago
> I also needed to pay my bills and so I thought: why not help others getting access to this time saver app? That's how I decided to make it becomes a Saas and to share it with you today! this probably answers your qtn
- kawsper 2y ago
  It could be a paid plugin. There's several successful paid plugins for both applications.
tkzed49 2y ago
I really doubt most people are using either of those for TikToks. _Maybe_ YouTube shorts if they also produce long form content.
arommelaere 2y ago
I'm not sure how hard it is and if it's feasible but I will add it to my notes! Thanks for this suggestion! Do you know if plugins on those tools can "connect to the internet and run the process via my videofa.st servers" even if they are plugins?

qingcharles 2y ago

I signed up. I often have to caption videos for TikTok/Reels etc. I normally use CapCut mobile (desktop version has weak subtitling). It is a massive PITA to make compelling engaging captions like this. Going back and adding emojis etc is another PITA. I'm willing to give it a test. The pricing is a little steep for simply captioning, especially as a lot of short videos are very speculative and most won't make any return on investment. One idea I would love: the tool to identify different speakers and change the color of the captions. I had to do this manually the other day on an interview and it was a real time sink.

cloakedcode 2y ago
I’m in a similar boat where I produce about one video a day and will definitely give this a try. But if I I want to caption them all that means $149/mo because the $15 plan has a cap of 20 videos/mo. At that price I’d much more likely pay for DaVinci Resolve Studio one time at $295 to use their subtitle generator (and that fits into my existing workflow). If there was a better pricing plan for someone who has yet to see a dime from videos, I’d consider this long term.
- dowakin 2y ago
  You could try `https://github.com/jianfch/stable-ts https://github.com/jianfch/stable-ts`, it's an improvement for OpenAI Whisper. It does pretty good job.
- emmanueloga_ 2y ago
  whisper.cpp [1] has a karaoke example that uses ffmpeg's drawtext filter to display rudimentary karaoke-like captions. It also supports diarisation. Perhaps it could be a starting point to create a better script that does what you need. -- 1: https://github.com/ggerganov/whisper.cpp/blob/master/README.md#karaoke-style-movie-generation-experimental https://github.com/ggerganov/whisper.cpp/blob/master/README....

2024throwaway 2y ago

Short video content is cancer. I wish you the best, looks like a good solution. Don’t worry about me, I just hate it.

makeitshine 2y ago
I normally agree, but there are times when I want a short video. For example, my wife makes crochet and knitting videos. She has series which are 30-60s long and cover one technique. The opposite are people who want to waste your time with a 30 second intro, a talk about whatever and then spend 30s showing you the same technique. This is not ops target, I understand.
- unglaublich 2y ago
  Okay! BUT ((why)) this VERY Annoying!? Format :head_explode:
- Timwi 2y ago
  There's no reason people can't make 30 seconds-long videos. Your crochet/knitting use-case sounds perfect! The problem is with how YouTube treats short videos as its own thing, expects them to be in vertical video format (because rotating your phone is asking too much) and completely messes up the UI (can't even seek properly). I think channels with short videos that teach a technique or a nugget of knowledge could be very popular and very effective (educationally speaking) if it weren't for the capitalist enshittification.
  - makeitshine 2y ago
    It's funny you mention vertical videos. I'm currently working on a toy project using MobileNet which classifies YouTube thumbnails as clickbait or not, as per my preferences, and I classify every vertical video as such. It needs more training data but it's up to ～78% accuracy at the moment. Next I'm loading it in a Chrome extension.
mrtksn 2y ago
These days long videos are short videos with 10x fillers to optimize the runtime for advertisers. A video about this psychological phenomenon? Why don’t we start with defining what is psychology, a history of it and and some surprising fact before telling you this thing rahat takes 20s to tell? Also, let’s hear from our sponsors first. The really long videos are just background noise people play when doing other stuff. Really low quality garbage. IMHO short videos == bad is unjustified meme. Some TikTokers have mastered it, they will have an intro that is shorter than the actual content, you can decide if you want to watch this in just a few moments and then deliver you the money shot. The long 3 min videos that keep building up are insufferable though. Short videos are not really for the people with low attention span, it’s just that it takes a few seconds to explain anything and it takes 10,000 to master it. 22 minutes video about why the Catholic Churches look the way they they look isn’t a step towards mastering architecture or theology, it’s just 21 minutes of fillers like history and examples to deliver the idea that Catholicism is very centered around the church, therefor they design it to look like an authority. A 20 second TikTok video will be available to convey the same information without wasting your time and if you’re actually interested and master this stuff you can go read books or join a program that teaches the subject. It’s no surprise that some of the best tech videos on YouTube are by Fireship. Their videos are somewhere between 1 and 4 minutes long, including the jokes.

attentiongroper 2y ago

From an ethical point of view, have you considered your “attention grabbing tool” evil ? We really need our attention to ourselves, please stop groping it. Yours looks particularly evil.

thorncorona 2y ago
lots of people prefer subtitles to audio. i am one of them. this seems like it will do fine.
- nullandvoid 2y ago
  I think it's the unnecessary insertion of emoji's next to words, and flashing of the current word that OP is talking to. No one is is arguing subtitles are bad, it's the way this tries to increase engagement through abusing attention mechanisms, that I also agree with OP is likely damaging to us. It's part of the many reasons I dislike the current short form media craze - horrible tactics like this are used more often than not.
  - unglaublich 2y ago
    Exactly. And since the human mind has limited attention capacity, I feel it desensitizes people from paying attention to "normal life". People look like zombies.
- Timwi 2y ago
  Nobody said you couldn't have subtitles. It sounds like you want to impose subtitles on everyone just because you want them.

deleted 2y ago

[deleted]

lionkor 2y ago

I love that, as a demo, you're just stealing content. Keep it up

arommelaere 2y ago
The content is from a creative common video

aestetix 2y ago

This service is awesome. Just fyi, I sent you a message via the contact form on the website :)

arommelaere 2y ago
Thank you for your interest in videofa.st! I've replied to your message ;)

ale42 2y ago

I guess some people like this kind of videos, personally I can't stand it. Flashing subtitles that just look like an interference. I want to watch the video, not the damn subtitles. At a bare minimum, they should be optional!

arommelaere 2y ago
Thanks for your opinion, would provide slow/soft animations would solve this problem?
- ale42 2y ago
  Good question, I'd have to try. Usually I simply don't want to see subtitles, unless the audio is in a foreign language I don't speak, or if I want to quickly check a video (e.g. a tutorial) without having audio, like during a meeting.
  - jfoster 2y ago
    Subtitles are extremely useful if you want to watch videos without audio, though. I think the whole idea of short videos is that you might be in-between doing things and be looking for some quick entertainment. In such a scenario, you're likely in a public place.
    - ale42 2y ago
      Agreed, but why embedding them in the video stream? Missing a proper subtitle support where the videos are supposed to be played?
      - jfoster 2y ago
        This product seems to be targeted toward short videos, like TikTok, Instagram reels, or YouTube Shorts. I'm not sure, but I don't think any of those offer built in subtitle support yet? I know YouTube does on regular videos, but I don't think I've seen it for Shorts.

fareesh 2y ago

Which whisper model do you use?

pipe2devnull 2y ago

Ignoring any comments about wether or not short videos are good or bad, the demo seems like the subtitles are appearing and disappearing way too quickly, it’s a bit hard to read but maybe that’s just the video

arommelaere 2y ago
Thanks for your feedback, current subtitles are sync with speech speed, so yes the demo video might be too fast, maybe I should make a slower video

tamimio 2y ago

You should add the pricing in the main page, not hiding it behind the registration..