7 ms·
VLC tops 6B downloads, previews AI-generated subtitles
- eviks 2y agoWould be great to add a system of sharing the results instead of having to waste resources transcribing the same video over and over again with no chance of correction
- tessellated 2y agointeresting for comparison, tweaking, analysis,..
- deleted 2y ago[deleted]
- asddubs 2y agonow here's an AI integration that doesn't seem pointless and stupid
- WithinReason 2y agoIn fact it's a feature I've been waiting for someone to do!
- moffkalast 2y agoFinally something that can potentially run on those new shoehorned NPUs and isn't just more spying.
- sseagull 2y agoWindows has this as a feature in the OS with their stupidly-named Copilot+ PCs. I tried it a while ago and it seemed ok-ish at translating. Wasn’t terrible for doing it all on-device https://support.microsoft.com/en-us/windows/use-live-captions-to-better-understand-audio-b52da59c-14b8-4031-aeeb-f6a47e6055df https://support.microsoft.com/en-us/windows/use-live-caption...
- redserk 2y agoThere’s a reason it’s being implemented in an open source application that gets actual use instead of a random-word-generated-elevator-pitch being chased by VC funds and dinosaurs in the stock market.
- cynicalsecurity 2y agoPlease no AI.
- compootr 2y ago> using open-source AI models that run locally on users’ devices, eliminating the need for internet connectivity or cloud services this isn't the bad kind of ai we hate... it could be actually useful!
- soulofmischief 2y agoThe unmitigated tendency for people to selectively morally approve homeomorphic elements of the same set based on whether or not each element is personally useful to them is extremely alarming.
- eesmith 2y agoThere isn't enough information to judge. For example, the article doesn't mention if the training data was legally and ethically acquired.
- TheMode 2y agoSorry if this is a hot take, but this is of no importance to me.
- deneas 2y agoUnfortunately the term "open-source" AI alone is meaningless without knowing the users definition of open-source. More often than not it just means "openly available". In this case for example they could be using Whisper, which is openly available, but from OpenAI and still has all the caveats around used data, resource usage for training etc.
- Almondsetat 2y agoOk, stop using your CPU then
- lofaszvanitt 2y agoWhat about fixing subtitle regressions first. Then put the subtitles on a separate layer.
- haunter 2y agoMaybe we will get AI generated frame-stepping backwards by 2035? /s
- rvnx 2y agoEdit: mentioned frame interpolation, but seems like it was about something else; being allowed to go one frame backward
- thanatos519 2y ago> frame-stepping backwards That's not frame interpolation. Just being able to step back one frame at a time.
- thrdbndndn 2y agoThe OP means to be able to play backwards frame by frame, which I believe VLC can't. To be fair, technically speaking this isn't nearly as easy as step forward due to how codecs are constructed, but lots of players have this feature already.
- matsemann 2y agoImplement it yourself, then? Or how is this relevant to the feature this post is about?
- deleted 2y ago[deleted]
- WithinReason 2y agoWhich is a very common feature, even YouTube can do it if you press "," (or the button next to "M" I guess)
- Semaphor 2y agoA while ago I used whisper (or rather an OSS subtitle tool that used whisper, sadly can’t remember the name; it also converted burned in subs to proper ones via OCR) to generate subtitles for a season of a Show (4 season DVD set, 1 had proper subs, 2 burned in subs, 1 no subs -.-), too old and not popular enough to have "scene" subs, it worked impressively well. The most memorable thing for me was that a character’s scream was properly attributed to the correct character. I’d love a feature like that for Jellyfin eventually.
- stzsch 2y agoYou can setup a bazarr instance to use whisper as a subtitle provider.
- Semaphor 2y agoLooks like it only works if you use the *arr stack, which I don’t.
- diggan 2y agoIf you're already using Jellyfin, then why not? Don't want to complicate the stack?
- aquova 2y agoI've seen this sentiment a lot, and it's always confused me a bit. I only use Jellyfin as well, I've never really seen the appeal of the *arr stack. I really don't see the appeal of having a firehose pointed at my Jellyfin instance, which requires like eight additional services that need to run.
- diggan 2y ago> I've never really seen the appeal of the *arr stack Automation I suppose. I only run Sonarr/Radarr, so two additional services. They both work exactly the same, so it's basically "configure once, run twice". I eventually ran out of patience to find the right releases and what not, and now enjoy a life where I can just input the title, then return one hour later to everything setup for me. Any new episodes/movies will just be available automatically in Jellyfin next time I open it up on the TV. Helps that other people in the household are also able to use it, so we've got rid of all streaming services now.
- sans_souse 2y agoHmm I wonder if this had anything to do with this? https://www.youtube.com/watch?v=0dUnY1641WM&themeRefresh=1 https://www.youtube.com/watch?v=0dUnY1641WM&themeRefresh=1
- neilv 2y agoWill the ML model be fully open? (Like open source software, so that, in theory, someone could see see the source code, source data, and process for how it was trained, and reproduce that? And could they change how that model is built, not just tuning atop it, and distribute that under the same open source license?)
- Tiberium 2y agoIt's highly likely that they'll be using Whisper, if so, it won't be fully open. But I can be wrong, of course.
- bdz 2y agoCan’t wait to watch anime with AI generated subs, it will be a beautiful trainwreck.
- GaggiX 2y agoYou don't need to wait, you can use: https://github.com/m-bain/whisperX https://github.com/m-bain/whisperX right now for STT with timestamps and diarization.
- mossTechnician 2y agoAmazon has already rolled out AI-generated subtitles for Crunchyroll anime. I discovered this while watching Last Exile, a show that has already been subtitled and dubbed in English. It's funny for a second, and then incredibly frustrating after that. Character names are regularly wrong, and sentences get rewritten to have different meanings. Instead of just reading subtitles, you have to mentally evaluate them for errors. At best, AI generated subtitles should be a last resort.
- portaouflop 2y agoYou get what you pay for I guess. Translators are paid pennies and AI is making it much worse. Who in their right mind would aspire to become a professional translator today?
- mossTechnician 2y agoThat's the part that really confuses me. Somebody already did pay for translation from Japanese to English subtitles, then for a voice cast to dub the anime. The original subtitles are nowhere to be found; the closed captions are based on the English language dub.
- Dalewyn 2y agoAs a former fansub translator, it's better for mankind this way. Further thoughts on the matter: https://old.reddit.com/r/grandorder/comments/dnpzrh/everyone_flipping_out_over_osakabehimes/f5f895x/?context=3 https://old.reddit.com/r/grandorder/comments/dnpzrh/everyone...
- GaggiX 2y agoI wonder what STT model that is fast enough and accurate VLC could be use for the application.
- torben-friis 2y agoI recently used some subtitles that I later found out had been AI generated. The experience wasn't really good to be honest - the text was technically correct but the way multiline phrases were split made them somehow extremely hard to parse. The only reason I learned AI was involved is that it was bad enough for me to stop viewing and check what was wrong. Hopefully it's an implementation detail and we'll see better options in the future, as finding subtitles for foreign shows is always a pain.
- shinycode 2y agoI had exactly the same experience. Human expertise in subtitle makes the experience really better
- esperent 2y agoIf you have the luxury of requiring subtitles in English, sure. There's a huge scene of people making them and high quality subtitles available for pretty much everything. If you need subs in another language though your experience might change dramatically. Especially for any media that is old or less popular, in which case your options are probably either really bad subs, out of sync subs, or most likely, none whatsoever.
- Dr4kn 2y agoProper subtitles are obviously better, but it's impossible to do on everything. The tech is going to get better, and is already a game changer for hearing impaired people. Subtitles that are mostly correct are much better than none at all. VLC has the option to find subtitles. If you use plex or jellyfin there are add-ons or bazarr, which does it automatically
- adinisom 2y agoOn Youtube I really like the auto-generated captions and often prefer them to the creator's because: - Sometimes the creator bases their captions on the script and misses changes in edit - Sometimes the creator's captions are perfect transcriptions but broken up and timed awkwardly Auto-generated captions aren't always perfect but unlike human captions provide word-by-word timing.
- pylua 2y agoI wonder how strong your pc will have to be to run this locally.
- marcodiego 2y agoWhisper.cpp ran fine on my GPU-less 2013 era 8GB i5 Dell laptop.
- bayindirh 2y agoGiven modern mirrorless cameras can detect and track humans or mammals using DSPs and NNs without any discernible battery penalty at 30/60/120/240FPS (depending on camera class), I think they can do it with very small and efficient models with some frequency clamping and downsampling. Initial versions will be more power hungry, but it'll be negligible at the end, given that modern processors have accelerators for image and voice based AI applications.
- EmpireoftheSun 2y agoI wonder if there could be an option to share the generated ai subtitiles so there is no duplicate effort. For example Movie_torrent.vlc subs can be shared on some other users machine with the same Movie_torrent.vlc file. Obv you can find majority of subs online for popular files but niche cases usually have low subs available and sharing AI could help the ecosystem.
- marcodiego 2y agoThis is how a media player should work and be developed. Learn that, RealPlayer.
- n144q 2y agome: waiting years for VLC to fix basic usability issues and reconcile UI across different platforms VLC: we're gonna work on AI
- petee 2y agoDon't they still consider "app locks up when playlist free-loops" not a bug?
- jillesvangurp 2y agoThis boils down to software development not being free. In VLC's case, the development is funded by several for profit companies (like videolabs) that make their money from stuff they do with VLC (consulting, commercial services, etc.). VLC is a good example of an OSS project that is pretty well run with decades of history that has a healthy ecosystem of people and companies earning their living supporting all that and a foundation to orchestrate the development. I don't think there ever was a lot of VC money they need to worry about. This is all organic growth and OSS working as it should. So, this boils down to what paying customers of these companies are paying for. The project also accepts donations but those go to the foundation and not the companies. It's the companies that employ most of the developers. And you can't fault them on working on things that they value. If AI features is what they pay for then that is what they work on. I happen to share your reservations about the UX. It's a bit old school, to put it mildy. And they obviously don't have professional designers that they work with. Like many OSS products, it looks and feels like product made by techies for techies. It doesn't bother me that much but I do notice these things. I actually talked to one of their IOS developers a few years ago. Pretty interesting person and not a huge team as I recall. I remember talking to her about some of the frustrations she had with the UX and the lack of appreciation of that. I think she moved to Netflix afterwards. Like with most OSS projects you are welcome to take part in the meritocracy and push your favorite features or pay somebody to do that for you. But otherwise, you should just be grateful for this awesome thing existing and prospering.
- ttoinou 2y agoThey don’t need designers when they are the free media player that stood the test of time and being used by the masses. It’s true organic bottom up design, tweaked little by little over the years
- Yaina 2y agoIt's really sad that I'm reading "open source model" and think "hmhm, as if". Maybe they're really using a truly open source model (probably not) but the meaning of the word is muddied already.
- rvnx 2y agohttps://code.videolan.org/videolan/vlc/-/merge_requests/5155 https://code.videolan.org/videolan/vlc/-/merge_requests/5155 Here they are working on integrating Whisper.cpp In the search bar it says "Updated 2 weeks ago", like if there were additional recent comments or actions in this thread that we cannot see. So it could actually be OpenAI Whisper model, for which we have the final binary format (the weights), but not the source training data, but it is the best you can get for free.
- _heimdall 2y agoThe meaning of "AI" and "open source model" have both been muddied enough to be pretty meaningless.
- tmtvl 2y agoYeah, it'd be nice if we could all use 'open source' to mean 'open weights' + 'open training set', instead of just 'open weights'. I fear that ship has sailed though. Maybe call it a 'libre' model or something?
- peterpans01 2y agoProblem with this one is the quality is horrible, especially, if the language is not English. Sometime, completely wrong.
- TheAceOfHearts 2y agoAI subtitle generation seems like a useful feature. Hopefully they'll integrate with a subtitle sharing service so we don't have computers repeatedly duplicating work. In the ideal case you'd probably generate a first pass of subtitles with AI, have a human review and tweak it as needed, and then share that. There's no reason for people to be repeatedly generating their own subtitles in most cases.
- newjersey 2y agoHow do you do this in a privacy preserving way?
- dillydogg 2y agoThe only feasible way I can think of is a locally run model. Perhaps whisper?
- mavamaarten 2y agoRun the model on your PC. Done?
- _heimdall 2y agoA system like that would be pretty nice as long as it wasn't a privacy problem. You wouldn't really need LLMs to do the subtitles as all then though, for any video common enough to be sharable the subtitles probably already exist from the original source.
- Dr4kn 2y ago
- daemin 2y agoIsn't this what YouTube has been doing with automatically generated subtitles for videos for years? I've always found their quality to be somewhat lacking.
- wodenokoto 2y agoYes and no. Likely due to cost, Google hasn’t decided to use any LLM model (what basically everybody refers to when they say AI now) to generate the subtitles and by modern standards they aren’t very good.
- nicoburns 2y agoQuality definitely can be lacking, on the other hand, it is significantly better than no subtitles at all.
- harvie 2y agoI feel like the quality got bit better recently...
- c16 2y agoVery excited for this, but a waste of energy if everyone is needing to process their video in real time.
- _flux 2y agoWhat is the alternative? The results could be cached, but it's probably unlikely that they would need to be used again later, as I imagine most videos are watched only once. Another option would be to upload the generated subtitles to some service or p2p, but I believe that would also problem (e.g. privacy, who runs the service, firewalls for p2p, etc).
- CaptainFever 2y agoOpenSubtitles as a cache, maybe? (With their collaboration, of course.)
- _flux 2y agoIt seems like some kind of review process would need to be included to reduce abuse possibilities. But yes, that would be quite nice, if there are enough people who don't mind uploading the file names of the media they play (or some other unique media identifier?) along with the subtitles to that service—and with user credentials? It would certainly need to be opt-in, which makes one wonder if it would be much effective at all.
- a3w 2y agoVLC has a "check my media online" feature, next to "check for updates to VLC on startup" already. Could they offer subtitle downloads? Previously, that was used for mp3 album covers or something?
- rvnx 2y agoIt can actually send to Google the information on what you are playing: https://github.com/videolan/vlc/blob/f908ef4981c93a8b76805ade15246981002affc9/share/lua/meta/art/01_googleimage.lua https://github.com/videolan/vlc/blob/f908ef4981c93a8b76805ad... and to their own servers: https://github.com/videolan/vlc/blob/f908ef4981c93a8b76805ade15246981002affc9/share/lua/meta/art/00_musicbrainz.lua https://github.com/videolan/vlc/blob/f908ef4981c93a8b76805ad... should could fetch subtitles as the same time ? edit: cf, what "a3w" says too.
- jl6 2y agoThere's an art to subtitling that goes beyond mere speech-to-text processing. Sometimes it's better to paraphrase dialog to reduce the amount of text that needs to be read. Sometimes you need to name a voice as unknown, to avoid spoilers. Sometimes the positioning on the screen matters. I hope the model can be made to understand all this.
- entropie 2y ago> Sometimes it's better to paraphrase dialog to reduce the amount of text that needs to be read. Questionable. It drives me crazy to have subtitles that are paraphrase in a way that changes the meaning of statements.
- lm28469 2y agoAs long as they're synced properly I don't care much, some movies/shows have really bad sound mix and it's not always possible to find good subs in the first place
- eviks 2y ago> better to paraphrase dialog to reduce the amount of text that needs to be read. That's just bad destructive art, especially for a foreign language that you partially know. > Sometimes you need to name a voice as unknown, to avoid spoilers. Don't name any, that's what your own eye-ear voice recognition/matching and positioning are for (also reduces the amount of text) > Sometimes the positioning on the screen matters. This is rather valuable art indeed! Though unlikely fit to be modelled well
- netsharc 2y agoI wonder if VLC tracks their downloads per day or region. Similar to Pornhub Trends, it'd be interesting to correlate a spike of downloads with events like Netflix hiking up their prices/enforcing anti-password-sharing policies, dropping some series from their catalogue, or some hyped-up movie being released...
- benjijay 2y agoTotally personally skewed perspective, but thinking about the number of systems I've installed VLC on, some multiple times through rebuilds etc, 6 billion feels like a huge underestimate!
- diggan 2y agoI'm guessing that's just from the main website. It's distributed so wildly I'd wager we'll never know the complete number. But yeah, I've also must have installed it at least 50 times or so, since the first time I came across it.
- cess11 2y agoAs of yet big auto-subtitle providers like YouTube fails miserably at this, and as far as I've tested the Whispers they also do. Would be nice if embedding it wouldn't become the default.
- a3w 2y agoFinally, we can daisy-chain translations to freshly generated anime subtitles and the world will find out what "dattebayo" stand for.
- opunsoars 2y agoYet another classic garbage-in-garbage-out expected from AI gen subtitles. But it is an excellent undertaking nevertheless. Will help improve viewership in general
- bluesounddirect 2y agono , time find a ai less music player .
- runnr_az 2y agoOut of curiosity, what’s VlC doing at CES? Do they have a commercial offering?
- hardwaresofton 2y agoVLC is excellent on iOS -- highly recommended! Go-to player with easy wifi loading and ability to connect to file shares to find files. Simple and actually easy to use (of course having a file server is another question)
- on_the_train 2y agoJesus Christ, is any software safe from ai? How long until curl gets ai? How about ls? Git?