4 ms·

> You haven't been seeing what's been coming out of models from China. Citations? I haven't, and would be genuinely curious if it is better than the AI short

> You haven't been seeing what's been coming out of models from China. Citations? I haven't, and would be genuinely curious if it is better than the AI short clip slop that appears on YouTube and really just needs to be taken out back and shot. I have no doubt that its coming, I just haven't seen it yet.

ben_w 8mo ago
I think there's two things contributing to YouTube slop, the earlier generations of free model being one, the other is more fundamental: most people using these tools for slop only care about advertising money, they neither know nor care about how much of a difference artistic taste makes (hence also why there's so many YouTube shorts which are just a clip from a random TV show with inappropriate music added, sometimes as a mic-drop, sometimes just obscuring the dialogue, but how even are YT shorts monetised?) FWIW, there are models which are better than the majority of the YouTube slop I've seen*; but even then, when I show short clips from e.g. Google's "Flow" generator to others, those others find the results unimpressive. Also, the voice range is weird, my experience playing with them it had a much harder time of creating a decent working-class British accent than Dick Van Dyke. However, amongst the various issues Flow has, the most obvious is length, because even though Flow lets me "extend" a clip, it's not got enough coherency between each (8 second?) segment when I do. The second biggest issue is that, as with Stable Diffusion before it, it quite often produces stuff you just don't want. Junk dealers don't care about that and use the first result, an artist can afford to do 10 attempts and pick the right one, and will care to, too. That's something I can at least expect will be solved, or has been on non-free tools (control nets etc.) even though I don't expect slop dealers to ever bother with them. If each 8 seconds of video was actually Hollywood quality (they're not, in any sense including resolution) and they were coherent when extended (likewise, they're not), as there's 750 times 8 seconds in a 100 minute film, it would still be an existential threat to Hollywood if those segments cost $1333, because that's a million dollars and in the cost range of "micro-budget indie", rather than the tens of millions that even normal-indie films cost (before marketing etc.) or hundreds of millions that blockbusters cost. I have no idea how video generators scale up in cost with time or resolution, let alone quality, so I don't want to make guesses on how much extra compute (or indeed training data) would be needed for a coherent 2 minute cut-equivalent at 4k. As I don't like the epistemic collapse we're already witnessing with clips from video games being passed off as coverage of warfare, I very much hope this is one of those things which fails to scale up. * although even with early models, if you wield them as an artist rather than as a slop merchant you get interesting output; you may remember the Harry Potter Balenciaga videos from a few years back, same channel and you can see how things have changed: https://www.youtube.com/@demonflyingfox/videos https://www.youtube.com/@demonflyingfox/videos
- fragmede 8mo ago
  https://www.reddit.com/r/aivideo/comments/1pu8smq/lord_of_the_rings_disco_one_funk_to_rule_them_all/ https://www.reddit.com/r/aivideo/comments/1pu8smq/lord_of_th... is the tip of the iceberg.
  - ben_w 8mo ago
    Cuts/scene transitions: 5s, 7s, 8s, 11s, 13s, 14s, 17s, 20s, 25s, 27s, 29s, (and then several continuously overlapping transitions), 36s, 38s, 39s, 40s, 42s, 43s, 47s, 49s, 50s, 52s, another at 52s, 54s, two more at 55s, two at 57s, 1:00, … This is demonstrating the specific limitation I was talking about. And also, some of those scenes, if you pause on them, they've got the slightly-off vibes of GenAI content, a casual viewer is only missing that because the scenes change so quickly. Something something moonwalking gorilla. An artist can make a work of art despite the limitations of whatever tool they use (and indeed may get a kick out of doing just that), that doesn't mean the tool can do everything. I myself am working on a GenAI musical comedy sketch video, but it is only possible with current tech because there's a good opportunity to make cuts every few seconds.

ben_w 8mo ago

I think there's two things contributing to YouTube slop, the earlier generations of free model being one, the other is more fundamental: most people using these tools for slop only care about advertising money, they neither know nor care about how much of a difference artistic taste makes (hence also why there's so many YouTube shorts which are just a clip from a random TV show with inappropriate music added, sometimes as a mic-drop, sometimes just obscuring the dialogue, but how even are YT shorts monetised?) FWIW, there are models which are better than the majority of the YouTube slop I've seen*; but even then, when I show short clips from e.g. Google's "Flow" generator to others, those others find the results unimpressive. Also, the voice range is weird, my experience playing with them it had a much harder time of creating a decent working-class British accent than Dick Van Dyke. However, amongst the various issues Flow has, the most obvious is length, because even though Flow lets me "extend" a clip, it's not got enough coherency between each (8 second?) segment when I do. The second biggest issue is that, as with Stable Diffusion before it, it quite often produces stuff you just don't want. Junk dealers don't care about that and use the first result, an artist can afford to do 10 attempts and pick the right one, and will care to, too. That's something I can at least expect will be solved, or has been on non-free tools (control nets etc.) even though I don't expect slop dealers to ever bother with them. If each 8 seconds of video was actually Hollywood quality (they're not, in any sense including resolution) and they were coherent when extended (likewise, they're not), as there's 750 times 8 seconds in a 100 minute film, it would still be an existential threat to Hollywood if those segments cost $1333, because that's a million dollars and in the cost range of "micro-budget indie", rather than the tens of millions that even normal-indie films cost (before marketing etc.) or hundreds of millions that blockbusters cost. I have no idea how video generators scale up in cost with time or resolution, let alone quality, so I don't want to make guesses on how much extra compute (or indeed training data) would be needed for a coherent 2 minute cut-equivalent at 4k. As I don't like the epistemic collapse we're already witnessing with clips from video games being passed off as coverage of warfare, I very much hope this is one of those things which fails to scale up. * although even with early models, if you wield them as an artist rather than as a slop merchant you get interesting output; you may remember the Harry Potter Balenciaga videos from a few years back, same channel and you can see how things have changed: https://www.youtube.com/@demonflyingfox/videos https://www.youtube.com/@demonflyingfox/videos

fragmede 8mo ago
https://www.reddit.com/r/aivideo/comments/1pu8smq/lord_of_the_rings_disco_one_funk_to_rule_them_all/ https://www.reddit.com/r/aivideo/comments/1pu8smq/lord_of_th... is the tip of the iceberg.
- ben_w 8mo ago
  Cuts/scene transitions: 5s, 7s, 8s, 11s, 13s, 14s, 17s, 20s, 25s, 27s, 29s, (and then several continuously overlapping transitions), 36s, 38s, 39s, 40s, 42s, 43s, 47s, 49s, 50s, 52s, another at 52s, 54s, two more at 55s, two at 57s, 1:00, … This is demonstrating the specific limitation I was talking about. And also, some of those scenes, if you pause on them, they've got the slightly-off vibes of GenAI content, a casual viewer is only missing that because the scenes change so quickly. Something something moonwalking gorilla. An artist can make a work of art despite the limitations of whatever tool they use (and indeed may get a kick out of doing just that), that doesn't mean the tool can do everything. I myself am working on a GenAI musical comedy sketch video, but it is only possible with current tech because there's a good opportunity to make cuts every few seconds.