50 ms·
Make-A-Video: AI system that generates videos from text
- greenhearth 4y agoThis tech will definitely revolutionize stock footage. Generating a post-modern cyber-ghost glitchcore horse drinking water is definitely way more exciting than just using some boring old video of some stupid real horse drinking water
- sva_ 4y agoI think we're pretty close to the moving and talking portraits of deceased people from the Harry Potter universe.
- strangattractor 4y agoAll the animals have human hands - kind of creepy looking. I assume that is because the images used for training the models only humans have hands.
- hattmall 4y agoSo long Hollywood. You had a good run, now all videos will be auto generated from scripts in the Metaverse.
- Datenstrom 4y agoThe > A golden retriever eating ice cream on a beautiful tropical beach at sunset, high resolution example is terrifying.
- mensetmanusman 4y agoThey forgot ‘trending on onlyfans’ But actually, this technology is super exciting. Imagine a future where movies and games are choose your own adventure.
- Bakary 4y agoWe'll have procedural generation that will be hard to distinguish from human-made content. Goodbye repetitive Skyrim filler caves!
- anigbrowl 4y agoI've never played Skyrim - was that the immersive 3d version of a maze of twisty little passages? Besides just textural content, it's intriguing to consider the possibilities of full-3d roguelikes.
- permo-w 4y agoif people weren’t so repressed, this could also be used to severely reduce exploitation in the porn industry. what’s the point in making and selling exploitative porn when it can be auto-generated at will?
- tpmx 4y agoIn case anyone else had problems finding it - it's at the bottom of the page: https://makeavideo.studio/assets/a_golden_retriever_eating_ice_cream_on_a_beautiful_tropical_beach_at_sunset,_high_resolution__22.webp https://makeavideo.studio/assets/a_golden_retriever_eating_i... (webp) That grasp though. These things still feel a bit like e.g. Google/GCP services to me: Super appealing at first glance, quite close to what you want, but somehow never quite there. Maybe they'll asymptotically get there, eventually? Perhaps that statistical model can't really make it to the level we want it to?
- mach1ne 4y agoIt may be that it's the deep learning tech which will never quite get there. GPT-3 has similar shortcomings in its mimicry. We're 95% there, I guess, but may never quite reach 100%.
- rany_ 4y ago> the deep learning tech which will never quite get there Never say never, we've come a long way since GPT-2! All this was unthinkable back then
- anigbrowl 4y agoNah, the current issues are just because we're trying to do everything in one step. Because we've built tools that have so much of a stimulus-response approach, few efforts have been made toward interfaces that ask for clarification ('when you say X, do you mean XYZ or XXX?'). Image-to-image and tuning already addresses many of these issues; just as inpainting works really well, it won't be long before we have select-and-repair, where you add an additional prompt like 'improve this part - the ice cream is fine, just work on the dog's muzzle.'
- kraquepype 4y agoThe hands throw me off. The same with the cat holding the remote... never thought that hands on animals would be able to trigger my uncanny valley response, but here we are
- bckr 4y agoEugh, thanks for warning me.
- Jackson__ 4y agoIt is, but to make up for it we got a man drifting a horse.[1] [1] https://makeavideo.studio/assets/A_knight_riding_on_a_horse_through_the_countryside_second_upsample__12.webp https://makeavideo.studio/assets/A_knight_riding_on_a_horse_...
- BizarroLand 4y agoYeah, put some blue sparks on those hooves and you've got a meme
- thedorkknight 4y agoJust needs to show the part where he ends the drift and gets a speed boost
- tmjdev 4y agoThis looks like the video equivalent of Dall-E 1. Hard to believe how far we've come so quickly. The paper talks about "pseudo 3D attention layers" that are used in place of temporal attention layers for each dimension due to memory consumption. It seems like AI research is vastly outpacing GPU development.
- elil17 4y agoLooks a bit better than DALLE1 IMHO. They've demonstrated greater range.
- londons_explore 4y agoIndeed - it's not hard from a research point of view - it's hard from a compute perspective because adding one more dimension requires hundreds of times more compute. Even then, these videos are only like 50 frames long - and a real movie you would want to be hundreds of thousands of frames long.
- Filligree 4y agoSo you need to optimise on compressed version, not the whole thing. What they’re doing right now is akin to a human trying to hold an entire picture - or entire movie - in their head all at once. We can’t do it. AIs can sort of do it. Latent diffusion models already demonstrated that operating on a compressed representation gives far better results, faster, but I don’t think we’re anywhere near the limit for what’s possible there. It’s no coincidence that this is how humans work.
- tmjdev 4y agoI am curious if there has been any research on temporal attention in humans. I'm not sure how you'd quantify it. But in myself I know that I'm constantly predicting where something will be or what it will look like based on how it did a second ago. It's probably the root of reflexes.
- 4y ago
- shisisms 4y agoDelighted this is in Facebook’s hands. No doubt responsible and careful thought of implications have been baked in from the start.
- dang 4y agoPlease don't post snarky comments, or shallow dismissals, to HN. You may not owe BigCo better but you owe this community better if you're participating in it: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- fasteddie31003 4y agoI'm rooting for this tech. Hopefully this will get modern movies out of their low risk reboot loop since it will be cheaper to make a movie that have new story lines that are commercially untested. I'd be happy to watch a movie that doesn't look AAA, but has compelling writing and makes me think. Or maybe I'll just stick to books.
- RupertEisenhart 4y agoHave you tried MUBI? It takes a lot of the hassle out of finding quality arthouse films, there is a lot of good stuff on there. Though I must admit that if I didn't have friends holding my hand through the minefield of modern cinema, I would also just stick to books.
- Xelynega 4y agoI don't think modern movies are stuck in a "low risk reboot loop" because of the cost to produce, it's because of the potential profit. Why spend money on a film with new IP and ideas that you're not sure will be popular when the data science team has already worked with marketing to figure out exactly what movie will sell well? Good luck finding your movie with compelling and thought provoking writing in the big pile of movies produced by comittee to show up above yours in discovery algorithms.
- asutekku 4y agoWith the future version of this you could theoretically prototype movies way faster and try ideas with test audiences without requiring to actually film them
- generalizations 4y agoIn the future version of this, the end user asks for a movie and gets a custom movie generated just for them. You want a thought-provoking Bourne-style action thriller with hints of Jane Austen and a bollywood dance sequence? How about a Matrix sequel that lives up to the first one, but ends just the way you like it? Just ask.
- dr_dshiv 4y agoI couldn’t find how long the video generation took
- BizarroLand 4y agoI would guess that it didn't take long running on Meta's servers, but if you were doing this at home with a 3090 it would take at least 6 minutes per 15 second video, assuming your settings take 15 seconds to render a single frame and there are 24 frames per second in the video.
- VoodooJuJu 4y agoA lot of people saying it's over for traditional movie-making - lmao. I look at these and see nothing but uncanny valley artifacts, and I don't think it will improve much from here. It's like self-driving cars. They use almost very effective statistical models, certainly better than our previous models, but they never seem to shake off that "almost" and become truly effective.
- appletrotter 4y ago> I don't think it will improve much from here. Any more specific reason why you feel this way? Curious
- criddell 4y ago> I don't think it will improve much from here That's a bold prediction. Why do you think that? The first thing I thought was the exact opposite. This isn't very good, but it's only version 1. Motion pictures are less than 150 years old. In another 150 years I bet virtual filmmaking will progress a lot.
- dgritsko 4y agoI agree, it feels like this technology is progressing by leaps and bounds almost by the day.
- paulgb 4y agoCase in point, just compare the results of Stable Diffusion / Dall-E / midjourney to papers from 5-10 years ago (here's a random one I found http://proceedings.mlr.press/v48/reed16.pdf http://proceedings.mlr.press/v48/reed16.pdf). Remarkable progress in that time, and as I understand it, a good amount (but certainly not all) of the improvement comes “free” from being able to scale up to more weights.
- javajosh 4y agoI don't agree with the prediction but I will say that at least in America, we've been overly optimistic about almost everything since world war II. Is particularly painful when you look at the imagination of science fiction versus the reality of science in the realm of say space travel, especially interstellar travel. This imagination effect gets the public behind your new invention in part because it seems to be just the beginning the start of something new. The tip of the iceberg. But sometimes it's really just a tiny bit of ice. For the record, I'm actually rather bullish on self-driving cars. There's nothing physically impossible about solving the problem, but I'm not surprised it's harder than it sounds. But I don't see humans being fundamentally prevented from solving the problem in the same way that humans are fundamentally prevented from ever engaging in everyday space travel.
- bottlepalm 4y agoI don’t want to think reality is a simulation, but wouldn’t our everyday experience being generated by a neural network be far far simpler to achieve than a universe of infinite minuscule atoms all interacting.. like ozcam’s razor is pointing towards our lives being a realistic dream. Like in 10 years you could plug this tech into high end VR and get a prompted reality dynamically generated that would be indistinguishable from our own.
- rmah 4y agoThat what we're seeing is really reality is the simpler explanation. Because the alternative is that what we see is a simulation ... inside another reality that has full complexity. Which increases the overall complexity. Thus, Occam's Razor says what we see is likely real and not a simulation.
- bottlepalm 4y agoThe ‘outside’ reality only needs to be the size of a data center, not the entire universe. And maybe the outside reality only needs to be a trained 100gb model. A much simpler outside reality than we have now. You don’t even need to sim 8 billion people, just 1, you.
- TOMDM 4y agoOccans razor in this case applies to the complexity giving rise to a situation here though, not the complexity of a described system. Conventional physics giving birth to our universe is currently the model with the fewest assumptions. What would have to take place to give rise to a universe the size of a data center, running an AI model of a human? It feels like we have to bake in assumptions of stable physics, a rise of a stable system for that data center, and some path towards creating it and modelling a human. That said, if we believe we're capable of running billions of believable simulations, then we're more likely to be in such a simulation than ground reality. But a datacenter pocket dimension bakes in a lot of assumptions that make it less likely than our own universe.
- xena 4y agoStable Diffusion but for video is going to eat these guys lunch.
- langitbiru 4y agoThe creator of Stable Diffusion gave the indication of text-to-video: https://twitter.com/EMostaque/status/1539881228773429249 https://twitter.com/EMostaque/status/1539881228773429249
- j_m_b 4y agoWhat I want to see is a model that can generate 3D models for use in applications such as Blender. It would provide a good starting point for someone with talent to make beautiful. Or just save people like me time for making games.
- deleted 4y ago[deleted]
- sphars 4y agoI've been looking into some of the 3D model generators this past week, and there is some work happening in that field. See the following non-exhaustive list: https://github.com/snap-research/NeROIC https://github.com/snap-research/NeROIC https://github.com/threedle/text2mesh https://github.com/threedle/text2mesh https://github.com/AutodeskAILab/Clip-Forge https://github.com/AutodeskAILab/Clip-Forge https://nv-tlabs.github.io/GET3D/ https://nv-tlabs.github.io/GET3D/
- castlecrasher2 4y agoHave you by chance tried out NeROIC? I'm a 3D printing enthusiast, mostly video game stuff, and it seems like like it would be excellent for that purpose.
- sphars 4y agoI actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24 hours so far, using a 3070Ti with 8GB VRAM). From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10 https://github.com/snap-research/NeROIC/issues/10 I too was looking into it to generate 3D models for some software I've been working on.
- hahamrfunnyguy 4y agoThe human hands on the animals are in the uncanny valley for sure. Pretty creepy stuff!
- petesergeant 4y agoOh great, we can finally start fuzzing the human brain
- soheil 4y ago> A knight riding on a horse through the countryside I didn't know you could be drifting with a horse.
- psychomugs 4y agoI'm very interested in what will come out of this new (sub)medium. By virtue of video being a collaborative medium, I never feel like I'm getting a message from a singular consciousness like I do from less resource-intensive mediums like books (I know that book editors exist, but the medium has less filters to pass through compared to large products like movies). I could see this substantively lowering the barrier of entry for video and enabling a lot of new stories to be told.
- sphars 4y agoYou can view all their example videos at https://make-a-video.github.io https://make-a-video.github.io (warning: all the ~95 videos are webp and are loaded at once on the page so it may take some time to load)
- erwinh 4y agoHardmaru recently returned to twitter and everything shared is immediately fire
- birracerveza 4y agoThis is getting more and more impressive by the day.
- scifibestfi 4y ago"Our research takes the following steps to reduce the creation of harmful, biased, or misleading content." "Our goal is to eventually make this technology available to the public, but for now we will continue to analyze, test, and trial Make-A-Video to ensure that each step of release is safe and intentional." Are they really going to do a replay of OpenAI and Stable Diffusion? Deja vu coming soon.
- xhkkffbf 4y agoWhat does "intentional" mean here? Does it mean that the person intends to type the text that they do? My cat sometimes walks across the keyboard, but for the most part I pretty much intend to type everything I type. Am I missing something?
- hoppyhoppy2 4y agoIt's referring to the release of the software. I read it as "released with minimal embarassing unintended features". Like, as a rough made-up example, a "boy saluting" prompt producing a video of a Hitler youth giving a Nazi salute wouldn't make for great PR.
- 5350-uiop-1130 4y agoWonder if all these things will bring about a kind of cambrian explosion of creativity. Imagine a future of Prompt Wizards, who are able to coax the AI to generate things in a very specific way. Although we would probably need a much greater level of human curation. The way algorithms curate on youtube and spotify just doesn't really hit the spot. Perhaps stability and Dall-e already kind of showed that the value is not so much in the physical act of creating, but more-so in the ability to express something that the AI can represent and which can connect with you.
- dahfizz 4y agoI think we will definitely see a content explosion. The pessimist in me says that general levels of creativity and quality will go down, if anything. It's AI written blogspam, but for images and video too. The signal to noise ratio is getting worse and worse
- Bakary 4y agoWe already have evidence that the explosion in availability decreases the overall quality, even if the quantity of high-quality productions also increases greatly. Every creative platform is going to be flooded with the equivalent of a Reddit comment. (yes I am aware of the irony)
- algoatecorn 4y agohttps://twitter.com/chrisalbon/status/1567688342124503040 https://twitter.com/chrisalbon/status/1567688342124503040
- jklein11 4y agoHow do I get access to this?
- hoppyhoppy2 4y ago>Our goal is to eventually make this technology available to the public, but for now we will continue to analyze, test, and trial Make-A-Video to ensure that each step of release is safe and intentional.
- aero-glide2 4y agoCan't wait for the Stability-equivalent.
- deworms 4y agoI remember the threads about StableDiffusion less than 4 weeks ago where people were confidently saying this kind of thing is 6 to 12 months away. Turns out it was a month. Singularity soon?
- pesenti 4y agoResearch paper: https://makeavideo.studio/Make-A-Video.pdf https://makeavideo.studio/Make-A-Video.pdf Examples: https://make-a-video.github.io/ https://make-a-video.github.io/ Demo site: https://makeavideo.studio/ https://makeavideo.studio/ I am told live demo and open model are on the way.
- propercoil 4y agoThey almost always say that but never deliver. Won't be out even in a year's time.
- apexalpha 4y agoWhile I am absolutely amazed at the speed of the technology from dall-E, stable diffusion and now this. This will absolutely snowball easy videos all over social media such as TikTok, Insta and YouTube.
- deleted 4y ago[deleted]
- olivermarks 4y agoA goofy oligarch promoting open source AI -> Meta is falling fast imo
- folex 4y agoI wish to have such a tool to generate tutorials for different technologies. Like distributed systems, consensuses, replication, this kind if stuff. Something like: there are 10 peers, A sends to B, B waits for C, yadayada.
- Jyaif 4y agoThe progress in AI we are seeing is unprecedented. I can't think of any other time in history where I would have had my mind blown on such a regular basis.
- indigodaddy 4y agoSo how long before we can “Hey music-stable-diffusion-service,” play me “relaxing baroque violin and flute music in the style of game of thrones” ?
- echelon 4y agoMy start-up is working on this! We'll be launching a web and downloadable version soon. Eg. https://www.youtube.com/watch?v=r_0JjYUe5jo https://www.youtube.com/watch?v=r_0JjYUe5jo --> https://vocaroo.com/1hgjjnVNqWjk https://vocaroo.com/1hgjjnVNqWjk We're also working on film generation.
- mjr00 4y agoNot to insult your startup or your work (as this is pretty cool), but this is a good illustration of why that last 10% is so important. While the AI-generated song is 90% similar at a surface level, there's also no way anyone would listen to that track on Spotify if it came up on an Eminem playlist.
- o_____________o 4y agoI don't understand what I'm seeing here, in ingested the Eminem track and recreated it using a different voice?
- anonymouse008 4y agoI think we all can view a video of 'nails on a chalkboard' before we hear audio of nails on a chalkboard. For some reason, unacceptable uncanny sounds is a much wider valley than unacceptable videos/pictures. The hand holding is uncanny in the family video, but I'm fine watching it for a second - it doesn't cause pain the way that same error would in music.
- colordrops 4y agoI recall AI generated Beethoven years ago that sounded great. What happened to that research?
- 4y ago
- harrisonjackson 4y agoMore examples here: https://make-a-video.github.io/ https://make-a-video.github.io/ Some of these look like existing videos that have just be garbled - check the clownfish one and the litter of puppies - maybe that is because the prompts aren't that detailed or they just need to up the "creative/randomness" factor. The sloth one though has a more specific prompt and came out looking more better? and more original. With the huge amount of data being used to back these, how do we know the "uniqueness" of the generated content? is it original or is it just mangling of existing content? Even if it is just garbling existing content, it is pretty amazing. Frame by frame the mutant unicorns float across the beach hah.
- slhomme 4y agoAs an owner of a Video Production studio, this kind of tech is blowing my mind and makes me equally excited and scared. I can see how we could incorporate such tools in our workflows, and at the same time I'm worried it'll be used to spam the internet with thousands and thousands of souless generated videos, making it even harder to look through the noise. A fun related experiment, I thought it was fun to see what kind of movies AI would generate, so I created a "This Movie Does Not Exist" website[1] that auto generates fake movies (movie posters + synopsis). It basically uses GPT-3 to generate some story plots, and then uses that as a prompt (with in-between steps) for Stable Diffusion. Results may vary, but it definitely surprises sometimes with movies that look and sound amazing! [1] This Movie Does Not Exist: https://thismoviedoesnotexist.org/ https://thismoviedoesnotexist.org/
- skocznymroczny 4y agoReminds me of South Park when Cartman was pretending to be a robot and was made to invent movie prompts "Adam Sandler is like, in love with some girl, but then it turns out that the girl is actually a Golden Retriever. Or something.""
- spaceman_2020 4y ago“AWESOME-O”, S8 E5, if anyone wants to check it out. A South Park classic imo https://en.wikipedia.org/wiki/Awesom-O https://en.wikipedia.org/wiki/Awesom-O
- Dracophoenix 4y agoOr the "Two Brothers" skit from Rick and Morty. https://www.youtube.com/watch?v=ba9k5SWwE38 https://www.youtube.com/watch?v=ba9k5SWwE38
- raghava 4y agoOnce a capable AI that can make a "Two Brothers" trailer like this one below comes up, humanity has some serious issues to tackle, other than climate change ofc! :D https://www.youtube.com/watch?v=tBUUoyZoja4 https://www.youtube.com/watch?v=tBUUoyZoja4
- caxco93 4y agoSo it begins... I am genuinely excited and terrified for the technology we will have in 10 or even 5 years
- theschwa 4y agoSince this uses an T2I model under the hood, would it be possible to build a version of this on top of Stable Diffusion?
- andrewnc 4y agoYes! It definitely should be
- lairv 4y agoReading the paper, what's interesting is that they really build upon text-to-image models. Instead of generating 1 image with diffusion model, they generate 16, and then use several upsampling/interpolation methods to increase the resolution/framerate. For the text-image model, they use the DALL-E 2 architecture, so in theory just switching to a latent diffusion model (like stable diffusion) would boost training/inference cost of the model The shortcoming is that, since they don't use any video/text annotation (just image/text pairs), complex temporal prompts would probably not work, something which require multiple steps like "a bear scoring a goal, then doing the Ronaldo celebration"
- kitx 4y agoOne can imagine that resolving this will require a separate model to construct a timeline/keyframe series from a text description then interpolate between them. Fascinating times, either way this is allegedly being open–sourced, so I'm hopeful others will be able to build on top.
- lairv 4y agoI wouldn't have thought we'd have these text-to-video models before the end of this year, so we might actually not be that far from minutes-long video generation
- karmasimida 4y agoThis is incredible but I can't help feeling those videos are cherrypicked from a cherrypicked subset...
- forgingahead 4y agoAmazing. And lucidrains is on the case as well: https://github.com/lucidrains/make-a-video-pytorch https://github.com/lucidrains/make-a-video-pytorch
- testmzakrzewska 4y agoTedtq
- karmasimida 4y agoAlso I could see this would be a major breakthrough for Anime industry. Imagine, gives a initial frame and a reference video on the side, that would be pretty dope.
- luis02lopez 4y agoMeta Inc? no, thanks.
- jcims 4y agoMy daughter just had her first child, a baby boy that was born in the first week of July. The rate of progress in AI/ML technology is nearly unbelievable and my mind boggles at how it's going to influence his childhood (both good and bad). My daughter was born in 1998. The Internet made a bit of a phase change during her early childhood and there were many instances where there was no precedent to guide a decision on what to do, so we just had to wing it as parents based on our own intuition and values. I certainly got some things wrong but we made it in one piece. I just worry a bit about what kind of new challenges my daughter will face in raising her son in this new environment, where powerful organizations wield unbelievable resources to create deep, life long co-dependencies on their revenue streams and intellectual empires.
- greenhearth 4y agoNo worries, mate, just another shiny toy. As far as us getting milked for every bit of money, this has been going on for some time now, nothing really new, right? Douglas Adams put it best - "I've been to the future. It's the same as anywhere else. Just the same old stuff in faster cars and smellier air."
- Xeoncross 4y agoWhat is interesting to me is the decline in both understanding and communication abilities. When I look at modern speech and compare it to books written hundreds of years ago the deficiencies are stark. I can't imagine how poorly people will speak and mentally process emotions in 20 years.
- EMIRELADERO 4y agoThat is a non-sequitor. If you took an average working-class, "blue collar" person from hundreds of years ago, they would speak very differently from how those books are written.
- duderific 4y agoOk, maybe a better comparison that is sort of apples to apples would be to take a speech by George W. Bush or Donald Trump compared to say, George Washington or Abraham Lincoln. Surely you can see there is a sharp drop off in vocabulary and ability (or desire) to convey complex ideas.
- danso 4y agoIt's funny that in terms of popular culture, we've all but accepted that Star Trek's Holodeck-type fantasy creation is something that will/should exist in a "proper" future. Yet all the intermediary technology to get there -- AI-generated visuals would be among the first such steps -- still makes most of us feel a little uncomfortable at first glance.
- ThisIsMyAltFace 4y agoBecause it's Star Trek technology without Star Trek society, and in fact anti-Star Trek society.
- user982 4y agoWell, anti-Federation society. Ferengi-owned holosuites.
- drexlspivey 4y agoThe tech is very cool but this website fires all 10 cores at 100% in safari
- kleer001 4y agoAfter a while they all get kinda samey-samey. It's that whole 10,000 bowls of oatmeal problem that lives forever in this generative space.
- dzink 4y agoWhenever there is an explosion of content, curation and search become important. Meta has many of the products people use to show off their taste, so having more content to curate and share is good for their ecosystem (until people stop consuming as much because they know they can produce even better with their own imagination). This may be good for Google - the more content there is, the more you need search to find it. The downsides: there will be less money out there for creators, because that becomes a commodity. You will be able to make money if you are known for quality content polishing, editing and generally bringing that last 5-10% of generated content to look perfect (all the way until automated tools are trained to do that as well). AI will automate and improve most white-collar jobs. Instead of generators, everyone will become curators, as taste will be more important (until that's trained into the system as well). For deeper levels of consequence we have to look at history: how did the world change when people finally got paper after most writing was done on animal skins (more got to write, the richest or most powerful didn't have the only say), or water piped to your house after you had to carry buckets and dig wells (It freed up time for everyone for more interesting tasks). Now GPUs are going to be the new paper and the new PVC. Yes, software has been eating the world for a while, but you won't be able to brainstorm without AI generating the first pass.
- polyomino 4y agoIt's not just about the quality of content. Facebook's value prop is about creating a social experience around the content.
- DesaiAshu 4y agoCreators are already curators. Musicians often don’t produce their own sounds, they curate and piece together samples. Designers and software engineers cut and paste from existing work. The idea of a blank slate creator has been dead long before ML tools were introduced :)
- poopnugget 4y ago
- keepquestioning 4y agoOh my god.. this is the end.
- testmzakrzewska 4y agoTest week
- testmzakrzewska 4y agoFL ffhfhv ggbh
- dont__panic 4y agoWith DALL-E for pictures and this for video, I'm starting to wonder if Neal Stephenson was onto something when he wrote Fall, or Dodge in Hell. I know it's not the most loved Stephenson book, but bear with me (spoilers warning). The book features a couple of major plot points that feel increasingly prophetic: - A global hoax, carefully coordinated, convinces a good chunk of the world that Moab has been destroyed by an atomic bomb. This is managed via a flurry of confusion, thoughtfully deployed pre-recorded video footage, social media posts, and paid actors who don't realize the scope of the hoax at the time. Naturally, a massive chunk of the population refuses to believe that Moab still exists even after the hoax is exposed. - A group of hackers deploy and open source a massive suite of spambots that inundate social media and news sources with nonstop AI-generated misinformation to drown out real-world conversations about a topic. This is used specifically to drown out real conversation about a single individual as a proof of concept, but soon after has repercussions for the entire net... - Thanks to exactly this kind of spambot, the "raw" unfiltered internet becomes totally unusable. Those with means end up paying for filtering services to keep unwanted misinformation out of their perspective of the internet. Those without means... either don't use the internet, or work in factories where human eyes filter out content that can't be filtered by AI. I worry that exactly these kinds of developments are speeding us faster and faster down the road to a dystopian world where the "raw" internet is totally unusable. Right now, stuff like captcha and simple filters can keep out a lot of low-effort bot content on sites like Hacker News and niche forums (I think of home barista, bike forums, atlas, etc). Sites like Reddit are losing the war against bots and corporate propaganda; comment sections across the rest of the internet lost that war long ago, and just didn't realize it. But those filters and moderators can only keep up with the onslaught of content for so long. What happens when GPT is used to spew millions of comments and posts at a forum from millions of ephemeral cloud IPs? And when DALL-E creates memes and photographic content? And when Make-a-Video enables those same spambots to inundate YouTube and Vimeo? It's clear that captchas are not long for this world either. Will we see websites force more and more users to authenticate as a "real human" using their passport and government-issued ID? Maybe a Turing or Voight-Kampff test? And what does it mean when there's no longer a way to participate on the internet anonymously? As far as I can tell, limiting a site to only real human users doesn't guarantee quality -- all you have to do is look at Facebook to understand that. And somehow, despite being an incredibly easy target for bots and spam, niches of 4chan retain (racist, insane, and confusing) traces of genuine thought and conversation. The internet has been such a valuable tool in my life, and I still love browsing blogs, forums, etc. to learn about unique people doing unique things. What happens when human-generated content is hard to come by and near impossible to distinguish from promotional AI garbage? I fear for my ability to discover new content in that world.
- dbieber 4y agoVery exciting to see the field progressing so quickly. I wonder how quickly it's going to move forward from here. Will we be generating coherent audio to accompany these videos soon? Will we have multi-scene videos in the next year? Ones with coherent plot? Can we get there just by scaling up, or are other advances needed? Excited to see what comes next!
- jonplackett 4y agoIs this a Facebook thing? When you sign up for updates it asks if you will receive marketing from Meta.
- TaylorAlexander 4y agoFacebook changed their name to Meta
- jeffbee 4y agoAll the comments seem to be discussing the implications of this as if it worked, but almost all of these look like complete garbage. All the humans look deformed and broken, or transparent in places they should not be. The only ones that are passable are the fantastical ones like the sloth with the laptop.
- agitator 4y agoWhat's mind blowing is that you can extrapolate where this is going to go. Eventually, you will be able to generate full movie scenes from descriptions. What's interesting to me is how this is so similar to human imagination. Give me a description and I will fabricate the visuals in my mind. Some aspects will be detailed, others will be vague, or unimportant. Crazy to see how fast AI is progressing. Machines are approaching the ability to interpret and visualize text in the same way humans can. This also fascinates me as a form of compression. You can transmit concepts and descriptions without transmitting pixel data, and the visuals can be generated onsite. Wonder if there is some practical application for this.
- SuperCuber 4y agoRelated: https://pub.towardsai.net/stable-diffusion-based-image-compresssion-6f1f0a399202 https://pub.towardsai.net/stable-diffusion-based-image-compr...
- giarc 4y agoWould be interesting to input some existing screenplays into a future tool like this and see what comes out.
- bdickason 4y agoOr full 3D scenes that are interactive?
- treis 4y agoIMHO this particular avenue is a dead end. It's an extraordinarily impressive dead end but it's clear that there's no real understanding here. Look at this video of the knight riding a horse: >https://makeavideo.studio/assets/A_knight_riding_on_a_horse_through_the_countryside_second_upsample__12.webp https://makeavideo.studio/assets/A_knight_riding_on_a_horse_... The horse's face is all wrong The gait is wrong The interface with the ground & hooves is wrong The knight's upper body doesn't match with the lower and they're not moving correctly I think ultimately the right path is something like AI automated Blender. AI creates the models & actions while Blender renders it according to a rules based physics engine.
- sergiotapia 4y agoAt the rate AI is advancing, are we heading to a post-scarcity society, or a class of technolords and the other 99% scavenging for scraps.
- tracyhenry 4y agoIs the limitation that all pixels in the video would come from the first frame? since they are predicting pixel motion, basically.
- permo-w 4y agoI really don’t understand the fear people have about these things. have I missed something and everyone else was placing huge value in out of context videos and pictures? if you read “France declares war on Canada”, you’re not gonna believe it unless it’s coming from an extremely reputable source. so why would you trust a random unsourced video? the absolute worst thing that’s gonna happen is that video-based social media is gonna be flooded with low (or even high) quality AI videos. and I ask you: who the fuck cares? are these places doing wonders for society as it is? what’s a bit more rubbish amongst all the rest? I can think of many, many more upsides than down
- chatterhead 4y agoWe are quickly approaching a point where these independently created AI systems will be wrangled together the same away protocols were to create a computer network and the AI that emerges will likely be able to create its own code, solve its own problems and generate its own recursive processes. Everything at a conscious level without the ability to self recognize. If AI ever does wake up; we'll never know - first thing that happens is it will hide from us. The fear is real and only seems fantastical because life is often stranger than fiction. Instead of training on data these AIs will soon train on "creativity" and these layers of containerized thought will merge.
- permo-w 4y agowell perhaps that does sound like it might be an issue, but I’m talking more specifically about the societal consequences of video being made more like text
- lossolo 4y ago> first thing that happens is it will hide from us. To do this it would need to want to live and know what fear is. It's just a piece of code, without being conscious there is no need to be online/alive for it. What people call AI today is just a few/hundreds GBs of data that reacts to some text and push it through a heuristic like system to get an answer doing fancy pattern matching. It doesn't do anything when there is no input, there are no processes, no thinking, no anything, it's "dead". Fear and instinct is not taught in any species, it's inherited prior and biological process, you would need to explicitly code feeling into AI for it to be able to "feel" anything.
- mlsu 4y agoClearly, attention really is all you need. Are GPU vendors (well, gpu vendor, as far as I can tell) focusing heavily on increasing VRAM? My understanding is that transformers are pretty quick to train, but have significant memory costs. When they say that video is infeasible with memory... does that mean that if we had enough memory (128? 256? gb) we would be able to realistically train such networks with temporal attention? This is insanely exciting. It looks like we are limited, at this point, only by compute.
- sdenton4 4y agoWell, attention and aligned text+image datasets and diffusion processes... (and a fanatical devotion to the pope.)
- cainxinth 4y agoThe engineers of the future will be poets -Terence Mckenna
- nicoslepicos 4y agoReally cool seeing this & excited to really start playing with it. 2023 is going to be funky! It's pretty wild to see how quickly the space of Generative AI Media is coming along. I started a newsletter on the topic, called The Art of Intelligence (GPT-3 came up with the name) with the first post going out last Friday on the topic of how far are we from AI generated videos, and simulated worlds like the Holodeck given the rapid progress of these visual A.I. Thought y'all might find it interesting: https://artofintelligence.substack.com/p/dall-e-stable-diffusion-and-the-holodeck https://artofintelligence.substack.com/p/dall-e-stable-diffu... This type of progress also reminds me of a really lovely publication from 2017 in Distill.pub, on the topic of these A.I. enabled creation tools - I think y'all would enjoy seeing what folks were thinking even then: https://distill.pub/2017/aia/ https://distill.pub/2017/aia/
- anigbrowl 4y agoAlready obsolete XD https://news.ycombinator.com/item?id=33025189 https://news.ycombinator.com/item?id=33025189 Well, not quite obsolete, but you can see a significant qualitative difference between the two approaches.
- nonethewiser 4y agoImagine being able to spin up a completely novel movie on netflix
- bricemo 4y agoIf you look out long-term, 2030s and 2040s, this is where all of this is likely going. Right now the pool of content you might be interested in is constrained to all the content that has been made. But there may be better content that does not exist yet that you would be even more interested in. The future is going to be very weird, but also very entertaining
- rchaud 4y agoPrecisely, this is a dream for the likes of Netflix. They can perfect their vision of machine-generated assembly-line horseshit to feed to 190 countries. No staff, crew, no originality needed. Note the "AI by Meta" watermark in the example. So Zuck gets a cut as well.
- tehsauce 4y agoDoes anyone know anything about the size of this model? Number of parameters or flops? I couldn't find anything in the paper.
- sebringj 4y agoit looks like bizarre dreams as videos... so far but it will improve to less weird... eventually being used in VR or AR to give you instant environments/scenarios as you wish, I would imagine
- imhoguy 4y agoThis is going to be a revolution, I already imagine AI movies in cinemas, pioneering hits at scale of Avatar on IMAX 3D, remakes of classics just based on screenplay texts.
- nemo44x 4y agoWhen will one of these projects finally name themselves “Infinite Jest”? I’m guessing when the perfect pornography can be generated for you immediately, it will have an entertainment-to-death effect on a number of people. An Infinite Jest for conspiracy theories; one for shit posts, etc.
- qnsi 4y agoWe should just rename Internet to Infinite Jest. I mean, there are some theories that the mental illness epidemic is partially caused by internet use. I did strict 2 months Internet fast and I can attest it has a healing effect. Does it have entertainment-to-death effect? I guess there is a bigger than expected number of suicides that wouldn't happen have the people not been addicted to Internet. But I agree with every passing year we are moving further and further into entertainment-to-death with our civilisations. Makes me kinda sad to work in tech industry.
- simonw 4y agoI was pretty surprised to see that the WebVid-10M Dataset used as part of this training - https://m-bain.github.io/webvid-dataset/ https://m-bain.github.io/webvid-dataset/ - consists entirely of video preview clips scraped from Shutterstock! I built a quick search engine over that data: https://webvid.datasette.io/webvid/videos https://webvid.datasette.io/webvid/videos Wrote more about that here: https://simonwillison.net/2022/Sep/29/webvid/ https://simonwillison.net/2022/Sep/29/webvid/
- thomasahle 4y agoI wonder why the generated videos don't all contain the shutterstock watermark?
- tzot 4y agoThe maximum compression for a video file of a movie is its script. Perhaps eventually the most common (and time-tested) piracy platform will be email. Or maybe newsgroups (are they still on?), but the message payloads will be losslessly compressed text.
- ag8 4y agoAnother video generation model from today: https://news.ycombinator.com/item?id=33025189 https://news.ycombinator.com/item?id=33025189
- achr2 4y agoThe thing that frightens me is that we are rapidly reaching broad humanity disrupting ML technologies without any of the social or societal frameworks to cope with it.
- area51org 4y ago“Your scientists were so preoccupied with whether they could, they didn't stop to think if they should.” –pithy quote from a summer action movie I'll just say it now: this is a mistake, quite possibly a huge mistake. The average human is not intelligent enough to deal with computer-generated video that they can mistake for reality, and so this can and will become a tool for despots.
- washywashy 4y agoI could see this type of technology being used at some point in the future to essentially algorithmically generate content (movies, tv shows, etc) for viewers to watch or engage with. I wonder if it leads to extremely customized content where what I watch is 100% different from any other user on the same platform. However, I also wonder if people enjoy watching the same things because it becomes harder to talk about a movie you’ve seen if there’s no way anyone else could have ever watched it without you sharing it with them.
- deleted 4y ago[deleted]
- chrsig 4y agoNow we just need someone to ask it to make john oliver marrying a cabbage.
- gcanyon 4y agoAt this rate, within a few years we'll have people creating low quality movies just by describing what they want the movie to look like. Soon, every book will have a video version. As someone who has written several unpublished science fiction novels, I can't wait.
- dcl 4y agoOnly a small matter of time now before we have Vec2TikTok (or similar) where you'll be served up videos specifically optimised with the kind of content that will keep you scrolling or watching more. People will end up watching news about events that never happened or product reviews for things that don't exist.