7 ms·
Show HN: Create your own video clips with Stable Diffusion
A few months ago, I started making video clips with stable diffusion and noticed that the tools to do this were too complicated for everyday people. That's why I built neural frames. Enjoy.
- RichardGao112 4y agoNice! This is much simpler than using Deforum Is everything set up on the cloud yourself, or do you use an API?
- nicollegah 4y agoI've set it up myself. The APIs I saw didn't seem suitable for this real-time inference thing.
- jsjohnst 4y agoI’d much prefer to run this locally on my own. Any chance you’re willing to share code (or suggestions / links since it looks like you might be trying to monetize this per your Twitter posts)?
- nicollegah 4y agoI understand and have been thinking about it. Would somehow like to monetize this but if it doesn't work out I might just open-source it.
- RichardGao112 4y agoWhat were the APIs you looked at and how were they not suitable? Curious to know, since I'm currently developing a stable diffusion API at Evoke: https://evoke-app.com/ https://evoke-app.com/
- kruuuder 4y agoIs there an example video somewhere? Edit: Nevermind - I don't have autoplay active and there was no way to see that the "image" on the landing page is actually a video (no UI like a play button).
- nicollegah 4y agoit's on the landing page directly. You can also check https://twitter.com/neuralframes https://twitter.com/neuralframes
- lxe 4y agoLol did you mean to past that last one on your twitter? I think you might need some content filtering.
- nicollegah 4y agohahaha, i think you're right.
- nicollegah 4y agoHi all, thanks for all that attention to the site. I am super happy about that. A word of caution: I can only rent 4 GPU instances currently on AWS due to service quota limits. "Unfortunately" the traffic due to hackernews is too high for that. Sorry for any inconveniences. If you are having troubles at the moment, come back later or so. I asked for service quote increase but those take usually 24h or so. Also, very little people actually pay so I can only afford so much.
- snissn 4y agohey! FYI i think this limit is usally per geograhic site, so if you can only rent 4 in us-east-1 try us-west-1 etc
- nicollegah 4y agoah sick, i didn't know that. thanks!
- atylerrice 4y agoI don’t usually comment, but aws is actually rather expensive and i’ve hit this annoying quota problem as well. Been back and forth and still hasn’t been raised. I can also recommend coreweave banana.dev and pipeline.ai as great alternatives. This service looks awesome good luck with the launch!
- jamiedg 4y agoGreat project! We can probably host this for you for free until you work out your AWS credits. Email me at community@together.xyz.
- francis_lewis 4y agoThis is very cool! What sort of values are you using for the denoising strength/guidance scale? Each frame is a nice level of similar/different to the last to create flowing video.
- ecliptik 4y agoReminds me a lot of the "Ghost" music video for by Gunship[1] made by aiplague[2]. 1. https://www.youtube.com/watch?v=aUJuwNxNUWQ https://www.youtube.com/watch?v=aUJuwNxNUWQ 2. https://aiplague.com/ https://aiplague.com/
- jsjohnst 4y agoCouple suggestions after playing with this: 1) the experience mostly works on mobile (where I first tried it). With only minimal changes in the fixed sizing I think you could make this mobile friendly. 2) for posts shared via your Twitter, would be interesting to see details about the prompt(s) used vs “new post” 3) I’d like to have a bit more customization in the options Overall really nice and good luck with monetizing it. I’d love to see a blog post write up on the technical implementation. That’s something I’d more be willing to pay to see personally.
- nicollegah 4y agoThose are very good suggestions. I built it with mobile in mind but somehow I failed to get it right, yet. Will continue to work on it. Would also be nice to make a native app at some point.
- will5421 4y agoWhat prompts did you use to make the video?
- gmuslera 4y agoThe description of what you have to do reminds me the instructions on how to draw an owl (https://knowyourmeme.com/photos/572078-how-to-draw-an-owl https://knowyourmeme.com/photos/572078-how-to-draw-an-owl)
- nerdponx 4y agoWhat will be interesting is whether we will soon have a model that can actually follow these steps.
- deleted 4y ago[deleted]
- throwmeup123 4y ago...no noise overlay on the input images to at least get some sort of frame by frame consistency? -.-
- nicollegah 4y agoDo you have some resource to learn how this works? I'd love to implement it.
- refulgentis 4y agoIt's baked into Deforum, the bit you'd want to look into here is the recent changes for Perlin noise.
- pcrh 4y agoThe more AI-generated art I see the more convinced I am that it will generate entire new modes of art creation, rather than make creative work redundant. I'm now waiting for a creation that could not have been done without AI, the labour that would be involved to create these works manually not being considered.
- nuclearsugar 4y agoI've been experimenting with creating image datasets via Stable Diffusion, then training StyleGAN2, and compositing in After Effects. It's a new mode of art creation for me that I cannot recreate otherwise. https://www.jasonfletcher.info/vjloops/ https://www.jasonfletcher.info/vjloops/
- adzm 4y agoThis is some truly amazing art. I kept reading one post after another of yours.
- pcrh 4y agoAwesome! It reminds me of time-lapse movies in developmental biology, showing, for example, the various shapes an organism has as it develops.
- mustacheemperor 4y agoThis takes me back to the demo scene days. Seeing something that’s both a technical and artistic achievement that could only have been borne out of the cutting edge. Thanks, awesome stuff.
- imhoguy 4y agoOne video brought me memory of Autechre - Gantz Graf music video (warning, hard IDM style ;) [0]. It was made manually in 2002, per Wikipedia: "Rutterford also stated that there was no generative element to the imagery; every three-dimensional object in the agglomeration was painstakingly and manually synchronised with a specific element or frequency range within the track" [1]. [0] https://www.youtube.com/watch?v=ev3vENli7wQ https://www.youtube.com/watch?v=ev3vENli7wQ [1] https://en.wikipedia.org/wiki/Gantz_Graf https://en.wikipedia.org/wiki/Gantz_Graf
- deleted 4y ago[deleted]
- Magi604 4y agoReminds me of something you would see in a newer modern remake of Decasia. https://en.wikipedia.org/wiki/Decasia https://en.wikipedia.org/wiki/Decasia https://www.youtube.com/watch?v=hDa-mmSldDg https://www.youtube.com/watch?v=hDa-mmSldDg
- O__________O 4y agoMight want to put notice: “There's a lot of requests currently and our servers are overloaded - sorry. I am trying to increase the capacity. Please try again later.” …prior to making users do multiple clicks, opt to not provide an email, etc.
- deleted 4y ago[deleted]
- O__________O 4y agoThen, if I finally try to render a 5 second click I get: “There's high demand on the servers currently. Sorry for any inconvenience. I am trying to scale up the servers.” RIP
- O__________O 4y agoMight want to put notice: “There's a lot of requests currently and our servers are overloaded - sorry. I am trying to increase the capacity. Please try again later.” …prior to making users do multiple clicks, opt to not provide an email, etc. Even a prerecorded demo posted to Youtube would be a better experience.
- deleted 4y ago[deleted]
- lukeplato 4y agoI suggest that people simply run Stable Diffusion Deforum themselves (there's an extension for automatic1111's web UI). You can run it in a google colab notebook and the cost will likely be cheaper or the same, though I haven't bothered to compare.
- tehsauce 4y agoVery cool! I just released a tutorial on how to do this using the computerender api! https://github.com/computerender/tutorials/tree/main/python/vid2vid https://github.com/computerender/tutorials/tree/main/python/... If you’re interested in saving money on expensive cloud gpus, our api is much cheaper than this. (only $0.001-0.0025 per frame)
- conidig 4y agodo you plan on adding dreambooth? I would give it a try.
- tehsauce 4y agoI would like to support dreambooth, if there is a way to store the fine-tuned models more efficiently. The challenge is that each trained model is quite large and a bunch of models can't fit into one gpus memory at once.
- conidig 4y agotextual inversion maybe? lightweight embeddings and I haven't seen any API offering it at the moment.
- spyder 4y agoTry the Lora fine-tuning method with few MB results: https://github.com/cloneofsimo/lora https://github.com/cloneofsimo/lora
- tehsauce 4y agoThis looks great! Thanks for sharing.
- nicollegah 4y agolooks cool. somehow the website looks screwed up in my browser (chrome) and I cannot get an API key after signing up.
- NicoleJO 4y agoThe company that makes stablediffusion is being sued for copyright infringement. Don't use this.
- jtsiskin 4y agoEveryone’s being sued for something somewhere. You can use this.
- wellthisisgreat 4y agoThat’s a strange statement. Who cares about frivolous lawsuits. Stable Diffision and the like tools will prevail. The main hope is that it will be their OSS versions and not corporate (Dall-E). The works and ways of lives that are threatened by SD and the like are not worth preserving
- ghaff 4y agoI wouldn't call them frivolous. Quite a few people believe training generative AI models on their copyrighted work and providing others with access to that model violates their copyright. IANAL but it seems a weak argument to me. And it seems weak to some IP lawyers I know as well. It also seems like technology that can't be realistically bottled back up. However, I wouldn't call lawsuits frivolous and it might actually be useful to get some legal clarity
- cornel_io 4y agoI have yet to hear a copyright lawyer who actually understands how these tools work say anything other than that it's going to be fair use; the ones that I have seen comment on it as such seem to have extremely misunderstood how the tech works and how the images are used, they've just regurgitated the arguments that the artists opposed to it make (about how it copy and pasted images, etc). Maybe I'm wrong, also not a lawyer, but I've heard from enough at this point (including ones that I've paid and are actually our lawyers as we evaluate internal use of the tools) that I definitely wouldn't bet on these suits succeeding.
- 4y ago
- tekni5 4y agoVery cool, your demo video looks great, would love to try it when it's working again. You can also do this type of thing on colab with Deforum Stable Diffusion: https://colab.research.google.com/github/deforum-art/deforum-stable-diffusion/blob/main/Deforum_Stable_Diffusion.ipynb https://colab.research.google.com/github/deforum-art/deforum... I've been messing with it myself, here is an example: https://youtu.be/FsVskNtNazk https://youtu.be/FsVskNtNazk
- nicollegah 4y agoIt should absolutely work now. Doesn't it?
- tekni5 4y agoSeems to be overloaded. "There's a lot of requests currently and our servers are overloaded - sorry. I am trying to increase the capacity. Please try again later."
- jimhi 4y agoI put it on for hundreds of hours to try out a 24 fps video myself: https://www.youtube.com/watch?v=f3GfUKJBUYA https://www.youtube.com/watch?v=f3GfUKJBUYA
- antipotoad 4y agoSadly YouTube’s compression absolutely crushes the quality out of your video. Love the concept though.
- tekni5 4y agoPretty cool, I would upscale to 1080p or even resize as youtube will compress it better or less for hd videos.
- seydor 4y agoYannic kilcher had created a videoclip for lyrics made of imagenet labels last year: https://www.youtube.com/watch?v=rR5_emVeyBk https://www.youtube.com/watch?v=rR5_emVeyBk
- theogravity 4y agoThe site should have a demo video of what an output looks like instead of a static image. I stopped at the signup form.
- selcuka 4y agoIt's a video, not a static image. You should try reloading the page.
- jtbayly 4y agoI don't see an image or a video... :shrug:.
- ada1981 4y agoI’m just getting a static image on iPhone as well. Is there a youtube channel?
- selcuka 4y agoIt's an embedded mp4 file: https://neuralframes.s3.eu-central-1.amazonaws.com/20221214100256.mp4 https://neuralframes.s3.eu-central-1.amazonaws.com/202212141...
- deleted 4y ago[deleted]