31 ms·
Open-source rival for OpenAI’s DALL-E runs on your graphics card
- belltaco 4y agoArticle says it needs 5.1GB of Graphics RAM. Does any one know how much data download and disk storage does it need?
- _blop 4y agoThe v1.3 model weighs in at 4.3 GB. There's an additional download of 1.6 GB of other models due to usage of huggingface's transformers (only once on startup). And the conda env takes another 6 GBs due to pytorch and cuda. Larger images will require (much) more than 5.1 GB. In my case, a target resolution of 768x384 (landscape) with a batch size of 1 will max out my 12GB card, an RTX3080Ti.
- mdorazio 4y agoI think this is a good time to ask if anyone is working on parallelizing machine learning compute anymore? For at-home computation like this it seems like it would be a lot better to allow people to stack a few cheaper GPUs rather than having to pony up thousands of dollars for ML-oriented beast cards to be able to do things like generate large images.
- andybak 4y agoAI upscaling will solve everything ;) I've generated some remarkably good-looking print quality images by upscaling 512x512 sources
- bambax 4y agoWhat do you use for upscaling? Standard software like Photoshop or Affinity, etc. or more dedicated software? Any recommendations for options, etc.?
- andybak 4y agoRecently impressed by https://replicate.com/nightmareai/latent-sr https://replicate.com/nightmareai/latent-sr but otherwise - Cupscale
- andybak 4y agoHere's a particularly impressive result I got from the former (considering it's not especially optimized for "vector art" enlargement): https://twitter.com/andybak/status/1558737805546749953?s=20&t=-8KHnJ1EYc2T4l5700n0TQ https://twitter.com/andybak/status/1558737805546749953?s=20&...
- glenneroo 4y agoFor videos in particular, if you don't mind shelling out cash, the current go-to (at least according to various AI discord servers I'm on) for AI animation nerds is currently Topaz upscaler. There are free alternatives but I've yet to see any of them work as well as Topaz, though I'm sure that will change soon. For interpolating frames Flowframes is "free" (new features if you join the Patreon) and is IMO very good. I've seen a number of 80s/90s VHS recordings of concerts being uploaded to YouTube in 4K (using Topaz) and they look like they were recorded that way, truly amazing. I do hear it can be a bit of work though getting the settings right.
- luismmolina 4y agoIf you read directly from the site. The requirements for the graphic card are 10 VRAM as a minimum. Because it's ruins locally you don't need to download anything apart from the initial model, this applies to the disk space too.
- axg11 4y agoI’m excited for the coming race to improve and miniaturise this tech. Apple has a great track record of making ML models light enough to run locally. There will come a day when photorealistic image generation can run on an iPhone.
- andrewacove 4y agoMaybe this is their long term plan for getting rid of the camera bump.
- fswd 4y agoUnfortunately it's a commercial license and the model isn't available to the public so it isn't very useful.
- andybak 4y agoIt's going to be MIT from what I have heard. On phone atm so can't provide sources.
- fswd 4y agohttps://stability.ai/research-access-form https://stability.ai/research-access-form So far, I haven't got a response but it's a Monday Here's the license https://github.com/CompVis/stable-diffusion/blob/main/LICENSE https://github.com/CompVis/stable-diffusion/blob/main/LICENS...
- andybak 4y agoThat’s just a restricted interim release. The proper public release isn’t ready yet. No timescale but sounds like days/weeks rather than months/years.
- fswd 4y agolike OpenAI?
- andybak 4y agoNot sure I follow. OpenAI are not claiming they are going to release their model at all. The team behind Stable Diffusion have currently kept every promise they've made. (And if you're insinuating something, just come out and say it so people can engage appropriately)
- donkarma 4y agoi think he meant are they releasing a form like openAI in the interim release? someone's a bit aggro
- kgc 4y agoDoes this work on Apple silicon processors? They have plenty of RAM accessible to the GPU.
- sroussey 4y agoThe articles says it will, but that it is not using the GPU unfortunately.
- humanistbot 4y agoCan someone tell me how this compares to the guide and repo shared a few days ago on HN: https://news.ycombinator.com/item?id=32384646 https://news.ycombinator.com/item?id=32384646
- sailingparrot 4y agoThis version is a bit more optimized, and better packaged. Also the model has been trained longer, so when the weights become publicly available the resulting quality should be much higher.
- Geee 4y agoThere's also Disco Diffusion: https://www.reddit.com/r/DiscoDiffusion/ https://www.reddit.com/r/DiscoDiffusion/ Not sure how they compare. DD seems to be quite popular. I'm currently setting up DD locally.
- glenneroo 4y agoI've been running DD for a few months now... I tend to just edit the python script or use e.g. entmike's fork which can read config files to make changes to the 50+ parameters (basically everything is better than having to use Jupyter Notebooks IMO), granted if you don't have a GPU with 6+ GB of VRAM, you can often get a decent enough GPU for free from Google Colab. For running locally, I can also highly recommend Visions of Chaos, which includes multiple versions/forks of Disco Diffusion, as well as a ton of other latent diffusion scripts, not to mention many many many other generation features such as fractals and even music. They also recently added the ability to train your own diffusion models which I've been doing the last few days using thousands of my own photographs. It also has a pretty nice GUI and the dev is extremely responsive on Discord. Also after you do the setup for VoC it handles running all the python venv setup stuff otherwise necessary with local DD installs. In any case, check out DD Discord and/or VoC discord for lots of info, tips, help, examples, and support.
- Geee 4y agoThanks for the info. It is possible to do something like transfer learning on top of existing models or do you train your own models from scratch? I'll check out that Vision of Chaos thing. I'm just beginning my journey into this generative art stuff and just basically trying to get this running right now.
- upupandup 4y agoMy friend wants to know when she can use this to generate porn, are we close?
- sailingparrot 4y agoThe data used to train those model is specifically filtered to remove sexual content, so the model can't generate porn because it has no idea what it looks like, beyond a few samples that made it past the filter. So no, your "friend" can't use it for that.
- upupandup 4y agoWhy is it that sexual content is so frowned upon in this space? If it's a content publishing platform I would understand that advertisers don't want that, but this is literally dictating people what is bad and good. I just don't understand this Puritan outrage with text-to-image porn generation.
- blowski 4y agoI'd guess that, for general purpose companies, it's an area full of legal ambiguity and potential for media outrage, so just not worth the risk. However, given the evidence of human history, it's certain that someone with an appetite for exploiting this niche will develop exactly that kind of tool.
- spywaregorilla 4y agoBecause it's a lot more annoying for your innocuous content to be rendered as porn when the ai happens to interpret it that way than it is for you to be unable to render your pervy desires intentionally. A porn model should really be it's own thing.
- sailingparrot 4y agoBecause you can't control what the model is going to output in response to a query. The model is trained to respond in a way that is aligned but there is no guarantee. Since we certainly don't want to show generated image of porn or violence to someone that didn't specifically ask for that, the easiest way to ensure that's not going to happen is to just not train on that kind of data in the first place. The worst that can happen with a model trained on "safe" images is that the image is irrelevant or makes no sense, meaning you could deploy systems with no human curator on the other end, and nothing bad is going to happen. You lose that ability as soon as you integrate porn. Also with techniques like in-painting, the potential for misuse of a model trained on porn/violence would be pretty terrifying. So the benefits of training on porn seems very small compared to the inconvenience. I don't think it's anything to do with puritanism, it's just that if I am the one putting dollars and time to train such a model I am certainly not going to be taking on the added complexity and implications of dealing with porn to just to make a few people realize their fetishes at the risk of my entire model being undeployable because it's outputting too much porn or violence.
- dang 4y agoRecent and related: Stable Diffusion launch announcement - https://news.ycombinator.com/item?id=32414811 https://news.ycombinator.com/item?id=32414811 - Aug 2022 (37 comments)
- unethical_ban 4y agoWait, so the closed source generator known as DALL-E is owned by a company called OpenAI?
- fariszr 4y agoAs Elon said "OpenAI should be more Open IMO"
- glenneroo 4y agoCuriously it seemed to lock down even more after they "partnered" with Microsoft.
- alephxyz 4y agoIt's a bit of a dead horse at this point but yes. See the previous discussion: https://news.ycombinator.com/item?id=28416997 https://news.ycombinator.com/item?id=28416997
- humanistbot 4y agoIf anyone from Stability is reading, the confirmation e-mail to sign up is sending a broken link: "We couldn't process your request at this time. Please try again later. If you are seeing this message repeatedly, please contact Support with the following information: ip: XXXX date: Mon Aug 15 2022 XX:XX:XX GMT-0700 (Pacific Daylight Time) url: https://stability.us18.list-manage.com/subscribe/confirm https://stability.us18.list-manage.com/subscribe/confirm"
- wccrawford 4y agoIt worked for me just now, so maybe it was temporary, or they already fixed it?
- hifikuno 4y agoI also had this response.
- notrealyme123 4y agoI forwarded this Thread to a member of the Project.
- stuckinhell 4y agoThis is pretty amazing, anyone have any tips on building a pc for machine learning with a RAID device ?
- cellis 4y agoLook into building an ethereum mining machine... it can double as an ML workstation. That's what I did.
- jessfyi 4y agoHasn't been updated since 2020, but Tim Dettmer's guide [0] is pretty much the gold standard for optimizing what to buy for which area of DL/ML you're interested in. The pricing has changed thanks to GPU prices coming back down to earth a bit, but what to look out/how much ram you need for which task hasn't. Check out the "TL;DR advice" section then scroll back up for detail info on why and common misconceptions. For the tips on a RAID/NAS setup alongside it, just head to the datahoarders subreddit and their FAQ. [0] https://timdettmers.com/2020/09/07/which-gpu-for-deep-learning/ https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...
- hedora 4y agoIf you just want to try it out, consider using a remote cad workstation from a company like paperspace. (No affiliation.)
- 999900000999 4y agoHas anyone made a pixel art generator, that can create the animation sprites ?
- gxqoz 4y agoYou can use DALL-E and other models to make pixel art ("as pixel art"), although it can both be overkill and hard to get consistent results that you'd put into animation. I'm guessing that starting from more of a video model and then converting to pixel art could be better. Although it's also non-trivial to turn "realistic" video into convincing animation.
- 999900000999 4y agoI pay good money for a specialized machine learning algorithm that can take a pixel art character, and then generate all the animated sprites for it. I actually tried to get Dalle to do this, And it made like three good sprites in the rest were just broken. But it was so strange, because you could see it was still organized as a sprite sheet, it's just the sprites were useless. I think the practical applications of this technology will be hyper specialized models for specific purposes.
- 0xdead1eaf 4y agoCheck out NUWA-Infinity[0][1], submitted to arxiv jul 20, 2022. It captures artistic style very well (though can't speak to the quality of the pixel art it would generate) and can do image to video. [0] https://nuwa-infinity.microsoft.com/#/ https://nuwa-infinity.microsoft.com/#/ [1] https://arxiv.org/abs/2207.09814 https://arxiv.org/abs/2207.09814
- tckerr 4y agoThis is exactly the type of application I am interested in as well. As a hobby game dev with only mediocre pixel art skills, having a generator to finish the busy work would be an absolute lifesaver. I'm also interested in using it for fleshing out artistic vision through generating variations of an initial concept. Hopefully we aren't more than a few years away from something practical like this.
- 4y ago
- ruuda 4y agoThe site shows a notification in German that I need to enable JavaScript to use the site, after the first paragraph. But then after that is the full article, including images, which is almost perfectly readable, except it's at 5% opacity (or maybe the JavaScript popup is 95% opacity overlaid on the article), which makes it impossible to read again. :'(
- deleted 4y ago[deleted]
- PoignardAzur 4y ago> Of course, with open access and the ability to run the model on a widely available GPU, the opportunity for abuse increases dramatically. > “A percentage of people are simply unpleasant and weird, but that’s humanity,” Mostaque said. “Indeed, it is our belief this technology will be prevalent, and the paternalistic and somewhat condescending attitude of many AI aficionados is misguided in not trusting society.” Holy shit. On the one hand, I'm super excited by this technology, and the novel applications that will become possible with these open-source models (stuff that would never be usable if Google and OpenAI had a monopoly on image generation). On the other hand, I really really really hope Bostrom's urn[0] has no black ball in it, because we as a society seem to be rushing to extract as many balls as possible over increasingly short timescales. [0] https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf
- sinenomine 4y agoWhat if the black ball was a red herring all along and the usual suspect tech-CEO's hand(s) rushing to control the said crystal ball are the real hazard?
- jackblemming 4y agoYes, I feel much safer if OpenAI and Google are the sole keepers of such technology. They have my and the publics best interest at heart.
- geraldwhen 4y agoIs this satire?
- keepquestioning 4y agoWe are heading into uncharted territory :(
- vanadium1st 4y agoStable Diffusion is mind-blowingly good at some things. If you are looking for modern artistic illustrations (like the stuff that you would find on the front page of Artstation) - it's state of the art, better in my opinion then Dalle-2 and Midjourney. But, the interesting thing is that while it is so good in producing detailed artworks and matching the styles of popular artists, it's surprisingly weak at other things, like interpreting complex original prompts. We've all seen the meme pictures made in Craiyon (previously Dalle-mini) of photoshop-collage-like visual jokes. Stable Diffusion with all its sophistication is much worse at those and is struggling to interpret a lot of prompts that the free and public Craiyon is great with. The compositions are worse, it misses a lot of requested objects or even misses the idea entirely. Also as good as it is at complex artistic illustrations, it is as bad at minimalistic and simple ones, like logos and icons. I am a logo designer and I am already using AI a lot to produce sketches and ideas for commercial logos, and right now the free and publicly available Craiyon is head and shoulders better at that then Stable Diffusion. Maybe in the future we will have a universal winner AI that is the best at any style of pictures that you can imagine. But right now we have an interesting competition when different AI have surprising strengths and weaknesses and there's a lot of reason in trying them all.
- russdill 4y agoJust think where we'll be two more papers down the line
- elil17 4y agoFor those unaware this is a catchphrase of Dr. Karoly Feher from the absolutely wonderful YouTube channel "Two Minute Papers" which focuses on advances in computer graphics and AI.
- pizza 4y ago> Now squeeeze those papers!
- PoignardAzur 4y ago
- colordrops 4y agoYet another "open" model that isn't open. We shall see if they actually do release to the public. We keep seeing promises from various orgs but it never pans out.
- TulliusCicero 4y agoTheir plan seems less hand wavey, they're being explicit with "first we release it like this, then like that, then freely to everyone". You're right that they could always change their minds and that would suck, but so far they seem to be being up front.
- runnerup 4y agoZero of their flagship models have been released AFAIK, e.g. the pre-DALLE model, CLIP, has not had weights for its ViT-H/16 model released from a paper 19 months ago. This was before DALL-E and way before DALL-E 2. Not that I think they need to. It’s just weird people defend them as being open just because they release crippled versions of their model.
- TulliusCicero 4y agoThey've said that they intend to release the model soon-ish for anyone to run on their own machine, with no censorship*. I don't think the other major players, like OpenAI, have committed to doing the same. * They're actually working on a filter right now, but IIRC it's an optional one, for when you don't want to accidentally generate NSFW output.
- runnerup 4y agoI'm so sorry I thought the discussion was about OpenAI, I'm glad you clarified it was about StableDiffusion. Hope I didn't muddy the waters too much for others.
- TulliusCicero 4y ago
- thorum 4y agoIf you want to see more examples of what this AI is capable of, check out the subreddit: https://reddit.com/r/stablediffusion https://reddit.com/r/stablediffusion
- dustingetz 4y agodoesn’t build on my mac studio due to a dependency whose mac version is two major versions behind
- rasz 4y ago3 days from launch to getting your twitter account suspended. https://twitter.com/DiffusionPics/ https://twitter.com/DiffusionPics/
- ShamelessC 4y agoContext?
- kragen 4y agoThis article says both that it's "open-source" and that it's "available for [only] research purposes upon request". These can't both be correct. Where is the error?
- th1s1sit 4y agoWould be a blast if the cloud is up ended by RISC and GPUs powerful enough to crunch “big data” at home. Would love to see FAANG and SV crash and burn, margins chipped away to nothing.