5 ms·
I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon. They seem like pretty much the perfect fit for cloud - b
by Closi 3y ago
I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon.
They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.
Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?
There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).
- baz00 3y agoYou can run this on an M2 Mac Studio. That’ll be consumer level hardware in a few years.
- Closi 3y agoRun what? Adobe's offering is cloud-only, and I don't think the hardware requirements are disclosed.
- unixhero 3y agoThe models obviously. Thsse are for the most part available open source on Github
- ShrigmaMale 3y agoFirefly models are not publicly available. The integration with creative cloud apps is not customizable. You can't point it to a local stablediffusion setup. Open-source competitors will flop just like GIMP.
- nickthegreek 3y agoFyi, there are Adobe plugins that do allow y you to use a local SD install.
- ShrigmaMale 3y agoI mean that sounds good and cool and worth trying for the average hackernews, but not for the average adobe user. Like people are not paying for AI tech to get the 80% mostly-there version, they're paying for the 100% spit-and-polished end product that has the 80% of the work needed to achieve the last 20% of productizing. Consumers have demonstrated over and over they will pay a lot more to avoid dicking around with stuff and professionals can have an even stronger tendency to do this bc their time has a cost.
- wayfinder 3y agoI have an M1 but I first remember thinking it was very impressive that my Mac could run these AI models… …until I tried the same on my RTX 4070 and it made my Mac look like a joke. For the 30 seconds my Mac would have taken for 1 result, which will probably need revising, the RTX would give me 30 results. However the RTX was half the cost of my Mac, so it’s not a good investment if I just want to generate some images. I’d rather pay for the cloud if I didn’t have the RTX already.
- teaearlgraycold 3y agoWhich M1? That makes a big difference. And the Mac Studio runs the M2 Pro/Max/Ultra.
- internet101010 3y agoYeah my M1 Max is about as fast as my old 1070, at 24 steps per image it comes out to little over 2 images per minute. My new 4090 does an image per second.
- liuliu 3y agoIt should be about 6 images per minutes (for SD v1 at 512x512) and 1 image per minute on SDXL for the said hardware (M1 Max): https://engineering.drawthings.ai/integrating-metal-flashattention-accelerating-the-heart-of-image-generation-in-the-apple-ecosystem-16a86142eb18 https://engineering.drawthings.ai/integrating-metal-flashatt...
- anonylizard 3y agoM2 will be a joke for the AI models of the future. The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM). That's why their quality is leaps and bounds above stable diffusion XL, its because the model size simply allows for it. Model sizes continuously grow to exploit the available hardware to the limit. Midjourney and GPT-4 have both proven that model quality is decisive to success and paying customers, so consumer hardware can never catchup to whatever Nvidia sells to the cloud.
- l33t7332273 3y agoHow much vram does an a100 have? The M2 ultra has more than 100 GB.
- Auracle 3y agoI don’t think I’d say Midjourney is “leaps and bounds” above SDXL. It’s better overall, but SDXL closed the gap a significant amount.
- polygamous_bat 3y ago> The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run Do you have any source for this speculation? In my experience image models are always much smaller than language and even the largest llama will fit in a smaller GPU machine than that.
- GaggiX 3y ago>The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM) Unet are really expensive to run compare to a regular GPT model and they are compute-bound thx to convolution, a reason why no one has trained a unet that comes close to consume 100GB of VRAM during inference. I doubt that MJ is much bigger than SD XL, it's good but not revolutionary.
- saberience 3y agoHave you tried actually downloading SD and all the various custom models and tools? For me it kicks the shit out of Midjourney in flexibility and quality. I can make more images of higher quality, faster and cheaper.
- asynchronous 3y agoNot true. Stable Diffusion locally on my 3080 is already faster than Adobe’s offering, not to mention after all my model tweaking and fine tuning much more useful and accurate. Is it overall easier to use or setup? No, but I can run it for free over Adobe.
- freeere 3y agoIt's not free for you. Energy, partial hardware cost, setup time, fine-tuning time.
- weird-eye-issue 3y agoThat's good for you but not 99.99% of the population
- tavavex 3y agoI honestly don't think that only 1 in every 10000 Adobe users own a modern-ish gaming-capable PC (and that's what's required to run SD as of now). Point being, the fact that you can already run these models on off-the-shelf consumer hardware is very promising, and as time goes on, more and more new hardware will be capable of running this.
- freeere 3y agoI assume that they actually use the hardware /GPU they have and might not be able to run SD next to it constantly without upgrading. But someone needs to make this possible and maintain such a solution which would cost also money.either you pay adobe what you already do or pay someone else who maintains the model, the infrastructure etc. Sure some will run it themselves but my guess is that this is a niche group of people as most don't care .
- Closi 3y agoOP will have been exaggerating with 99.99%, but I suspect 90% of people don't have a PC with something equivalent to a 3080. And designing your software to a minimum-spec of a 3080 would be pretty wild.
- Sulf1re 3y agoAt 5usd for 100 operations, that thousand dollar gpu will break even very swiftly. I used 5000 or more runs when I was learning to use stable diffusion. In a commercial context, staff training alone will pay for the cards
- _jal 3y agoMy thoughts exactly. I did some guesstimates last month, the experimentation and playing I've done with my 3090 so far would have cost somewhere around $7k on the metered OPC options.