9 ms·

I'm convinced this will be a short lived business revenue structure - paying per use of generative AI in the cloud. I'm sure that in the not too distant future

I'm convinced this will be a short lived business revenue structure - paying per use of generative AI in the cloud. I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. I do wander if companies working to develop these type of revenue models truly think it's a long term structure?

gochi 3y ago
They can upgrade hardware to bleeding edge faster than consumers can get their hands on remotely comparable products. Most consumers also don't upgrade every year to stay on bleeding edge, they just tolerate what they can currently do. While this will be fine for most, those actually using generative AI for work aren't likely going to tolerate stagnation as easily.
coder543 3y ago
I would have said the same thing about CAD simulations, but then Autodesk decided to move backwards and remove an existing feature just so they could charge for per-simulation credits: https://hackaday.com/2022/08/12/local-simulation-feature-to-be-removed-from-all-autodesk-fusion-360-versions/ https://hackaday.com/2022/08/12/local-simulation-feature-to-... Whether Adobe ever decides to let their model run locally or lock it forever into the cloud is a choice they will have to make. A lot of people trust Adobe products, so it's entirely conceivable that some people will always choose to pay for a pay-per-use generative solution from Adobe rather than try to run competing solutions locally. The question is probably whether it generates more revenue than negativity for Adobe. If most Adobe users are running their own models locally and avoiding the feature, then I think Adobe will be more likely to follow suit and move away from the pay-per-use cloud approach.
- flangola7 3y ago
  Long term they may offer both, with the local model consuming fewer credits per request. The new GPUs all have secure enclave remote attestation architecture, so Adobe would be able to offer this without the risk of someone jailbreaking the model and running it for free.
  - fragmede 3y ago
    Assuming that their secure enclave is, in fact, unhackable, that is.
    - flangola7 3y ago
      The Xbox One has not been hacked in the 10 years since its release. Apple already uses the same kind of tech to protect the various AI models in iPhones and MacBooks, for tasks like OCR search of stored image files and to auto-cutout people in photos. Enclaves are rarely broken and the people that can are selling it to the CIA, not leaking it on the pirate bay.
  - tavavex 3y ago
    What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so. It'd be like charging per use of any other advanced features in locally-run Adobe software.
    - taneq 3y ago
      Yeah, that’d be as crazy as charging car owners a subscription fee to use their heated seats.
    - flangola7 3y ago
      > What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so. To pay for the privilege of using a very advanced AI model. That's more reasonable than paying to unlock a game character skin that's already on your SSD, and that happens millions of times a day. > It'd be like charging per use of any other advanced features in locally-run Adobe software. I don't see anything stopping this.
    - coldtea 3y ago
      >It'd be like charging per use of any other advanced features in locally-run Adobe software. Software companies already do that. There are all kinds of locally-run advanced features that are only enabled with a more expensive subscription tier even though you already have the code and assets for them. Sure, most are merely subscription based, but there are others that are per use.
- xmprt 3y ago
  > A lot of people trust Adobe products I would have said the consumer sentiment amongst Adobe users is the exact opposite - that people don't trust Adobe products but they use them because they either have to or because they're currently the best products available.
- gmerc 3y ago
  This hinges on having a quasi monopoly.
amelius 3y ago
> on customer level hardware Unless nVidia changes their monetization model, and for example introduces an App Store for AI, with subscriptions, of course on locked down hardware.
code51 3y ago
Adobe is planning for a post-regulation, post-biggest-lawsuit world for sure. All their steps show that they'll base their offering on their own commercial data - paying training license fees to stock images and artists. Whether the amounts they pay would make licensing your work sensible or not, Adobe is surely assuming this will ultimately end up as Napster-to-Spotify transition. If we end up "happily" (means legally as well) running these on customer level hardware, then the question won't be about credits of computation. It'll be about credits to use licensed work.
- sebmellen 3y ago
  This is the most plausible reason from what I see.
- GaggiX 3y ago
  > It'll be about credits to use licensed work. If this is true (which I kinda doubt), is it going to matter to most people? Like you can't really tell the images used to train a model from the images it generates (if it's trained correctly), so I doubt the majority of people would care, like those who already use MJ for example. Training models on copyrighted data for academic research will be allowed, the models will be published, and good luck enforcing the licence; and here I'm talking about the worst case scenario where a court would find an image generated by AI to be derivative of another image in a pool of billions in a dataset (this goes way beyond any definition of derivative work for now).
  - gmerc 3y ago
    Ya, enforcement would have to go along with a war on general purpose computing and privacy. Oh wait, …
- whiddershins 3y ago
  That’s a big bet. My instinct is it doesn’t go that way. We will see.
yieldcrv 3y ago
I agree, these models are going to get smaller and more performant, the software to leverage your hardware is going to have major improvements, the hardware is going to change to prioritize processing these with more memory to specific coprocessors, and the OS is going to start having these models baked into them, with improved default models being a core feature of upgrading each OS version people are looking at an extremely limited view of “bigger models on better hardware will always be in the cloud” when that reality simply won’t matter for most use cases
YurgenJurgensen 3y ago
Wouldn't this require a complete reversal of course of, like, the last fifteen years of computing? Basically everyone wants to get out of selling products and into selling services.
thorum 3y ago
When we reach the point when average people can easily run today’s models on their own devices, today’s models will no longer be SOTA and there will still be demand to run better models in the cloud.
- brucethemoose2 3y ago
  Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL. The tooling around them is far more important. However, Stable diffusion already can run on mobile devices. There is already a good iOS app for it (and the dev is here on HN) but the problem seems to be that no one cares. There are 700,000 cloud imagegen apps crowding it out, because thats what's easier and more profitable to spam across the store and web.
  - thorum 3y ago
    > Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL. For image quality, sure - language understanding is still an issue. SDXL can generate a beautiful image, but if it doesn’t show exactly what you asked for in the prompt, on the first try, there is still room for improvement. The gap between LLMs and image generators in this regard is huge.
  - Closi 3y ago
    Depends if the limit on parameter count is a 'real' limit, or just a limit based on what current-technology models can effectively use. Back in the 'Google Daydream' days, Google might have found that they didn't get any more image-generation performance by raising the parameter count - but that's just because the technology at the time couldn't effectively utilise more parameters. It's impossible to know what next-gen models might be able to use, but I suspect we will find ways to allow the models to take advantage of even higher parameter counts. Stable diffusion can run on mobile devices, but it's painful and image generation takes a fraction of the time via cloud services.
GaggiX 3y ago
We already run these on customer level hardware, you need a good PC but still, in the future the pool of people that can do is just going to be bigger.
bhouston 3y ago
> I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. I doubt that is what Adobe will do. This is a new revenue stream for them, why would they remove it? Gimp will use local generation but Adobe is using a proprietary dataset that they can keep secure in the cloud. So yeah this is going to be sticking around.
giancarlostoro 3y ago
> we will be happily running these on customer level hardware. This is the stage of AI that will impress me most. If I can use your AI completely offline on my device on a spaceship orbiting Pluto, then I will say we have achieved an AI capacity that is impressive, even if its got the quirks of chatgpt today.
- brianwawok 3y ago
  We can already do that. But why give away the keys to a money machine?
  - giancarlostoro 3y ago
    So you can fit all of ChatGPT into a 32GB storage phone? Because my understanding is something like ChatGPT takes terrabytes upon terrabytes of data on-disk and I can't imagine how many servers being piped together to make it churn out coherent sentences... I'm on about all of that on a phone that doesn't cost thousands of dollars. It would mean we have finally reached significant technological advances.
    - brianwawok 3y ago
      Okay this is called a moving goalpost. This was my response to > If I can use your AI completely offline on my device on a spaceship orbiting Pluto And the answer was yes. I do not know the exact system requirement of the current ChatGPT, but I am fairly confident 1) ChatGPT no network mode could fit in a half of a server rack, maybe way less. 2) You can fit half a server work on a spaceship that can go to Pluto. My guess is it's more like 1 server worth. Google tells me GPT3 was 1TB which is a very small laptop.
      - giancarlostoro 3y ago
        Is it moving a goal post if I'm just going off on a tangent from one simple remark? I'm just going on a tangent of something I find fascinating as a thought experiment, what will things look like when we can have isolated AIs in our pockets without needing connectivity? The reason I want this is because you could leave your house for a long roadtrip, and have an AI companion that never loses a step. Imagine if your AI had every road memorized and nuances about them, so if GPS signal dies, you can still keep going. Sidenote, ChatGPT uses tens of thousands of GPUs to run its architecture, I think it'll take a little more than just some laptop. Thank you for answering though :) I do think its an invaluable goal to have off-line first AI.
janehdoch 3y ago
I think that we still need a few years 3-5 alone for better models, different architectures and more finetuning. Those models will affect us more than today already and change how we perceive AI. Than we will start to see AI optimized hardware (much more optimized). And than perhaps in 10 years we all run a lot more models locally. Nonetheless or despite this, the normal consumer doesn't run open models and will probably not do that for a very long time. Searching, keeping up-to-date and running models is still effort and the usage model makes a ton of sense. Escpecially in time of SaaS. Im not running wikipedia locally. And none of my social circle operates infrastructure / server. People just want to use it. Besides that, whatever local models or open models will be able to do, AIaaS will have faster models, better models and more convinient models. I'm just waiting to pay for google assistent if it becomes smart and can manage my emails my calendar and everything else. After all my gmail account already has access (through email and password reset) to most services i use. I'm more curiuos when we will see AI service integration through much more system to system communication. Machine friendly apis (which partially already exist anyway) PS: Look at how fast hardware development currently is. Not much change in Memory etc. Models will not just become 100x smaller in just a few years. We are right now at optimizing those models to be cost efficient. Alone this phase will take a few years.
artursapek 3y ago
Isn't this the revenue model for most cloud computing, like EC2 (compute) S3 (storage) etc?
heavyset_go 3y ago
> I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. The models themselves will be hoarded as IP. Doesn't matter if they're in the cloud or on devices, they'll be licensed like commercial proprietary software with the same restrictions commercial software has.
- whiddershins 3y ago
  Yeah but that isn’t t pay - per - operation I mean I guess my electricity provider gets paid per compute.
  - heavyset_go 3y ago
    You can write software that is pay per use, and you can do the same with local models.
- api 3y ago
  This will be true as long as they are crazy expensive to produce. Eventually the cost of training a base model could drop to the range where crowdfunding could do it. Or alternately someone could make a major advance in distributed training and we could all contribute cycles in a distributed effort like Folding@Home. As it stands training requires far too much bandwidth for synchronization and moving model data around. Some approach to sharding training would have to be discovered. It’s an open problem area. Neural networks are very parallelizable and training is stochastic so my intuition is that it should be possible. Even if it were less efficient than synchronous training you could make up for that by harnessing 100X the compute from a huge crowd.
  - heavyset_go 3y ago
    Platforms and rightsholders know what they're sitting on now, and I worry that the datasets required to train sufficient models in the future will also be hoarded as IP. It's one thing to train on Common Crawl in 2023, but what about when you have to shell out millions of dollars just for access to data sets to train on in the future? Same thing with human reinforcement. The customers for both are willing to pay much more than a crowdfunding campaign would. Training is expensive now, but data sets can be expensive in the future.
api 3y ago
I run stable diffusion XL with control nets on a laptop, albeit a high end one. It’s pretty decent. I also run LLMs such as trains of llama2, though LLMs on commodity hardware are not as “there” yet as image generators. It’s a decent question and answer bot and summarizer but isn’t GPT-4 level. I could see another iteration approaching that but I’d probably need more RAM.
Closi 3y ago
I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon. They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally. Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised? There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).
- baz00 3y ago
  You can run this on an M2 Mac Studio. That’ll be consumer level hardware in a few years.
  - Closi 3y ago
    Run what? Adobe's offering is cloud-only, and I don't think the hardware requirements are disclosed.
    - unixhero 3y ago
      The models obviously. Thsse are for the most part available open source on Github
      - ShrigmaMale 3y ago
        Firefly models are not publicly available. The integration with creative cloud apps is not customizable. You can't point it to a local stablediffusion setup. Open-source competitors will flop just like GIMP.
        nickthegreek 3y ago
        Fyi, there are Adobe plugins that do allow y you to use a local SD install.
financypants 3y ago
Sure we will be happily running these on customer level hardware, but there will always be a stronger version in the cloud “worth paying for,” won’t there?
jowea 3y ago
But when we will be able to run the current models on customer level hardware, won't the models available from cloud hardware so much better we would rather pay for the cloud?
summerlight 3y ago
We have lots of legal uncertainties on the model training side and we don't know what's the conclusion yet. If most of those "open source" models deem to be problematic with copyrights or whatsoever, then it might make more sense to use proprietary models for commercial use cases since many of those use cases are not really critical (e.g. replacement of stock images) but giving them lots of potential headaches with legal complications.
rebolek 3y ago
The only thing they care about in long term is greed.
dgb23 3y ago
The trend for big software is to charge more and more, while locking you in. Payments become more granular and gradual rather than predictable and discrete. Are the technical requirements driving these monetization schemes or is it the other way around?
cratermoon 3y ago
I saw a Mastodon thread by Paul Cantrell that equated CEOs and AI with children and the year's hot Christmas toy. Must have and have it NOW, not after Christmas when we find out if kids sustain interest in playing with it for past New Years. https://hachyderm.io/@inthehands/111075245991998920 https://hachyderm.io/@inthehands/111075245991998920
- fragmede 3y ago
  How infantilizing. For those who have been able to make it work for them, it's already a load-bearing piece of infrastructure. How do CEOs and children see the new bridge on their commute to work/school?
whywhywhywhy 3y ago
>I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. Plenty of us already are. SDXL is as good as anything in the cloud.
coldtea 3y ago
>I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. On the contrary. In a few years there wont be a lot of customer level hardware software (especially business software) without a subscription.
Postosuchus 3y ago
This is absolutely not about hardware (or even software). It is about indemnification from copyright infringement claims. Adobe's main selling point for Firefly (or whatever it is called) is that is trained on Adobe's data.
faeriechangling 3y ago
Running models locally is a moving target. By the time we have GPT3.5 quality at home we'll have better models in the cloud. I'm sure the workloads will shift locally more and more, if for no better reasons than latency and privacy.

gochi 3y ago

They can upgrade hardware to bleeding edge faster than consumers can get their hands on remotely comparable products. Most consumers also don't upgrade every year to stay on bleeding edge, they just tolerate what they can currently do. While this will be fine for most, those actually using generative AI for work aren't likely going to tolerate stagnation as easily.

coder543 3y ago

I would have said the same thing about CAD simulations, but then Autodesk decided to move backwards and remove an existing feature just so they could charge for per-simulation credits: https://hackaday.com/2022/08/12/local-simulation-feature-to-be-removed-from-all-autodesk-fusion-360-versions/ https://hackaday.com/2022/08/12/local-simulation-feature-to-... Whether Adobe ever decides to let their model run locally or lock it forever into the cloud is a choice they will have to make. A lot of people trust Adobe products, so it's entirely conceivable that some people will always choose to pay for a pay-per-use generative solution from Adobe rather than try to run competing solutions locally. The question is probably whether it generates more revenue than negativity for Adobe. If most Adobe users are running their own models locally and avoiding the feature, then I think Adobe will be more likely to follow suit and move away from the pay-per-use cloud approach.

flangola7 3y ago
Long term they may offer both, with the local model consuming fewer credits per request. The new GPUs all have secure enclave remote attestation architecture, so Adobe would be able to offer this without the risk of someone jailbreaking the model and running it for free.
- fragmede 3y ago
  Assuming that their secure enclave is, in fact, unhackable, that is.
  - flangola7 3y ago
    The Xbox One has not been hacked in the 10 years since its release. Apple already uses the same kind of tech to protect the various AI models in iPhones and MacBooks, for tasks like OCR search of stored image files and to auto-cutout people in photos. Enclaves are rarely broken and the people that can are selling it to the CIA, not leaking it on the pirate bay.
- tavavex 3y ago
  What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so. It'd be like charging per use of any other advanced features in locally-run Adobe software.
  - taneq 3y ago
    Yeah, that’d be as crazy as charging car owners a subscription fee to use their heated seats.
  - flangola7 3y ago
    > What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so. To pay for the privilege of using a very advanced AI model. That's more reasonable than paying to unlock a game character skin that's already on your SSD, and that happens millions of times a day. > It'd be like charging per use of any other advanced features in locally-run Adobe software. I don't see anything stopping this.
  - coldtea 3y ago
    >It'd be like charging per use of any other advanced features in locally-run Adobe software. Software companies already do that. There are all kinds of locally-run advanced features that are only enabled with a more expensive subscription tier even though you already have the code and assets for them. Sure, most are merely subscription based, but there are others that are per use.
xmprt 3y ago
> A lot of people trust Adobe products I would have said the consumer sentiment amongst Adobe users is the exact opposite - that people don't trust Adobe products but they use them because they either have to or because they're currently the best products available.
gmerc 3y ago
This hinges on having a quasi monopoly.

amelius 3y ago

> on customer level hardware Unless nVidia changes their monetization model, and for example introduces an App Store for AI, with subscriptions, of course on locked down hardware.

code51 3y ago

Adobe is planning for a post-regulation, post-biggest-lawsuit world for sure. All their steps show that they'll base their offering on their own commercial data - paying training license fees to stock images and artists. Whether the amounts they pay would make licensing your work sensible or not, Adobe is surely assuming this will ultimately end up as Napster-to-Spotify transition. If we end up "happily" (means legally as well) running these on customer level hardware, then the question won't be about credits of computation. It'll be about credits to use licensed work.

sebmellen 3y ago
This is the most plausible reason from what I see.
GaggiX 3y ago
> It'll be about credits to use licensed work. If this is true (which I kinda doubt), is it going to matter to most people? Like you can't really tell the images used to train a model from the images it generates (if it's trained correctly), so I doubt the majority of people would care, like those who already use MJ for example. Training models on copyrighted data for academic research will be allowed, the models will be published, and good luck enforcing the licence; and here I'm talking about the worst case scenario where a court would find an image generated by AI to be derivative of another image in a pool of billions in a dataset (this goes way beyond any definition of derivative work for now).
- gmerc 3y ago
  Ya, enforcement would have to go along with a war on general purpose computing and privacy. Oh wait, …
whiddershins 3y ago
That’s a big bet. My instinct is it doesn’t go that way. We will see.

yieldcrv 3y ago

I agree, these models are going to get smaller and more performant, the software to leverage your hardware is going to have major improvements, the hardware is going to change to prioritize processing these with more memory to specific coprocessors, and the OS is going to start having these models baked into them, with improved default models being a core feature of upgrading each OS version people are looking at an extremely limited view of “bigger models on better hardware will always be in the cloud” when that reality simply won’t matter for most use cases

YurgenJurgensen 3y ago

Wouldn't this require a complete reversal of course of, like, the last fifteen years of computing? Basically everyone wants to get out of selling products and into selling services.

thorum 3y ago

When we reach the point when average people can easily run today’s models on their own devices, today’s models will no longer be SOTA and there will still be demand to run better models in the cloud.

brucethemoose2 3y ago
Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL. The tooling around them is far more important. However, Stable diffusion already can run on mobile devices. There is already a good iOS app for it (and the dev is here on HN) but the problem seems to be that no one cares. There are 700,000 cloud imagegen apps crowding it out, because thats what's easier and more profitable to spam across the store and web.
- thorum 3y ago
  > Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL. For image quality, sure - language understanding is still an issue. SDXL can generate a beautiful image, but if it doesn’t show exactly what you asked for in the prompt, on the first try, there is still room for improvement. The gap between LLMs and image generators in this regard is huge.
- Closi 3y ago
  Depends if the limit on parameter count is a 'real' limit, or just a limit based on what current-technology models can effectively use. Back in the 'Google Daydream' days, Google might have found that they didn't get any more image-generation performance by raising the parameter count - but that's just because the technology at the time couldn't effectively utilise more parameters. It's impossible to know what next-gen models might be able to use, but I suspect we will find ways to allow the models to take advantage of even higher parameter counts. Stable diffusion can run on mobile devices, but it's painful and image generation takes a fraction of the time via cloud services.

GaggiX 3y ago

We already run these on customer level hardware, you need a good PC but still, in the future the pool of people that can do is just going to be bigger.

bhouston 3y ago

> I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. I doubt that is what Adobe will do. This is a new revenue stream for them, why would they remove it? Gimp will use local generation but Adobe is using a proprietary dataset that they can keep secure in the cloud. So yeah this is going to be sticking around.

giancarlostoro 3y ago

> we will be happily running these on customer level hardware. This is the stage of AI that will impress me most. If I can use your AI completely offline on my device on a spaceship orbiting Pluto, then I will say we have achieved an AI capacity that is impressive, even if its got the quirks of chatgpt today.

brianwawok 3y ago
We can already do that. But why give away the keys to a money machine?
- giancarlostoro 3y ago
  So you can fit all of ChatGPT into a 32GB storage phone? Because my understanding is something like ChatGPT takes terrabytes upon terrabytes of data on-disk and I can't imagine how many servers being piped together to make it churn out coherent sentences... I'm on about all of that on a phone that doesn't cost thousands of dollars. It would mean we have finally reached significant technological advances.
  - brianwawok 3y ago
    Okay this is called a moving goalpost. This was my response to > If I can use your AI completely offline on my device on a spaceship orbiting Pluto And the answer was yes. I do not know the exact system requirement of the current ChatGPT, but I am fairly confident 1) ChatGPT no network mode could fit in a half of a server rack, maybe way less. 2) You can fit half a server work on a spaceship that can go to Pluto. My guess is it's more like 1 server worth. Google tells me GPT3 was 1TB which is a very small laptop.
    - giancarlostoro 3y ago
      Is it moving a goal post if I'm just going off on a tangent from one simple remark? I'm just going on a tangent of something I find fascinating as a thought experiment, what will things look like when we can have isolated AIs in our pockets without needing connectivity? The reason I want this is because you could leave your house for a long roadtrip, and have an AI companion that never loses a step. Imagine if your AI had every road memorized and nuances about them, so if GPS signal dies, you can still keep going. Sidenote, ChatGPT uses tens of thousands of GPUs to run its architecture, I think it'll take a little more than just some laptop. Thank you for answering though :) I do think its an invaluable goal to have off-line first AI.

janehdoch 3y ago

I think that we still need a few years 3-5 alone for better models, different architectures and more finetuning. Those models will affect us more than today already and change how we perceive AI. Than we will start to see AI optimized hardware (much more optimized). And than perhaps in 10 years we all run a lot more models locally. Nonetheless or despite this, the normal consumer doesn't run open models and will probably not do that for a very long time. Searching, keeping up-to-date and running models is still effort and the usage model makes a ton of sense. Escpecially in time of SaaS. Im not running wikipedia locally. And none of my social circle operates infrastructure / server. People just want to use it. Besides that, whatever local models or open models will be able to do, AIaaS will have faster models, better models and more convinient models. I'm just waiting to pay for google assistent if it becomes smart and can manage my emails my calendar and everything else. After all my gmail account already has access (through email and password reset) to most services i use. I'm more curiuos when we will see AI service integration through much more system to system communication. Machine friendly apis (which partially already exist anyway) PS: Look at how fast hardware development currently is. Not much change in Memory etc. Models will not just become 100x smaller in just a few years. We are right now at optimizing those models to be cost efficient. Alone this phase will take a few years.

artursapek 3y ago

Isn't this the revenue model for most cloud computing, like EC2 (compute) S3 (storage) etc?

heavyset_go 3y ago

> I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. The models themselves will be hoarded as IP. Doesn't matter if they're in the cloud or on devices, they'll be licensed like commercial proprietary software with the same restrictions commercial software has.

whiddershins 3y ago
Yeah but that isn’t t pay - per - operation I mean I guess my electricity provider gets paid per compute.
- heavyset_go 3y ago
  You can write software that is pay per use, and you can do the same with local models.
api 3y ago
This will be true as long as they are crazy expensive to produce. Eventually the cost of training a base model could drop to the range where crowdfunding could do it. Or alternately someone could make a major advance in distributed training and we could all contribute cycles in a distributed effort like Folding@Home. As it stands training requires far too much bandwidth for synchronization and moving model data around. Some approach to sharding training would have to be discovered. It’s an open problem area. Neural networks are very parallelizable and training is stochastic so my intuition is that it should be possible. Even if it were less efficient than synchronous training you could make up for that by harnessing 100X the compute from a huge crowd.
- heavyset_go 3y ago
  Platforms and rightsholders know what they're sitting on now, and I worry that the datasets required to train sufficient models in the future will also be hoarded as IP. It's one thing to train on Common Crawl in 2023, but what about when you have to shell out millions of dollars just for access to data sets to train on in the future? Same thing with human reinforcement. The customers for both are willing to pay much more than a crowdfunding campaign would. Training is expensive now, but data sets can be expensive in the future.

api 3y ago

I run stable diffusion XL with control nets on a laptop, albeit a high end one. It’s pretty decent. I also run LLMs such as trains of llama2, though LLMs on commodity hardware are not as “there” yet as image generators. It’s a decent question and answer bot and summarizer but isn’t GPT-4 level. I could see another iteration approaching that but I’d probably need more RAM.

Closi 3y ago

I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon. They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally. Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised? There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).

baz00 3y ago
You can run this on an M2 Mac Studio. That’ll be consumer level hardware in a few years.
- Closi 3y ago
  Run what? Adobe's offering is cloud-only, and I don't think the hardware requirements are disclosed.
  - unixhero 3y ago
    The models obviously. Thsse are for the most part available open source on Github
    - ShrigmaMale 3y ago
      Firefly models are not publicly available. The integration with creative cloud apps is not customizable. You can't point it to a local stablediffusion setup. Open-source competitors will flop just like GIMP.
      - nickthegreek 3y ago
        Fyi, there are Adobe plugins that do allow y you to use a local SD install.

financypants 3y ago

Sure we will be happily running these on customer level hardware, but there will always be a stronger version in the cloud “worth paying for,” won’t there?

jowea 3y ago

But when we will be able to run the current models on customer level hardware, won't the models available from cloud hardware so much better we would rather pay for the cloud?

summerlight 3y ago

We have lots of legal uncertainties on the model training side and we don't know what's the conclusion yet. If most of those "open source" models deem to be problematic with copyrights or whatsoever, then it might make more sense to use proprietary models for commercial use cases since many of those use cases are not really critical (e.g. replacement of stock images) but giving them lots of potential headaches with legal complications.

rebolek 3y ago

The only thing they care about in long term is greed.

dgb23 3y ago

The trend for big software is to charge more and more, while locking you in. Payments become more granular and gradual rather than predictable and discrete. Are the technical requirements driving these monetization schemes or is it the other way around?

cratermoon 3y ago

I saw a Mastodon thread by Paul Cantrell that equated CEOs and AI with children and the year's hot Christmas toy. Must have and have it NOW, not after Christmas when we find out if kids sustain interest in playing with it for past New Years. https://hachyderm.io/@inthehands/111075245991998920 https://hachyderm.io/@inthehands/111075245991998920

fragmede 3y ago
How infantilizing. For those who have been able to make it work for them, it's already a load-bearing piece of infrastructure. How do CEOs and children see the new bridge on their commute to work/school?

whywhywhywhy 3y ago

>I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. Plenty of us already are. SDXL is as good as anything in the cloud.

coldtea 3y ago

>I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware. On the contrary. In a few years there wont be a lot of customer level hardware software (especially business software) without a subscription.

Postosuchus 3y ago

This is absolutely not about hardware (or even software). It is about indemnification from copyright infringement claims. Adobe's main selling point for Firefly (or whatever it is called) is that is trained on Adobe's data.

faeriechangling 3y ago

Running models locally is a moving target. By the time we have GPT3.5 quality at home we'll have better models in the cloud. I'm sure the workloads will shift locally more and more, if for no better reasons than latency and privacy.