4 ms·

I suspect many companies will do something like this - prepaid credits or tokens for AI features that have high inference costs. Inference costs per user are hi

by tikkun 3y ago

I suspect many companies will do something like this - prepaid credits or tokens for AI features that have high inference costs. Inference costs per user are high, at least much higher than most traditional software costs. This way, the costs are aligned with the usage. We’ll also see occasional subscription products, but only when it can be done in a way that is comfortably gross margin profitable for most users of a company. (Eg ChatGPT Plus, Claude Pro, Midjourney, 365 Copilot) This will only change when the cost of inference goes down by a lot.

vunderba 3y ago
I mean this is how DALL-E works, and they had a pretty good lock on the market until Stable diffusion came out and we got things like Midjourney.
redox99 3y ago
> Inference costs per user are high, at least much higher than most traditional software costs. This way, the costs are aligned with the usage. Meh, the price if MidJourney is like 2 orders of magnitude higher than their inference costs. The pricing is based on what they can get away with.
- gallabytes 3y ago
  this is not even close to true
  - redox99 3y ago
    Yeah, I'm sorry for the incorrect statement. MidJourney claims 3.3 hours of GPU for $10. If that's 3.3 hours of A100, at a rate of $2/hour that's $6.6 of GPU costs. My original statement was based on MJ feeling very expensive per image compared to the huge number of images I can generate in an hour with SDXL on my 3090 (and 3090s can be rented for $0.20 an hour). But I forgot how overpriced A100s are (I doubt MJ is running 3090s but that'd be pretty cool), and that MJ is probably 4x the size of SDXL (although surely more optimized). My revised statement is that for their $10 plan there's a few bucks of GPU compute cost. Probably between 2x and 5x margins.
    - joefourier 3y ago
      A100s aren’t $2/h at MJ scale tho. Firstly they likely own quite a few and they’re relatively cheap now at $4k a card, and secondly you can get much better deals than $2/h if you rent a large amount and pre-pay.
      - redox99 3y ago
        Sure, I was establishing a upper and lower bound. That's why I said between 2x and 5x margins.

by tikkun 3y ago

vunderba 3y ago

I mean this is how DALL-E works, and they had a pretty good lock on the market until Stable diffusion came out and we got things like Midjourney.

redox99 3y ago

> Inference costs per user are high, at least much higher than most traditional software costs. This way, the costs are aligned with the usage. Meh, the price if MidJourney is like 2 orders of magnitude higher than their inference costs. The pricing is based on what they can get away with.

gallabytes 3y ago
this is not even close to true
- redox99 3y ago
  Yeah, I'm sorry for the incorrect statement. MidJourney claims 3.3 hours of GPU for $10. If that's 3.3 hours of A100, at a rate of $2/hour that's $6.6 of GPU costs. My original statement was based on MJ feeling very expensive per image compared to the huge number of images I can generate in an hour with SDXL on my 3090 (and 3090s can be rented for $0.20 an hour). But I forgot how overpriced A100s are (I doubt MJ is running 3090s but that'd be pretty cool), and that MJ is probably 4x the size of SDXL (although surely more optimized). My revised statement is that for their $10 plan there's a few bucks of GPU compute cost. Probably between 2x and 5x margins.
  - joefourier 3y ago
    A100s aren’t $2/h at MJ scale tho. Firstly they likely own quite a few and they’re relatively cheap now at $4k a card, and secondly you can get much better deals than $2/h if you rent a large amount and pre-pay.
    - redox99 3y ago
      Sure, I was establishing a upper and lower bound. That's why I said between 2x and 5x margins.