7 ms·

Have to assume they literally can't get enough GPUs to respond to demand. Wonder if $20/mo is even profitable for them?

bloppe 4y ago
There's no way it's profitable. I'd wager this will materially affect Microsoft's next earnings report. Hard to tell how open ai itself is affected financially by the explosive growth but a rude awakening is coming for many users and shareholders alike
- unmole 4y ago
  > wager this will materially affect Microsoft's next earnings report How?
  - bloppe 4y ago
    They're footing the bill for this whole operation. They provide the compute and storage and they're also so heavily invested in OpenAI after their recent $10B round that they can't offload the costs. See my other comment on this post for a conservative cost estimate for each query.
    - chaxor 4y ago
      It's not necessarily a bad move to get them back in the game after many years of being useless. Deepmind is one of only a few thin threads that may hold Google together in the coming years, but it really doesn't look good for them
    - Rastonbury 4y ago
      It won't show up on their financials I reckon, they made equity investment into OpenAI - their P&L won't show on Microsoft's. If they have a sweetheart deal with MSFT, Azure revenues aren't going to change. Even if they built a new DC or bought a boatload of GPUs for OpenAI, it will get mixed into their massive Capex and they probably are using existing infrastructure
      - unmole 4y ago
        They apparently got a lot of Azure credits and Microsoft isn't making any revenue from running OpenAI's workloads.
        dekhn 4y ago
        The value to microsoft is in the marketing around a premier training company using their hardware... their actual hardware in this case is a bespoke supercomputer, but performance watchers like me are paying attention. From what I can tell, of the top 3 cloud providers, azure is doing the best work with high speed interconnects right now.
        htss2013 4y ago
        But what are the workloads costing MSFT? If the credits are part of the acquisition deal, maybe they're considered capex not opex, and MSFT gets to amortize the credits' operating cost over decades. What is the true marginal cost to MSFT for gpt workloads anyway? As long as it's not bumping paying customers then it's what the cost of electricity?
        dereg 4y ago
        Creating credits out of thin air is impossible as all financial transactions must be accounted for. In the case of Azure credits, they will appear as increased revenue on Microsoft's balance sheet. However, if Microsoft decides to price Azure services below cost as part of their agreement with OpenAI, their profit margins will be affected. I highly doubt that is the case because the alleviation of cost is in the form of credits. I would wager against comment 0 that said that this will materially affect earnings.
        rvz 4y ago
        It will, hence the MSFT layoffs happening at the same time. People here wanted Google scale of operations and replacing Google with ChatGPT as their search engine. But it seems that some here have forgotten about the high availability requirements and incalculable operational costs of running a search engine. Google.com's availability is a very high barrier of serving in the multi-billions of searches a day. Azure has lots of experience in unreliability and running services at a massive loss whilst minting credits to compete on cost.
- ralph84 4y ago
  At the same time MSFT is doing layoffs. GPT is already eliminating jobs.
rany_ 4y ago
It probably is. I'd bet for 99.99% of their users, simply using the API directly would be much cheaper. The ChatGPT premium subscription is overpriced if you compare it to the cost of each API call.. at least according to OpenAI's pricing.
- theshrike79 4y ago
  Yeah, I'm waiting for one more month for the API-bots to mature. Then I'll cancel my ChatGPT premium subscription and put up a private Discord server just for myself and a GPT4 bot. And save a ton of money at the same time.
- jerkstate 4y ago
  It’s pretty easy to spend a couple of dollars with a few interactions with the API. I tried one of those langchain type agents and it spent $2 in 10 minutes. My guess is they make more money per user from the API than they do from ChatGPT. I have found Bing and Bard a lot more efficient for tasks where you need to look stuff up on the web, probably because they both have local access to a crawler index. They do tend to need more guidance though.
- mike_hearn 4y ago
  Someone on Reddit claimed they attended a talk by Sam Altman where he said Plus pays for its own compute but that's it.
ChatGTP 4y ago
It's interesting to think about it this way. I'm sure serious competitors are just around the corner, at least Bard will catch up, No doubt Apple has something in the works, there's little to zero doubt about that. So do they make a huge capital investment hoping that growth will continue at this trajectory, or do they back off the gas pedal ? I bet there's also some contentious ideas going on internally too. If they go on a huge hiring spree, does that kind of invalidate the product which is supposed to replace everyone? How do the optics look there? Do they hire and fire once GPT-5 becomes a sentient AGI? Google already has a huge engineering team, operates are unprecedented scale, and IMO has a much more diverse set of products to integrate with, they also solve a lot of novel solutions which they require specialized engineers that can't be replaced easily by "AI". Be interesting to see how it plays out. Maybe a victim of their own success ?
- Rastonbury 4y ago
  The cynic in me sees they are making a vendor lock-in play, the API is dirt cheap so much so that every other bespoke chatbot or language model is now obsolete - every chat startup is now switching to OpenAI. Once everyone has their prompts and embeddings tuned and is serving customers, OpenAI can dial up the price and customers chose to keep paying or redo their prompts/embeddings to switch to Bard or the next competitor. It does seem like business as usual if you relied on GPT-3 now ChatGPT, then whatever iteration comes next, probably won't work the same, so its not a heavy heavy lift. Decent platform risk, I believe OpenAI shutdown Codex so now everyone who had a product based on Codex has to switch to ChatGPT
  - ChatGTP 4y ago
    I don’t know a lot about codex, but was this a more standard interface used by other providers ?
    - Rastonbury 4y ago
      It was their model for code and what was used for Github copilot, was also used by Replit

There's no way it's profitable. I'd wager this will materially affect Microsoft's next earnings report. Hard to tell how open ai itself is affected financially by the explosive growth but a rude awakening is coming for many users and shareholders alike

unmole 4y ago
> wager this will materially affect Microsoft's next earnings report How?
- bloppe 4y ago
  They're footing the bill for this whole operation. They provide the compute and storage and they're also so heavily invested in OpenAI after their recent $10B round that they can't offload the costs. See my other comment on this post for a conservative cost estimate for each query.
  - chaxor 4y ago
    It's not necessarily a bad move to get them back in the game after many years of being useless. Deepmind is one of only a few thin threads that may hold Google together in the coming years, but it really doesn't look good for them
  - Rastonbury 4y ago
    It won't show up on their financials I reckon, they made equity investment into OpenAI - their P&L won't show on Microsoft's. If they have a sweetheart deal with MSFT, Azure revenues aren't going to change. Even if they built a new DC or bought a boatload of GPUs for OpenAI, it will get mixed into their massive Capex and they probably are using existing infrastructure
    - unmole 4y ago
      They apparently got a lot of Azure credits and Microsoft isn't making any revenue from running OpenAI's workloads.
      - dekhn 4y ago
        The value to microsoft is in the marketing around a premier training company using their hardware... their actual hardware in this case is a bespoke supercomputer, but performance watchers like me are paying attention. From what I can tell, of the top 3 cloud providers, azure is doing the best work with high speed interconnects right now.
      - htss2013 4y ago
        But what are the workloads costing MSFT? If the credits are part of the acquisition deal, maybe they're considered capex not opex, and MSFT gets to amortize the credits' operating cost over decades. What is the true marginal cost to MSFT for gpt workloads anyway? As long as it's not bumping paying customers then it's what the cost of electricity?
      - dereg 4y ago
        Creating credits out of thin air is impossible as all financial transactions must be accounted for. In the case of Azure credits, they will appear as increased revenue on Microsoft's balance sheet. However, if Microsoft decides to price Azure services below cost as part of their agreement with OpenAI, their profit margins will be affected. I highly doubt that is the case because the alleviation of cost is in the form of credits. I would wager against comment 0 that said that this will materially affect earnings.
        rvz 4y ago
        It will, hence the MSFT layoffs happening at the same time. People here wanted Google scale of operations and replacing Google with ChatGPT as their search engine. But it seems that some here have forgotten about the high availability requirements and incalculable operational costs of running a search engine. Google.com's availability is a very high barrier of serving in the multi-billions of searches a day. Azure has lots of experience in unreliability and running services at a massive loss whilst minting credits to compete on cost.
ralph84 4y ago
At the same time MSFT is doing layoffs. GPT is already eliminating jobs.

rany_ 4y ago

It probably is. I'd bet for 99.99% of their users, simply using the API directly would be much cheaper. The ChatGPT premium subscription is overpriced if you compare it to the cost of each API call.. at least according to OpenAI's pricing.

theshrike79 4y ago
Yeah, I'm waiting for one more month for the API-bots to mature. Then I'll cancel my ChatGPT premium subscription and put up a private Discord server just for myself and a GPT4 bot. And save a ton of money at the same time.
jerkstate 4y ago
It’s pretty easy to spend a couple of dollars with a few interactions with the API. I tried one of those langchain type agents and it spent $2 in 10 minutes. My guess is they make more money per user from the API than they do from ChatGPT. I have found Bing and Bard a lot more efficient for tasks where you need to look stuff up on the web, probably because they both have local access to a crawler index. They do tend to need more guidance though.
mike_hearn 4y ago
Someone on Reddit claimed they attended a talk by Sam Altman where he said Plus pays for its own compute but that's it.

ChatGTP 4y ago

It's interesting to think about it this way. I'm sure serious competitors are just around the corner, at least Bard will catch up, No doubt Apple has something in the works, there's little to zero doubt about that. So do they make a huge capital investment hoping that growth will continue at this trajectory, or do they back off the gas pedal ? I bet there's also some contentious ideas going on internally too. If they go on a huge hiring spree, does that kind of invalidate the product which is supposed to replace everyone? How do the optics look there? Do they hire and fire once GPT-5 becomes a sentient AGI? Google already has a huge engineering team, operates are unprecedented scale, and IMO has a much more diverse set of products to integrate with, they also solve a lot of novel solutions which they require specialized engineers that can't be replaced easily by "AI". Be interesting to see how it plays out. Maybe a victim of their own success ?

Rastonbury 4y ago
The cynic in me sees they are making a vendor lock-in play, the API is dirt cheap so much so that every other bespoke chatbot or language model is now obsolete - every chat startup is now switching to OpenAI. Once everyone has their prompts and embeddings tuned and is serving customers, OpenAI can dial up the price and customers chose to keep paying or redo their prompts/embeddings to switch to Bard or the next competitor. It does seem like business as usual if you relied on GPT-3 now ChatGPT, then whatever iteration comes next, probably won't work the same, so its not a heavy heavy lift. Decent platform risk, I believe OpenAI shutdown Codex so now everyone who had a product based on Codex has to switch to ChatGPT
- ChatGTP 4y ago
  I don’t know a lot about codex, but was this a more standard interface used by other providers ?
  - Rastonbury 4y ago
    It was their model for code and what was used for Github copilot, was also used by Replit