4 ms·

at their scale they could also just run a large on-premise or rented (basically still cloud, but cheaper) GPU cluster and run through that. fixed costs, even li

by dkdcdev 4mo ago

at their scale they could also just run a large on-premise or rented (basically still cloud, but cheaper) GPU cluster and run through that. fixed costs, even license a SOTA model’s weights if you’d like

embedding-shape 4mo ago
> even license a SOTA model’s weights if you’d like Yeah, I bet all labs releasing SOTA models are more than happy to remove the main way they make money and let you run it locally, especially if you're a big spender like Uber who seems very willing to throw money into the sea as an experiment.
- throwway120385 4mo ago
  That's going to stop eventually, and I think at that point we're going to see business models more like the major CAD providers.
- deleted 4mo ago
  [deleted]
- idiotsecant 4mo ago
  I don't think they'll have a choice, open weights models are not far behind. At some point it's essentially a commodity game
  - dkdcdev 4mo ago
    they also already do this… Anthropic and OpenAI license to the public clouds. Google reportedly licenses to Apple. licensing to Fortune 100 companies running on their own infra is an obvious next step it is a race to the bottom and I’m not sure the labs win that race. we’ll see!
    - thewebguyd 4mo ago
      I'm not sure the labs will win either. I wouldn't be surprised to see OpenAI & Anthropic just get acquired, either by Microsoft or Amazon and their models just become another product offering in their public cloud and and some hybrid on-prem offering like Azure Stack HCI or Azure Stack Hub (already basically a "cloud in a black box" that could become "AI in a box")
mrweasel 4mo ago
The problem isn't really Uber, Microsoft or Nvidia, it's all the smaller none IT companies that also have developers on staff. They are screwed. $1500 per seat per month is just way to expensive, but they also can't afford to build and maintain their own on-premise solution. If Microsoft can't afford to run CoPilot for their own developer, what chance does any of their customers stand? If the large, well founded IT companies in the world believes the current AI cost is to high, then Anthropic, OpenAI and CoPilot have no actual customer base. AI is then relegated to very profitable niche business, but that can't fund the R&D for the models.
- skybrian 4mo ago
  It's an extra 18k a year for developer tools when they're paying how much a year per developer? Having software developers at all isn't cheap. Also, I don't believe you need to spend $1500 a month on a coding agent if you optimize usage at all.
  - ecshafer 4mo ago
    $18k a year is a non starter in most companies. Ive seen companies balk at Intellij.
  - mrweasel 4mo ago
    That depends on where you are. $18K is the equivalent of paying around 15% more for your developer.
    - ricardobayes 4mo ago
      In hcol locations yes, but in south of spain you can get full time talent for that figure. It's also an entry-level salary in eastern europe, with ukraine and turkey even being somewhat cheaper.
  - KronisLV 4mo ago
    In Latvia, the net salary for a Java dev is around 1729 - 4314 EUR, based on https://www.algas.lv/algu-informacija/informacijas-tehnologijas/java-programmetajs https://www.algas.lv/algu-informacija/informacijas-tehnologi... (crowd sourced data) For the employer those employees cost between 2945 - 7736 EUR per month based on https://kalkulatori.lv/lv/algas-kalkulators https://kalkulatori.lv/lv/algas-kalkulators (income and social taxes). So on the lower end that's (1500 USD ~ 1300 EUR) close to half the total expenses of such a developer, on the high end here around 15-20%. That's quite significant, depends on whether their productivity also improves (if that's what the orgs care about). And we’re not even the country with the worst pay out there, but pay the same for tokens, cause regional pricing isn’t a thing!

by dkdcdev 4mo ago

embedding-shape 4mo ago

> even license a SOTA model’s weights if you’d like Yeah, I bet all labs releasing SOTA models are more than happy to remove the main way they make money and let you run it locally, especially if you're a big spender like Uber who seems very willing to throw money into the sea as an experiment.

throwway120385 4mo ago
That's going to stop eventually, and I think at that point we're going to see business models more like the major CAD providers.
deleted 4mo ago
[deleted]
idiotsecant 4mo ago
I don't think they'll have a choice, open weights models are not far behind. At some point it's essentially a commodity game
- dkdcdev 4mo ago
  they also already do this… Anthropic and OpenAI license to the public clouds. Google reportedly licenses to Apple. licensing to Fortune 100 companies running on their own infra is an obvious next step it is a race to the bottom and I’m not sure the labs win that race. we’ll see!
  - thewebguyd 4mo ago
    I'm not sure the labs will win either. I wouldn't be surprised to see OpenAI & Anthropic just get acquired, either by Microsoft or Amazon and their models just become another product offering in their public cloud and and some hybrid on-prem offering like Azure Stack HCI or Azure Stack Hub (already basically a "cloud in a black box" that could become "AI in a box")

mrweasel 4mo ago

The problem isn't really Uber, Microsoft or Nvidia, it's all the smaller none IT companies that also have developers on staff. They are screwed. $1500 per seat per month is just way to expensive, but they also can't afford to build and maintain their own on-premise solution. If Microsoft can't afford to run CoPilot for their own developer, what chance does any of their customers stand? If the large, well founded IT companies in the world believes the current AI cost is to high, then Anthropic, OpenAI and CoPilot have no actual customer base. AI is then relegated to very profitable niche business, but that can't fund the R&D for the models.

skybrian 4mo ago
It's an extra 18k a year for developer tools when they're paying how much a year per developer? Having software developers at all isn't cheap. Also, I don't believe you need to spend $1500 a month on a coding agent if you optimize usage at all.
- ecshafer 4mo ago
  $18k a year is a non starter in most companies. Ive seen companies balk at Intellij.
- mrweasel 4mo ago
  That depends on where you are. $18K is the equivalent of paying around 15% more for your developer.
  - ricardobayes 4mo ago
    In hcol locations yes, but in south of spain you can get full time talent for that figure. It's also an entry-level salary in eastern europe, with ukraine and turkey even being somewhat cheaper.
- KronisLV 4mo ago
  In Latvia, the net salary for a Java dev is around 1729 - 4314 EUR, based on https://www.algas.lv/algu-informacija/informacijas-tehnologijas/java-programmetajs https://www.algas.lv/algu-informacija/informacijas-tehnologi... (crowd sourced data) For the employer those employees cost between 2945 - 7736 EUR per month based on https://kalkulatori.lv/lv/algas-kalkulators https://kalkulatori.lv/lv/algas-kalkulators (income and social taxes). So on the lower end that's (1500 USD ~ 1300 EUR) close to half the total expenses of such a developer, on the high end here around 15-20%. That's quite significant, depends on whether their productivity also improves (if that's what the orgs care about). And we’re not even the country with the worst pay out there, but pay the same for tokens, cause regional pricing isn’t a thing!