5 ms·

This was an interesting article, but he completely misses the only real threat to either Open AI or Anthro. Open source models like deepseek and llama 3 are ra

This was an interesting article, but he completely misses the only real threat to either Open AI or Anthro. Open source models like deepseek and llama 3 are rapidly catching up, if I can get 90% of the functionality for significantly less ( or free if I want to use my own GPU), what value does open AI really have . I'm a paid subscriber of open AI, but it's really just a matter of convenience. The app is really good, and I find it's really great for double checking some of my math. However I don't know how they're going to ever truly become a corporate necessity at the prices they're going to need to bill at to become profitable. Then again, open AI is obviously ran by some of the smartest people on planet Earth, with other extremely smart people giving them tons of money, so I can be completely wrong here

TechDebtDevin 1y ago
There is no threat, both are private public partnerships that greatly benefit the military/empire. Read Pynchon.
moomoo11 1y ago
For actual work and not toying around the 10% gap is absolutely worth the cost.
- 999900000999 1y ago
  You may be correct, but for my hobbyist projects I tend to use the cheaper models to get started, and then I'll switch to a more expensive one if the cheaper model gets stuck. Unless the actual race is to create an AI Employee that operates and can deliver work without constant supervision. At that level, of course it would be cheaper to pay $2,000 a month straight to Open AI vs hiring a junior SWE
- georgemcbay 1y ago
  > For actual work and not toying around the 10% gap is absolutely worth the cost. I agree with this... for now. But the hosted commercial models aren't widening the gap as far as I can tell, if anything it appears to be narrowing. And if the relative delta doesn't increase somehow I don't see any way in which the "AI race" doesn't end in a situation where locally run LLMs on relatively cheap hardware end up being good enough for virtually everyone. Which in many ways is the best possible outcome, except for the likely severe economic effects when the bubble bursts on the commercial AI side.
  - moomoo11 1y ago
    Sure. I'm talking about now. I don't know what will happen tomorrow, let alone 1 year or more down the line. If it ever becomes economical to run and maintain bare metal GPU compute to run LLMs, then that's what will need to be done..
- thijson 1y ago
  I feel like options for local inference are getting better. AMD has their Strix Halo. Intel's next CPU generation Arrow Lake will have better inference abilities as well.
- Beijinger 1y ago
  Yes. But there is not only OpenAI. There is Gemini, Grok, whatever. If it doesn't become a "the winner takes it all" but a commodity like web hosting, then the payoff breaks down.
- paulcole 1y ago
  How clearly are you able to define “actual work” and “toying around?” Or is this the case of every HN discussion where what you do is “actual work” and what other people do is “toying around?”
  - moomoo11 1y ago
    Are you making [actual] money? That's literally it.
    - paulcole 1y ago
      I don’t know why you’re downvoted. It’s a good and fair answer. I disagree with it to some extent but at least it makes sense.
      - moomoo11 1y ago
        No worries. I stopped caring about points and validation years ago lol just being honest is enough for me.
- samastur 1y ago
  This highly depends on the price difference and value you get form those 10%.
- tkz1312 1y ago
  Don’t underestimate the cost of getting locked into a tool that is 100% guaranteed to rugpull you on both cost and privacy.
  - moomoo11 1y ago
    You know that this is literally not a concern right? It is part of business life to navigate such a situation. When I was a SWE, I've done migrations between bare metal to AWS to GCP and then both, and then all plus Azure.. It is the cost of doing business. You pick what works best at a price that is optimized for your business needs now. You have a war chest so when they start becoming assholes, you have leverage to fight back or pivot. I'd rather spend $200-400/mo to unblock myself NOW than do something dumb with 5 or even 100 tokens per second output that isn't actually that good as what the current providers offer. I'm going through millions of tokens a day.. I couldn't do that with local "RIGHT NOW" (<--- important)
bwy 1y ago
I think you're wrong, in the same way that folks on HN were wrong about Dropbox–HN: why would I pay for something that provides so little value, it's just slightly more convenient file storage? Just because open source models are almost as good, doesn't mean you can underestimate the convenience factor. Both can be true: we're in an AI bubble, and the large incumbents will capture most of the value/be difficult to unseat.
- lcnPylGDnU4H9OF 1y ago
  > folks on HN were wrong about Dropbox–HN: why would I pay for something that provides so little value, it's just slightly more convenient file storage? https://news.ycombinator.com/item?id=42392302 https://news.ycombinator.com/item?id=42392302 (Before discussion of your comment devolves into nonsense about this.)
- hbn 1y ago
  On the other hand, no one has figured out how to make money providing AI yet, and everyone's operating at a loss. At some point they're going to need to monetize, and the cost/convenience compared to alternatives may not be worth it for a lot of people. At one point you could get a Netflix subscription and it was convenient enough that people were pirating less. Now there's so many subscription services, we're basically back to cable packages, paying ever increasing amounts and potentially still seeing ads. I know I'm pirating a lot more again. Uber vs cabs, Airbnb vs hotels - We've seen it time and time again, once the VC cashflow/infinite stonk growth dries up and they need to figure out how to monetize, the service becomes worse and people start looking for alternatives again.
  - VBprogrammer 1y ago
    Yeah, but not just that. I don't expect my mum to go find some high end consumer GPU and install it on a home server in order to run her own local LLM. I expect that people will be throwing chat interfaces running remixed versions of open weight models out on the internet so fast that it's impossible for anyone to monitise it in a reasonable way. I also wonder whether, similar to bitcoin mining, these things end up on specialist ASICS and before we know it a medium tier mobile phone is running your own local models.
JumpCrisscross 1y ago
> Open source models like deepseek and llama 3 are rapidly catching up, if I can get 90% of the functionality for significantly less ( or free if I want to use my own GPU), what value does open AI really have They pay for the hardware and electricity /s.
- 999900000999 1y ago
  You have several providers who host both deep-seek and llama 3. They pay for the hardware and electricity, you pay for usage but it's significantly cheaper than using OpenAIs models.
  - JumpCrisscross 1y ago
    > you pay for usage Plenty of people don't. That's an enduring advantage of using GPT over anything locally hosted.
  - jareds 1y ago
    Where are these providers and do they offer batch processing? If they don't how does there cost compare to Gemini and OpenAI batch processing? For the hobby project I'm working on batch processing is a great fit. The only cost comparison tool I've been able to find is openrouter and it doesn't support batch processing for cost savings.
    - getpokedagain 1y ago
      Aws?
- deleted 1y ago
  [deleted]
computerex 1y ago
I don’t think that’s the point he is making. The argument to me is looking at the numbers and grounding them in real life. It takes time to build data centers, it takes people to run them. The article makes the argument that the timelines are not feasible.
jsheard 1y ago
> Open source models like deepseek and llama 3 are rapidly catching up Catching up for how long though? Large models are very expensive to train, and the only reason any of them are open is because ostensibly for-profit companies are absorbing the enormous costs on behalf of the open source scene. What's the plan B when those unprofitable companies (or divisions within companies) pull the ladder up behind them in pursuit of profits?
- mgh95 1y ago
  > Catching up for how long though? Large models are expensive to train and the money has to come from somewhere, and for now that somewhere is the charity of ostensibly for-profit companies which release their models for free and just eat the losses. That doesn't seem very sustainable to me. Catching up enough to execute a business successfully on the open source (and free as in free beer) alternatives to OpenAI. Once you have a single model that works, you can bootstrap your own internal ML infra for just that one use case and grow from there.
- mark_l_watson 1y ago
  Martin Casado from a16z stated an opinion that 80% of US startups are likely using less expensive open models, usually from China. Chamath P. on the All In Podcast said that his company is using Chinese models, but hosted in the US and he cited his company as a huge inference user.
  - tcdent 1y ago
    Important to distinguish that they're likely using them for internal workflows (agents, etc) where the scope is well defined and they can tune their prompts and evals to accommodate a lower-performance model. Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.
- CheeseFromLidl 1y ago
  LLMs can be trained using a distributed approach [1]. So what’s stopping the OSS community from setting up a pool in the likes of seti@home and boinc? [1] I asked Gemini and apparently it’s quite common
jocaal 1y ago
> I'm a paid subscriber of open AI, but it's really just a matter of convenience. The app is really good, and I find it's really great for double checking some of my math. That right there is why they are valuable. Most people are absolutely incompetent when it comes to IT. That's why no one you meet in the real world uses ad blockers. OpenAI secured their position in the mind share of the masses. All they had to do to become the next google was find a way to force ads down the throats of their users. Instead they opted for the inflated bubble and scam investors strategy. Rookie mistake.
- array_key_first 1y ago
  The mind share openAI has is next to none. The reality is they're a paid service, and even if they 10x their prices they're still in the red. Consumers do actually care about price. They will easily, and quickly, move to a cheaper service. There's no lock in here.
  - Ekaros 1y ago
    There is talk of 800 million weekly users or whatever. But real question to me is how much actual disposable income they have or willingness to spend it on expensive AI subscription.
  - zaphirplane 1y ago
    Not true, for the non tech crowd ChatGPT is the AI. There are a few people using Grok or Gemini, fewer outside the coding crowd would know anthropic
    - array_key_first 1y ago
      This is just not true. They don't even know what an OpenAI is, they just know what chat is. It's a chat window. You make another chat window and you're golden.
      - zaphirplane 1y ago
tkz1312 1y ago
> free if I want to use my own GPU The hardware required to run something like deepseek / kimi / glm locally at any speed fast enough for coding is probably around $50,000. You need hundreds of gigabytes of fast VRAM to run models that can come anywhere close to openai or anthropic.
- edude03 1y ago
  $50k would be the cost to run it un-quantized, 10k could get you for example 4 5090 system, that would run the 671b q4 model which is 90% as good, which was the OPs target
  - tkz1312 1y ago
    which 671b quants can fit into 96GB VRAM? Everything I’m aware of needs hundreds at least (e.g. https://apxml.com/models/deepseek-r1-671b https://apxml.com/models/deepseek-r1-671b).
    - lossolo 1y ago
      5090 is 32 GB so it's 128GB, not 96.
      - tkz1312 1y ago
        128 is still not 300. Something like 4x 6000 blackwell is the minimum to run any model that is going to feel anything like claude locally. To my deep disappointment the economics are simply not there at the moment. Openrouter using only providers with zero data retention policies is probably the best option right now if you care about openness, privacy and vendor lock-in.
        999900000999 1y ago
        For local use and experimentation you don't need to match a top of the line model. In fact something that you train or rather fine-tune locally might be better for certain use cases. If I was working with sensitive data I sure would only use on prem models.
thenaturalist 1y ago
This is such a techno-centric view, you're not even remotely aware of your own biases. > but he completely misses the only real threat to either Open AI or Anthro. Hard disagree. Economics matter, in fact more than tech. Tech only gets to shine if the economics work out. > Then again, open AI is obviously ran by some of the smartest people on planet Earth, with other extremely smart people giving them tons of money, so I can be completely wrong here Nope. Money != Intelligence Open AI, SF and the West-Coast VC scene is run by very opionated, incentiviced people. Yes, money can make things move, but all the money of the world don't matter if your unit econonmics don't work out. And the startup graveyard is full of examples of this kind.
scrubs 1y ago
Meanwhile, the money asks, short time frames, and unclear demands for same are huge business problems. Self service (llama on my own gpus) I suppose is just another way to ask where the demand is coming from to argue for billions in new money. Something smells ...

TechDebtDevin 1y ago

There is no threat, both are private public partnerships that greatly benefit the military/empire. Read Pynchon.

moomoo11 1y ago

For actual work and not toying around the 10% gap is absolutely worth the cost.

999900000999 1y ago
You may be correct, but for my hobbyist projects I tend to use the cheaper models to get started, and then I'll switch to a more expensive one if the cheaper model gets stuck. Unless the actual race is to create an AI Employee that operates and can deliver work without constant supervision. At that level, of course it would be cheaper to pay $2,000 a month straight to Open AI vs hiring a junior SWE
georgemcbay 1y ago
> For actual work and not toying around the 10% gap is absolutely worth the cost. I agree with this... for now. But the hosted commercial models aren't widening the gap as far as I can tell, if anything it appears to be narrowing. And if the relative delta doesn't increase somehow I don't see any way in which the "AI race" doesn't end in a situation where locally run LLMs on relatively cheap hardware end up being good enough for virtually everyone. Which in many ways is the best possible outcome, except for the likely severe economic effects when the bubble bursts on the commercial AI side.
- moomoo11 1y ago
  Sure. I'm talking about now. I don't know what will happen tomorrow, let alone 1 year or more down the line. If it ever becomes economical to run and maintain bare metal GPU compute to run LLMs, then that's what will need to be done..
thijson 1y ago
I feel like options for local inference are getting better. AMD has their Strix Halo. Intel's next CPU generation Arrow Lake will have better inference abilities as well.
Beijinger 1y ago
Yes. But there is not only OpenAI. There is Gemini, Grok, whatever. If it doesn't become a "the winner takes it all" but a commodity like web hosting, then the payoff breaks down.
paulcole 1y ago
How clearly are you able to define “actual work” and “toying around?” Or is this the case of every HN discussion where what you do is “actual work” and what other people do is “toying around?”
- moomoo11 1y ago
  Are you making [actual] money? That's literally it.
  - paulcole 1y ago
    I don’t know why you’re downvoted. It’s a good and fair answer. I disagree with it to some extent but at least it makes sense.
    - moomoo11 1y ago
      No worries. I stopped caring about points and validation years ago lol just being honest is enough for me.
samastur 1y ago
This highly depends on the price difference and value you get form those 10%.
tkz1312 1y ago
Don’t underestimate the cost of getting locked into a tool that is 100% guaranteed to rugpull you on both cost and privacy.
- moomoo11 1y ago
  You know that this is literally not a concern right? It is part of business life to navigate such a situation. When I was a SWE, I've done migrations between bare metal to AWS to GCP and then both, and then all plus Azure.. It is the cost of doing business. You pick what works best at a price that is optimized for your business needs now. You have a war chest so when they start becoming assholes, you have leverage to fight back or pivot. I'd rather spend $200-400/mo to unblock myself NOW than do something dumb with 5 or even 100 tokens per second output that isn't actually that good as what the current providers offer. I'm going through millions of tokens a day.. I couldn't do that with local "RIGHT NOW" (<--- important)

bwy 1y ago

I think you're wrong, in the same way that folks on HN were wrong about Dropbox–HN: why would I pay for something that provides so little value, it's just slightly more convenient file storage? Just because open source models are almost as good, doesn't mean you can underestimate the convenience factor. Both can be true: we're in an AI bubble, and the large incumbents will capture most of the value/be difficult to unseat.

lcnPylGDnU4H9OF 1y ago
> folks on HN were wrong about Dropbox–HN: why would I pay for something that provides so little value, it's just slightly more convenient file storage? https://news.ycombinator.com/item?id=42392302 https://news.ycombinator.com/item?id=42392302 (Before discussion of your comment devolves into nonsense about this.)
hbn 1y ago
On the other hand, no one has figured out how to make money providing AI yet, and everyone's operating at a loss. At some point they're going to need to monetize, and the cost/convenience compared to alternatives may not be worth it for a lot of people. At one point you could get a Netflix subscription and it was convenient enough that people were pirating less. Now there's so many subscription services, we're basically back to cable packages, paying ever increasing amounts and potentially still seeing ads. I know I'm pirating a lot more again. Uber vs cabs, Airbnb vs hotels - We've seen it time and time again, once the VC cashflow/infinite stonk growth dries up and they need to figure out how to monetize, the service becomes worse and people start looking for alternatives again.
- VBprogrammer 1y ago
  Yeah, but not just that. I don't expect my mum to go find some high end consumer GPU and install it on a home server in order to run her own local LLM. I expect that people will be throwing chat interfaces running remixed versions of open weight models out on the internet so fast that it's impossible for anyone to monitise it in a reasonable way. I also wonder whether, similar to bitcoin mining, these things end up on specialist ASICS and before we know it a medium tier mobile phone is running your own local models.

JumpCrisscross 1y ago

> Open source models like deepseek and llama 3 are rapidly catching up, if I can get 90% of the functionality for significantly less ( or free if I want to use my own GPU), what value does open AI really have They pay for the hardware and electricity /s.

999900000999 1y ago
You have several providers who host both deep-seek and llama 3. They pay for the hardware and electricity, you pay for usage but it's significantly cheaper than using OpenAIs models.
- JumpCrisscross 1y ago
  > you pay for usage Plenty of people don't. That's an enduring advantage of using GPT over anything locally hosted.
- jareds 1y ago
  Where are these providers and do they offer batch processing? If they don't how does there cost compare to Gemini and OpenAI batch processing? For the hobby project I'm working on batch processing is a great fit. The only cost comparison tool I've been able to find is openrouter and it doesn't support batch processing for cost savings.
  - getpokedagain 1y ago
    Aws?
deleted 1y ago
[deleted]

computerex 1y ago

I don’t think that’s the point he is making. The argument to me is looking at the numbers and grounding them in real life. It takes time to build data centers, it takes people to run them. The article makes the argument that the timelines are not feasible.

jsheard 1y ago

> Open source models like deepseek and llama 3 are rapidly catching up Catching up for how long though? Large models are very expensive to train, and the only reason any of them are open is because ostensibly for-profit companies are absorbing the enormous costs on behalf of the open source scene. What's the plan B when those unprofitable companies (or divisions within companies) pull the ladder up behind them in pursuit of profits?

mgh95 1y ago
> Catching up for how long though? Large models are expensive to train and the money has to come from somewhere, and for now that somewhere is the charity of ostensibly for-profit companies which release their models for free and just eat the losses. That doesn't seem very sustainable to me. Catching up enough to execute a business successfully on the open source (and free as in free beer) alternatives to OpenAI. Once you have a single model that works, you can bootstrap your own internal ML infra for just that one use case and grow from there.
mark_l_watson 1y ago
Martin Casado from a16z stated an opinion that 80% of US startups are likely using less expensive open models, usually from China. Chamath P. on the All In Podcast said that his company is using Chinese models, but hosted in the US and he cited his company as a huge inference user.
- tcdent 1y ago
  Important to distinguish that they're likely using them for internal workflows (agents, etc) where the scope is well defined and they can tune their prompts and evals to accommodate a lower-performance model. Nobody is advocating switching their coding agents to open source (yet), but that's not the bulk of the tokens in companies that have automated workflows integrated into their business.
CheeseFromLidl 1y ago
LLMs can be trained using a distributed approach [1]. So what’s stopping the OSS community from setting up a pool in the likes of seti@home and boinc? [1] I asked Gemini and apparently it’s quite common

jocaal 1y ago

> I'm a paid subscriber of open AI, but it's really just a matter of convenience. The app is really good, and I find it's really great for double checking some of my math. That right there is why they are valuable. Most people are absolutely incompetent when it comes to IT. That's why no one you meet in the real world uses ad blockers. OpenAI secured their position in the mind share of the masses. All they had to do to become the next google was find a way to force ads down the throats of their users. Instead they opted for the inflated bubble and scam investors strategy. Rookie mistake.

array_key_first 1y ago
The mind share openAI has is next to none. The reality is they're a paid service, and even if they 10x their prices they're still in the red. Consumers do actually care about price. They will easily, and quickly, move to a cheaper service. There's no lock in here.
- Ekaros 1y ago
  There is talk of 800 million weekly users or whatever. But real question to me is how much actual disposable income they have or willingness to spend it on expensive AI subscription.
- zaphirplane 1y ago
  Not true, for the non tech crowd ChatGPT is the AI. There are a few people using Grok or Gemini, fewer outside the coding crowd would know anthropic
  - array_key_first 1y ago
    This is just not true. They don't even know what an OpenAI is, they just know what chat is. It's a chat window. You make another chat window and you're golden.
    - zaphirplane 1y ago

tkz1312 1y ago

> free if I want to use my own GPU The hardware required to run something like deepseek / kimi / glm locally at any speed fast enough for coding is probably around $50,000. You need hundreds of gigabytes of fast VRAM to run models that can come anywhere close to openai or anthropic.

edude03 1y ago
$50k would be the cost to run it un-quantized, 10k could get you for example 4 5090 system, that would run the 671b q4 model which is 90% as good, which was the OPs target
- tkz1312 1y ago
  which 671b quants can fit into 96GB VRAM? Everything I’m aware of needs hundreds at least (e.g. https://apxml.com/models/deepseek-r1-671b https://apxml.com/models/deepseek-r1-671b).
  - lossolo 1y ago
    5090 is 32 GB so it's 128GB, not 96.
    - tkz1312 1y ago
      128 is still not 300. Something like 4x 6000 blackwell is the minimum to run any model that is going to feel anything like claude locally. To my deep disappointment the economics are simply not there at the moment. Openrouter using only providers with zero data retention policies is probably the best option right now if you care about openness, privacy and vendor lock-in.
      - 999900000999 1y ago
        For local use and experimentation you don't need to match a top of the line model. In fact something that you train or rather fine-tune locally might be better for certain use cases. If I was working with sensitive data I sure would only use on prem models.

thenaturalist 1y ago

This is such a techno-centric view, you're not even remotely aware of your own biases. > but he completely misses the only real threat to either Open AI or Anthro. Hard disagree. Economics matter, in fact more than tech. Tech only gets to shine if the economics work out. > Then again, open AI is obviously ran by some of the smartest people on planet Earth, with other extremely smart people giving them tons of money, so I can be completely wrong here Nope. Money != Intelligence Open AI, SF and the West-Coast VC scene is run by very opionated, incentiviced people. Yes, money can make things move, but all the money of the world don't matter if your unit econonmics don't work out. And the startup graveyard is full of examples of this kind.

scrubs 1y ago

Meanwhile, the money asks, short time frames, and unclear demands for same are huge business problems. Self service (llama on my own gpus) I suppose is just another way to ask where the demand is coming from to argue for billions in new money. Something smells ...