4 ms·

I really think this is a coordinated effort to restrict computing capacity for individuals and force adoption of centralized AI - I think there already is evide

I really think this is a coordinated effort to restrict computing capacity for individuals and force adoption of centralized AI - I think there already is evidence of this from the moves OpenAI had made to lock up memory and gpu markets.

aroman 4mo ago
Who exactly is “coordinating” that effort? Surely everyone except the datacenter builders and the big hosted AI models has exactly the opposite incentive.
- angoragoats 4mo ago
  > Who exactly is “coordinating” that effort? The datacenter builders and the big hosted AI models. The person you're replying to even mentions OpenAI by name.
- varispeed 4mo ago
  The rich.
- fsflover 4mo ago
  https://news.ycombinator.com/item?id=48673500 https://news.ycombinator.com/item?id=48673500
- kamranjon 4mo ago
  I think one of the more ominous things to see in recent years was all of the tech execs at the presidential inauguration, after having collectively donated several million dollars to the inauguration fund. So if we go with that list, which happens to overlap with many of the circular deals we’ve seen in the AI space recently, you’d have people like: Sam Altman, Jeff Bezos, Elon Musk, Mark Zuckerberg, Tim Cook, Sundar Pichai and Sergey Brin I also wouldn’t be surprised if memory providers weren’t intimately involved, as they’ve been caught price fixing in the past: https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal
  - bigyabai 4mo ago
    We have to get real, here - most people are not replacing GPT or Claude with local inference, even on M5. If you can afford to do that (RAM shortage or not), then you are in the minority of customers. Alleviating the memory constraint would only really make Nvidia a danger to cloud margins, and their consumer sales are neutered while they focus on the datacenter segment. It's feels facetious to insinuate that people would be doing inference on their Macbook Neo or Wintel laptop if they only had a gorbillion gigabytes of memory and a 400W accelerator card plugged into the wall outlet.
    - kamranjon 4mo ago
      You’re out of the loop if you don’t think m series chips with unified memory aren’t one of the best platforms for running local inference
      - bigyabai 4mo ago
        They aren't. Apple Silicon is unusable for interactive prefill and decode speeds in agentic workflows and SOTA LLMs.
        kamranjon 4mo ago
        You’re just out of the loop, and that’s fine but it’s worth learning about. There is a pretty large and growing community of us using entirely local models for our agentic flows. From GLM 4.7 flash on 32gb machines with >60tok/s to Gemma and Qwen dense and MOE models on 64gb machines all the way up to Deepseek V4 flash on 128gb machines with 450tok/s prefill and 25-30tok/s decode. I use DS4 on the daily - it’s become my main model. I know it’s in fashion to talk trash about Apple but their hardware outperforms other options like DGX Sparc when it comes to local inference, they got the unified memory, memory bandwidth and the GPU cores to actually be useful in a way that most other hardware just isn’t.
        aroman 4mo ago

aroman 4mo ago

Who exactly is “coordinating” that effort? Surely everyone except the datacenter builders and the big hosted AI models has exactly the opposite incentive.

angoragoats 4mo ago
> Who exactly is “coordinating” that effort? The datacenter builders and the big hosted AI models. The person you're replying to even mentions OpenAI by name.
varispeed 4mo ago
The rich.
fsflover 4mo ago
https://news.ycombinator.com/item?id=48673500 https://news.ycombinator.com/item?id=48673500
kamranjon 4mo ago
I think one of the more ominous things to see in recent years was all of the tech execs at the presidential inauguration, after having collectively donated several million dollars to the inauguration fund. So if we go with that list, which happens to overlap with many of the circular deals we’ve seen in the AI space recently, you’d have people like: Sam Altman, Jeff Bezos, Elon Musk, Mark Zuckerberg, Tim Cook, Sundar Pichai and Sergey Brin I also wouldn’t be surprised if memory providers weren’t intimately involved, as they’ve been caught price fixing in the past: https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal
- bigyabai 4mo ago
  We have to get real, here - most people are not replacing GPT or Claude with local inference, even on M5. If you can afford to do that (RAM shortage or not), then you are in the minority of customers. Alleviating the memory constraint would only really make Nvidia a danger to cloud margins, and their consumer sales are neutered while they focus on the datacenter segment. It's feels facetious to insinuate that people would be doing inference on their Macbook Neo or Wintel laptop if they only had a gorbillion gigabytes of memory and a 400W accelerator card plugged into the wall outlet.
  - kamranjon 4mo ago
    You’re out of the loop if you don’t think m series chips with unified memory aren’t one of the best platforms for running local inference
    - bigyabai 4mo ago
      They aren't. Apple Silicon is unusable for interactive prefill and decode speeds in agentic workflows and SOTA LLMs.
      - kamranjon 4mo ago
        You’re just out of the loop, and that’s fine but it’s worth learning about. There is a pretty large and growing community of us using entirely local models for our agentic flows. From GLM 4.7 flash on 32gb machines with >60tok/s to Gemma and Qwen dense and MOE models on 64gb machines all the way up to Deepseek V4 flash on 128gb machines with 450tok/s prefill and 25-30tok/s decode. I use DS4 on the daily - it’s become my main model. I know it’s in fashion to talk trash about Apple but their hardware outperforms other options like DGX Sparc when it comes to local inference, they got the unified memory, memory bandwidth and the GPU cores to actually be useful in a way that most other hardware just isn’t.
        aroman 4mo ago