4 ms·

antirez running (quantized) DeepSeek V4 Pro on a Mac Studio M3 Ultra with 512GB of RAM: https://bsky.app/profile/antirez.bsky.social/post/3mlzwmvlov22r https:/

antirez running (quantized) DeepSeek V4 Pro on a Mac Studio M3 Ultra with 512GB of RAM: https://bsky.app/profile/antirez.bsky.social/post/3mlzwmvlov22r https://bsky.app/profile/antirez.bsky.social/post/3mlzwmvlov... It's much closer than you think. We're going to see specialized hardware in the next 24 months capable of running 2025-era frontier models. That's big.

dist-epoch 5mo ago
That specialized hardware will be scooped up by AI data-centers, just like RAM is today.
- nine_k 5mo ago
  No more than Mac Studios. Datacenters need different hardware.
- ffsm8 5mo ago
  The 512 GB ram studio can't even be purchased anymore. It's been delisted https://www.apple.com/shop/buy-mac/mac-studio https://www.apple.com/shop/buy-mac/mac-studio Same with the Mac mini. entirely removed from all store references
treis 5mo ago
It's big because it may take a big swath of people who will actually pay for LLMs out of the market. But for the average consumer they're going to primarily use their phone/tablet and we're far away from that being possible. Even if it were possible the LLMs are such a gold mine of user data. It's really hard to see that opportunity be passed up.
deleted 5mo ago
[deleted]
menaerus 5mo ago
2-bit quantization? That's a lot of signal being removed. Considering how quickly the AI models are progressing in their capabilities (still exponential curve), I will not want to use the 2025 model in two years time. Similarly, how I don't want to use llama-3 or old Anthropic model from 2023 or 2024. Newer models are so much better that it makes it very difficult to ignore. Once and if the advancements with the AI models slow down, only then IMHO it will become feasible to design the specialized HW for general-purpose consumption and general-purpose workloads.
- nmfisher 5mo ago
  Opus 4.6 was a 2025 model and many people (myself included) feel that if that's where models peaked, we won't be disappointed. Even at 2-bit quantization, DS4 is probably on par with a 2024 frontier model. You can run that today on local hardware, and at a minimum, local models are going to keep pace over the next 12-24 months. Even if they don't close the gap with frontier models, they'll still play an important role in the overall pipeline for cost, speed and privacy reasons. That's without even mentioning the additional capability that something like a Taalas chip churning out 17k tokens/sec could unlock.

dist-epoch 5mo ago

That specialized hardware will be scooped up by AI data-centers, just like RAM is today.

nine_k 5mo ago
No more than Mac Studios. Datacenters need different hardware.
ffsm8 5mo ago
The 512 GB ram studio can't even be purchased anymore. It's been delisted https://www.apple.com/shop/buy-mac/mac-studio https://www.apple.com/shop/buy-mac/mac-studio Same with the Mac mini. entirely removed from all store references

treis 5mo ago

It's big because it may take a big swath of people who will actually pay for LLMs out of the market. But for the average consumer they're going to primarily use their phone/tablet and we're far away from that being possible. Even if it were possible the LLMs are such a gold mine of user data. It's really hard to see that opportunity be passed up.

deleted 5mo ago

[deleted]

menaerus 5mo ago

2-bit quantization? That's a lot of signal being removed. Considering how quickly the AI models are progressing in their capabilities (still exponential curve), I will not want to use the 2025 model in two years time. Similarly, how I don't want to use llama-3 or old Anthropic model from 2023 or 2024. Newer models are so much better that it makes it very difficult to ignore. Once and if the advancements with the AI models slow down, only then IMHO it will become feasible to design the specialized HW for general-purpose consumption and general-purpose workloads.

nmfisher 5mo ago
Opus 4.6 was a 2025 model and many people (myself included) feel that if that's where models peaked, we won't be disappointed. Even at 2-bit quantization, DS4 is probably on par with a 2024 frontier model. You can run that today on local hardware, and at a minimum, local models are going to keep pace over the next 12-24 months. Even if they don't close the gap with frontier models, they'll still play an important role in the overall pipeline for cost, speed and privacy reasons. That's without even mentioning the additional capability that something like a Taalas chip churning out 17k tokens/sec could unlock.