3 ms·

I am eagerly awaiting being able to run a strong local model. I'd hand Apple $5k right now for a Claude in a box. I know the cost might not be there now, just s

I am eagerly awaiting being able to run a strong local model. I'd hand Apple $5k right now for a Claude in a box. I know the cost might not be there now, just saying that is around my ideal price point. $10k might even be worth it - but i'm assuming that the more expensive it is the beefier it is too, which also means more electricity.. and i already run ~6 computers/servers in my house. If a power surge happens i'm going to go live in the woods lol.

DANmode 5mo ago
You can run 6-12 month old state of the art models for that type of money, like, yesterday.
- unshavedyak 5mo ago
  Yea, but i don't consider them good enough. I barely consider SOTA good enough. I'm hoping that by the time the rugpull happens with SOTA (claude/etc) that at-home will be in the 4.7-5.5 range? We'll see.
  - DANmode 5mo ago
    They were good enough 6-12 months ago. Maybe your tooling is what’s keeping you from your dream.
atonse 5mo ago
I would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that. But maybe my limited understanding is thinking of this wrong.
- JamesLeonis 5mo ago
  I wouldn't worry about hardware. I've run the latest local models over the last year, including the recent Qwen 3.6 30B A3B, on a 9yo GTX 1080 and 32G RAM I have lying around[0]. If I can do that I don't think hardware will be a problem for you in the near term. The only updates I've needed were to Llama.cpp when a new class of model was released. [0]: In my case, I want to see how local models perform on limited hardware, sacrificing context size and intelligence compared to SOTA models, so I have to really limit my expectations.
- unshavedyak 5mo ago
  > I would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that. I think the same, and it's why i stopped caring about running llama/etc at home last year. That coupled with the models being dumb by comparison to SOTA really make me fine with waiting. But in a year or two it's going to be difficult to resist at home, assuming the pace of improvement holds.
- DANmode 5mo ago
  Focus on what’s actually required for your workflows. Anything beyond that is just hobby, or continued education.
templar_snow 5mo ago
Uh... get a UPS?
- unshavedyak 5mo ago
  I do, though they're not as bullet proof as you'd hope to my understanding. Hell i have one at the house level too - since i have an EV sitting behind that as well.
  - DANmode 5mo ago
    Are you in a region that doesn’t mandate grounded electrical systems? (UPS is still a great idea for your expensive gear.)

DANmode 5mo ago

You can run 6-12 month old state of the art models for that type of money, like, yesterday.

unshavedyak 5mo ago
Yea, but i don't consider them good enough. I barely consider SOTA good enough. I'm hoping that by the time the rugpull happens with SOTA (claude/etc) that at-home will be in the 4.7-5.5 range? We'll see.
- DANmode 5mo ago
  They were good enough 6-12 months ago. Maybe your tooling is what’s keeping you from your dream.

atonse 5mo ago

I would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that. But maybe my limited understanding is thinking of this wrong.

JamesLeonis 5mo ago
I wouldn't worry about hardware. I've run the latest local models over the last year, including the recent Qwen 3.6 30B A3B, on a 9yo GTX 1080 and 32G RAM I have lying around[0]. If I can do that I don't think hardware will be a problem for you in the near term. The only updates I've needed were to Llama.cpp when a new class of model was released. [0]: In my case, I want to see how local models perform on limited hardware, sacrificing context size and intelligence compared to SOTA models, so I have to really limit my expectations.
unshavedyak 5mo ago
> I would do the same but my issue is that the models are changing so fast, so I don't want to be left out of the next model cuz it only runs on an even newer GPU or something like that. I think the same, and it's why i stopped caring about running llama/etc at home last year. That coupled with the models being dumb by comparison to SOTA really make me fine with waiting. But in a year or two it's going to be difficult to resist at home, assuming the pace of improvement holds.
DANmode 5mo ago
Focus on what’s actually required for your workflows. Anything beyond that is just hobby, or continued education.

templar_snow 5mo ago

Uh... get a UPS?

unshavedyak 5mo ago
I do, though they're not as bullet proof as you'd hope to my understanding. Hell i have one at the house level too - since i have an EV sitting behind that as well.
- DANmode 5mo ago
  Are you in a region that doesn’t mandate grounded electrical systems? (UPS is still a great idea for your expensive gear.)