5 ms·

You can't run Claude on your PC; you need servers. Companies that have that kind of hardware are not going to touch a pirated model. And the next model will be

by wmf 1y ago

You can't run Claude on your PC; you need servers. Companies that have that kind of hardware are not going to touch a pirated model. And the next model will be out in a few months anyway.

jayd16 1y ago
If it was worth it, you'd see some easy self hostable package, no? And by definition, its profitable to self host or these AI companies are in trouble.
- quotemstr 1y ago
  Does your "self hostable package" come with its own electric substation?
  - jayd16 1y ago
    You're saying that's needed for inference?
- serf 1y ago
  I think this misunderstands the scale of these models. And honestly I don't think a lot of these companies would turn a profit on pure utility -- the electric and water company doesn't advertise like these groups do; I think that probably means something.
  - jayd16 1y ago
    What's the scale for inference? Is it truly that immense? Can you ballpark what you think would make such a thing impossible? > the electric and water company doesn't advertise like these groups do I'm trying to understand what you mean here. In the US these utilities usually operate in a monopoly so there's no point in advertising. Cell service has plenty of advertising though.
- tick_tock_tick 1y ago
  You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. You think anyone is going to make money on that vs $20 a month to anthropic?
  - deleted 1y ago
    [deleted]
  - jayd16 1y ago
    Can you explain to me where Anthropic (or it's investors) expect to be making money if that's what it actually costs to run this stuff?
    - lelanthran 1y ago
      > Can you explain to me where Anthropic (or it's investors) expect to be making money if that's what it actually costs to run this stuff? Not the GP (in fact I just replied to GP, disagreeing with them), but I think that economies of scale kick in when you are provisioning M GPUs for N users and both M and N are large. When you are provisioning for N=1 (a single user), then M=1 is the minimum you need, which makes it very expensive per user. When N=5 and M is still 1, then the cost per user is roughly a fifth of the original single-user cost.
  - lelanthran 1y ago
    > You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. This doesn't seem correct. I run legacy models with only slightly reduced performance on 32GB RAM with a 12GB VRAM GPU right now. BTW, that's not an expensive setup. > You think anyone is going to make money on that vs $20 a month to anthropic? Why does it have to be run as a profit-making machine for other users? It can run as a useful service for the entire household, when running at home. After all, we're not talking about specialised coding agents using this[1], just normal user requests. ==================================== [1] For an outlay of $1k for a new GPU I can run a reduced-performance coding LLM. Once again, when it's only myself using it, the economics work out. I don't need the agent to be fully autonomous because I'm not vibe coding - I can take the reduced-performance output, fix it and use it.

by wmf 1y ago

You can't run Claude on your PC; you need servers. Companies that have that kind of hardware are not going to touch a pirated model. And the next model will be out in a few months anyway.

jayd16 1y ago

If it was worth it, you'd see some easy self hostable package, no? And by definition, its profitable to self host or these AI companies are in trouble.

quotemstr 1y ago
Does your "self hostable package" come with its own electric substation?
- jayd16 1y ago
  You're saying that's needed for inference?
serf 1y ago
I think this misunderstands the scale of these models. And honestly I don't think a lot of these companies would turn a profit on pure utility -- the electric and water company doesn't advertise like these groups do; I think that probably means something.
- jayd16 1y ago
  What's the scale for inference? Is it truly that immense? Can you ballpark what you think would make such a thing impossible? > the electric and water company doesn't advertise like these groups do I'm trying to understand what you mean here. In the US these utilities usually operate in a monopoly so there's no point in advertising. Cell service has plenty of advertising though.
tick_tock_tick 1y ago
You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. You think anyone is going to make money on that vs $20 a month to anthropic?
- deleted 1y ago
  [deleted]
- jayd16 1y ago
  Can you explain to me where Anthropic (or it's investors) expect to be making money if that's what it actually costs to run this stuff?
  - lelanthran 1y ago
    > Can you explain to me where Anthropic (or it's investors) expect to be making money if that's what it actually costs to run this stuff? Not the GP (in fact I just replied to GP, disagreeing with them), but I think that economies of scale kick in when you are provisioning M GPUs for N users and both M and N are large. When you are provisioning for N=1 (a single user), then M=1 is the minimum you need, which makes it very expensive per user. When N=5 and M is still 1, then the cost per user is roughly a fifth of the original single-user cost.
- lelanthran 1y ago
  > You need a 100+gigs ram and a top of the line GPU to run legacy models at home. Maybe if you push it that setup will let you handle 2 people maybe 3 people. This doesn't seem correct. I run legacy models with only slightly reduced performance on 32GB RAM with a 12GB VRAM GPU right now. BTW, that's not an expensive setup. > You think anyone is going to make money on that vs $20 a month to anthropic? Why does it have to be run as a profit-making machine for other users? It can run as a useful service for the entire household, when running at home. After all, we're not talking about specialised coding agents using this[1], just normal user requests. ==================================== [1] For an outlay of $1k for a new GPU I can run a reduced-performance coding LLM. Once again, when it's only myself using it, the economics work out. I don't need the agent to be fully autonomous because I'm not vibe coding - I can take the reduced-performance output, fix it and use it.