6 ms·

In my eyes, it'd be cheaper for a company to simply purchase laptops with decent hardware specs, and run the LLMs locally. I've had decent results from various

In my eyes, it'd be cheaper for a company to simply purchase laptops with decent hardware specs, and run the LLMs locally. I've had decent results from various models I've run via LMStudio, and bonus points: It costs nothing and doesn't even use all that much CPU/GPU power. Just my opinion as a FORMER senior software dev (disabled now).

bilsbie 1y ago
Can you expand on this?
pruetj 1y ago
> purchase laptops with decent hardware specs > It costs nothing Seems like it does cost something?
- automatic6131 1y ago
  Quite, the typical 5 year depreciation on personal computing means a top-of-the-line $5k laptop works out to a ~$80/month spend... but it's on something you'd already spend for an employee
  - datadrivenangel 1y ago
    $2k / 5 years is ~$30/mo, and you'll get a better experience spending another $25/mo on one of the AI services (or with enough people a small pile of H100s)
zippyman55 1y ago
Same here. I already have the computer for work, so marginally, it costs nothing and it meets 90 percent of my LLM needs. Here comes the down vote!
dist-epoch 1y ago
Electricity is not free. If you do the math, online LLMs are much cheaper. And this is before considering capabilities/speed.
- the_snooze 1y ago
  They're cheaper right now because they're operating at a loss. At some point, the bill will come due. Netflix used to be $8/month for as many streams and password-shares as you wanted for a catalog that met your media consumption needs. It was a great deal back then. But then the bill came due. Online LLM companies are positioning themselves to do the same bait-and-switch techbro BS we've seen over the last 15+ years.
  - dist-epoch 1y ago
    Fundamentally it will always be cheaper to run LLMs in the cloud, because of batching. Unless somehow magically you'll have the need to run 1000 different prompts at the exact same time to also benefit from it locally. This is even without considering cloud GPUs which are much more efficient than local ones, especially from old hardware.
    - dns_snek 1y ago
      Yes they'll be cheaper to run, but will they be cheaper buy as a service? Because sooner or later these companies will be expected to produce eye-watering ROI to justify the risk of these moonshot investments and they won't be doing that by selling at cost.
      - kingstnap 1y ago
        Will they be cheaper to buy? Yes. You are effectively just buying compute with AI. From a simple correlational extrapolation compute has only gotten more cheaper over time. Massively so actually. From a more reasoned causal extrapolation hardware companies historically compete to bring the price of compute down. For AI this is extremely aggressive I might add. HotChips 2024 and 2025 had so much AI coverage. Nvidia is in an arms race with so many companies. All over the last few years we have literally only ever seen AI get cheaper for the same level or better. No one is releasing worse and more expensive AI right now. Literally just a few days ago Deepseek halved the price of V3.2. AI expenses have grown but that's because human's are extremely cognitively greedy. We value our time far more than compute efficiency.
mokoshhydro 1y ago
Still waiting for laptop able to run R1 locally...
y0eswddl 1y ago
Maybe you mean it'd be cheaper for companies to host centralized internal(ly trained) models... That seems to me more likely, more efficient to manage and more cost effective than individual laptop-local models. IMO, domain specific training is one of the areas I think LLMs can really shine.
onraglanroad 1y ago
> Just my opinion as a FORMER senior software dev (disabled now). I'm not sure what this means. Why would being disabled stop you being a senior software developer? I've known blind people who were great devs so I'm really not sure what disability would stop you working if you wanted to. Edit: by which I mean, you might have chosen to retire but the way you put it doesn't sound like that.

bilsbie 1y ago

Can you expand on this?

pruetj 1y ago

> purchase laptops with decent hardware specs > It costs nothing Seems like it does cost something?

automatic6131 1y ago
Quite, the typical 5 year depreciation on personal computing means a top-of-the-line $5k laptop works out to a ~$80/month spend... but it's on something you'd already spend for an employee
- datadrivenangel 1y ago
  $2k / 5 years is ~$30/mo, and you'll get a better experience spending another $25/mo on one of the AI services (or with enough people a small pile of H100s)

zippyman55 1y ago

Same here. I already have the computer for work, so marginally, it costs nothing and it meets 90 percent of my LLM needs. Here comes the down vote!

dist-epoch 1y ago

Electricity is not free. If you do the math, online LLMs are much cheaper. And this is before considering capabilities/speed.

the_snooze 1y ago
They're cheaper right now because they're operating at a loss. At some point, the bill will come due. Netflix used to be $8/month for as many streams and password-shares as you wanted for a catalog that met your media consumption needs. It was a great deal back then. But then the bill came due. Online LLM companies are positioning themselves to do the same bait-and-switch techbro BS we've seen over the last 15+ years.
- dist-epoch 1y ago
  Fundamentally it will always be cheaper to run LLMs in the cloud, because of batching. Unless somehow magically you'll have the need to run 1000 different prompts at the exact same time to also benefit from it locally. This is even without considering cloud GPUs which are much more efficient than local ones, especially from old hardware.
  - dns_snek 1y ago
    Yes they'll be cheaper to run, but will they be cheaper buy as a service? Because sooner or later these companies will be expected to produce eye-watering ROI to justify the risk of these moonshot investments and they won't be doing that by selling at cost.
    - kingstnap 1y ago
      Will they be cheaper to buy? Yes. You are effectively just buying compute with AI. From a simple correlational extrapolation compute has only gotten more cheaper over time. Massively so actually. From a more reasoned causal extrapolation hardware companies historically compete to bring the price of compute down. For AI this is extremely aggressive I might add. HotChips 2024 and 2025 had so much AI coverage. Nvidia is in an arms race with so many companies. All over the last few years we have literally only ever seen AI get cheaper for the same level or better. No one is releasing worse and more expensive AI right now. Literally just a few days ago Deepseek halved the price of V3.2. AI expenses have grown but that's because human's are extremely cognitively greedy. We value our time far more than compute efficiency.

mokoshhydro 1y ago

Still waiting for laptop able to run R1 locally...

y0eswddl 1y ago

Maybe you mean it'd be cheaper for companies to host centralized internal(ly trained) models... That seems to me more likely, more efficient to manage and more cost effective than individual laptop-local models. IMO, domain specific training is one of the areas I think LLMs can really shine.

onraglanroad 1y ago

> Just my opinion as a FORMER senior software dev (disabled now). I'm not sure what this means. Why would being disabled stop you being a senior software developer? I've known blind people who were great devs so I'm really not sure what disability would stop you working if you wanted to. Edit: by which I mean, you might have chosen to retire but the way you put it doesn't sound like that.