3 ms·
You don't necessarily, but each token costs money for the AI to spit out. And probably more money when that output is used as input later. Delegating to a libra
by IncreasePosts 6mo ago
You don't necessarily, but each token costs money for the AI to spit out. And probably more money when that output is used as input later. Delegating to a library makes sense financially.
- storus 6mo agoWith local inference on pretty decent local models we have nowadays (Qwen-3.5 and better) it's not much of a concern anymore.
- Bishonen88 6mo agowhat percentage of people is using local models for anything serious? I reckon single digits if even that. And for a corporate work environment, probably close to 0.
- walthamstow 6mo agoSure, if you've got a £5k laptop
- IncreasePosts 6mo agoSure it is - there's still an opportunity cost of spending tokens(time/energy) creating a library from scratch vs using a preexisting well understood API.