4 ms·
> Self-hosting an AI with your own hardware is probably just as cost-prohibitive, even if you don't value your time. In part because a ton of people will get th
by yellowapple 9mo ago
> Self-hosting an AI with your own hardware is probably just as cost-prohibitive, even if you don't value your time. In part because a ton of people will get this idea at the same time, impacting hardware prices even more. And the operating costs of AI seem significant. Would it even be possible to setup your own AI and achieve the same productivity level?
I know this is probably an annoying question, but… has the author actually tried self-hosting an AI with one's own hardware? I have; ollama (and various frontends thereof) makes it straightforward, and it's absolutely not cost-prohibitive — I've ran my share of LLMs even on laptops without dedicated GPUs at all, and while the experience wasn't great compared to the commercial options, it wasn't outright unusable, either. Locally-hosted LLMs are already finding their way into various applications; that's only going to get more viable over time, not less (unless the computing hardware industry takes a catastrophic nosedive, in which case AI affordability is arguably the least of our worries).
I'm sure the author understands this and is just being hyperbolic in the article's title, but the AI bubble bursting ≠ AI disappearing, for the same reason the dotcom bubble bursting ≠ the World Wide Web disappearing. The bubble will burst when AI shifts from being novel to being mundane, just as with any other technology-related bubble — and that entails a degree of affordability and ubiquity that's mutually exclusive with any notion of AI “disappearing”. Hopefully it'll mean companies being less motivated to shove AI “features” down everyone's throats, but the virtually-intelligent cat is already out of Pandora's box: the technology's here to stay, and I think it's presumptuous to think the race to the bottom w.r.t. cost is anywhere near the finish line.
- estimator7292 9mo agoIf you have a linux machine, you can just install ollama, ollama-cuda, or ollama-rocm. That's it. It runs out of the box. If your GPU is supported, that Just Works, too. Usually, anyway. I have an old dual Xeon server from about 2015. 32 2.4GHz cores and 128GB of RAM. It runs models painfully slow (and loud) but they run just fine. My modern Ryzen system from last year works out of the box with full AMD GPU support. I have yet to find a situation where ollama doesn't work at all out of the box. It literally just turns on and goes. Maybe slow, maybe without GPU, but by god you'll have an LLM running