4 ms·
You have two requirements that are at odds: * Macbook is not an option * Want to be able to mess around with some local LLMs. Your choices for a Window lapto
by FlyingAvatar 2y ago
You have two requirements that are at odds:
* Macbook is not an option
* Want to be able to mess around with some local LLMs.
Your choices for a Window laptop that can run a local LLM is either to get a large amount of system RAM and have it be abysmally slow, or to run a very tiny model on a discrete GPU which will (a) not be very good due to its small size and high quantization and (b) evaporate your battery life.
If you want to run local LLMs on a laptop and actually have them be useful, a Mac is currently the only real choice.
That said, with the money you save buying a Linux laptop instead, you can pay for a lot of tokens for whatever hosted LLM you want and it will be higher quality than what you could potentially run locally on a Mac.
- edtech_dev 2y agoRunning LLMs is more of a nice to have. If I can run something like DeepSeek-Coder-V2 even if it's a bit slow, I'll be happy.
- hn92726819 2y agoIf you have a powerful computer at home, you can also offload your ai work to it. It's still local in the sense it's your computer, but it would require network access.
- specproc 2y agoOP notes they switch between Linux and Windows. I've not tried local LLMs on Windows, but I do loads with 'em on a three-year-old Legion running Arch. That said, whilst small local models are nice for some use-cases, I'm leaning more towards APIs these days. I like the better selection of models and the ability to use them without bringing my machine to a halt.
- johann8384 2y agoPocketAI can be helpful: https://www.adlinktech.com/en/pocket-ai-with-nvidia-rtx-a500-egpu https://www.adlinktech.com/en/pocket-ai-with-nvidia-rtx-a500...
- kokada 2y ago> You have two requirements that are at odds: Not really now that we have the AMD Strix Halo: https://arstechnica.com/gadgets/2025/02/review-asus-rog-flow-z13-tablet-takes-the-asterisk-off-integrated-gpus/ https://arstechnica.com/gadgets/2025/02/review-asus-rog-flow.... The only available SKU right now is the above one that is a weird gaming tablet/laptop that seems to not be good in either (too heavy for a tablet, too cramped for laptop usage), but the performance is definitely there (similar performance of RTX 4060 for laptop, using a similar TDP of only the GPU for the whole APU) and you also have 32GB of unified memory for LLMs. Also, the chipset itself supports up to 128GB of RAM, so technically in future we could have an even better SKUs for LLMs (but nothing announced yet AFAIK).
- FlyingAvatar 2y agoCool, I will keep an eye on this.
- giantg2 2y agoYou have the third option of getting a USB processing unit, such as a Coral TPU.
- antifa 2y agoI'm going to second this, hard. You're much better off doing anybody's $10/month github copilot/codey/cursor plan, spend less to get a laptop that does everything else better, then in a year or 2 ask again to see if localLLMs have gotten better or if x86 laptops have gotten better. What I can do with localLLM on my MacBook is not worth paying extra for an x86 laptop that be heavier, hotter, louder, and less battery (especially if you're not going to play games).