3 ms·
It's a chicken and egg scenario. The main problem with running inference at home is the lack of hardware. If the hardware was there more people would do it. And
by ModernMech 2y ago
It's a chicken and egg scenario. The main problem with running inference at home is the lack of hardware. If the hardware was there more people would do it. And it's not a problem if "enthusiasts" are the only ones using it because that's to be expected at this stage of the tech cycle. If the market is small just charge more, the enthusiasts will pay it. Once more enthusiasts are running inference at home, then the late adopters will eventually come along.
- m00x 2y agoMac minis are great for this. They're cheap-ish and they can run quite large models at a decent speed if you run it with an MLX backend.
- alganet 2y agomini _Pro_ are great for this, ones with large RAM upgrades. If you get the base 16GB mini, it will have more or less the same VRAM but way worse performance than an Arc. If you already have a PC, it makes sense to go for the cheapest 12GB card instead of a base mac mini.