4 ms·
I would love a recommendation for an off the shelf "gpu server" good for most of this that I can run at home.
by beardedwizard 3y ago
I would love a recommendation for an off the shelf "gpu server" good for most of this that I can run at home.
- lakomen 3y agoI'm clueless about AI, but here's a benchmark list https://www.videocardbenchmark.net/high_end_gpus.html https://www.videocardbenchmark.net/high_end_gpus.html Imo the 4070 super is the best value and consumes the least amount of Watts, 220 in all the top 10. So anything with one and some ECC RAM aka AMD should be fine. Intel non-xeons need the expensive w680 boards and very specific RAM per board. ECC because you wrote server. We're professionals here after all, right?
- lardo 3y agoCivitAI has one https://civitai.com/builds https://civitai.com/builds
- macrolime 3y agoMac Studio or macbook pro if you want to run the larger models. Otherwise just a gaming pc with an rtx 4090 or a used rtx 3090 if you want something cheaper. A used dual 3090 can also be a good deal, but that is more in the build it yourself category than off the shelf.
- 101008 3y agoSorry if this is a silly question - I was never a Mac user, but I quick googled Mac Studio and it seems it's just the computer. Can I plug it to any monitor / use any keyboard and mouse, or do I need to use everything from Apple with it?
- timschmidt 3y agoAny monitor and keyboard will work, however Apple keyboards have a couple extra keys not present on Windows keyboards so require some key remapping to allow access to all typical shortcut key combinations.
- spectre3d 3y agoMainly to swap the Windows and Alt keys, which you can do in System Settings without any additional software. If you use a mouse with more than right-click and scroll wheel, with side buttons for example, then you’ll need extra software.
- macrolime 3y agoYou can, but with some caveats. Not all screen resolutions work well with MacOS, though using BetterDisplay it will still usually work. If you want touch id, it's better to get the Magic Keyboard with touch id.
- pksebben 3y agoI went the 4090 route myself recently, and I feel like all should be warned - memory is a major bottleneck. For a lot of tasks, folks may get more mileage out of multiple 3090s if they can get them set up to run parallel. Still waiting on being able to afford the next 4090 + egpu case et al. There are a lot of things this rig struggles with running OOM, even on inference with some of the more recent SD models.
- ckl1810 3y agoDepending on what models you want to run, RTX 4090 or RTX 3090 may not be enough. Grok-1 was running on a M2 Ultra with 196GB of ram. https://twitter.com/ibab_ml/status/1771340692364943750 https://twitter.com/ibab_ml/status/1771340692364943750
- batch12 3y agoSo I went really cheap and got a Thunderbolt dock for a gpu and a secondhand Intel nuc that supported it. So far it has met my needs.