3 ms·
They're not cheap, but they're not _that_ expensive compared to buying four NVLink'd Nvidia cards with a combined similar amount of VRAM. Plus you get a whole c
by Me1000 3y ago
They're not cheap, but they're not _that_ expensive compared to buying four NVLink'd Nvidia cards with a combined similar amount of VRAM. Plus you get a whole computer with it and you don't have to worry about casings and power supplies, etc. But yeah, you will be more compute constrained if you go down that route.
- brucethemoose2 3y agoThe price is similar to 2x RTX 8000s, or even A6000s, but yeah your point stands. Power efficiency is something too. You run into immense pain the moment you venture outside of llama inference though.