3 ms·
Anyone have experience and suggestions for an AI accelerator? Think prototype consumer product with total cost preferably < $500, definitely less than $1000.
by colechristensen 3y ago
Anyone have experience and suggestions for an AI accelerator?
Think prototype consumer product with total cost preferably < $500, definitely less than $1000.
- jsheard 3y agoThe default answer is to get the biggest Nvidia gaming card you can afford, prioritizing VRAM size over speed. Ideally one of the 24GB ones.
- mirekrusin 3y agoRent or 3090, maybe used 4090 if you're lucky.
- Hugsun 3y agoYou can get very cheap tesla P40s with 24gb of ram. They are much much slower than the newer cards but offer decent value for running a local chatbot. I can't speak to the ease of configuration but know that some people have used these successfully.
- jononor 3y agoWhat is the workload?
- hedgehog 3y agoWhat else in on the BOM? Volume? At that price you likely want to use whatever resources are on the SoC that runs the thing and work around that. Feel free to e-mail me.
- wmf 3y agoAMD Hawk Point?
- dist-epoch 3y agoAll new CPUs will have so called NPUs inside them. For helping running models locally.
- JonChesterfield 3y agoI liked my 5700XT. That seems to be $200 now. Ran arbitrary code on it just fine. Lots of machine learning seems to be obsessed with amount of memory though and increasing that is likely to increase the price. Also HN doesn't like ROCm much, so there's that.