2 ms·
It should work, I believe. And anything that doesn't fit you can leave on your system RAM. Looks like an H100 runs about $30K online for one. Are there any iss
by qingcharles 2y ago
It should work, I believe. And anything that doesn't fit you can leave on your system RAM.
Looks like an H100 runs about $30K online for one. Are there any issues with just sticking one of these in a stock desktop PC and running llama.cpp?
- joha4270 2y ago> Are there any issues with just sticking one of these in a stock desktop PC and running llama.cpp? Cooling might be a challenge. The H100 has a heatsink designed to make use of the case fans. So you need a fairly high airflow through a part which is itself passive. On a server this isn't too big a problem, you have fans in one end and GPU's blocking the exit on the other end, but in a desktop you probably need to get creative with cardboard/3d printed shrouds to force enough air through it.