2 ms·
How is throughput per watt affordable? Each card has a TDP of 275 watts, and a maximum draw of 375. If you try to run a big model like Llama-3 400B you're looki
by elorant 2y ago
How is throughput per watt affordable? Each card has a TDP of 275 watts, and a maximum draw of 375. If you try to run a big model like Llama-3 400B you're looking at a cluster with 1,8k cards, and a total power draw of nearly 500KWs per hour.