3 ms·
8 x h100 80's don't give you enough to run the latest 1tn + parameter models (especially at the context window lengths to be competitive with the frontier model
by htrp 6mo ago
8 x h100 80's don't give you enough to run the latest 1tn + parameter models (especially at the context window lengths to be competitive with the frontier models)
- dools 6mo agoVerda has B300 clusters, 8 for USD $55/hour in 10 minute billing blocks