3 ms·
Let me know when you can run Opus on H100.
by lvl155 1y ago
Let me know when you can run Opus on H100.
- fragmede 1y agoI don't understand. Obviously I can't run Opus on an H100, only Anthropic can do that since they are the only ones with the model. I am assuming they are using H100s, and that an all-in cost for an H100 comes to less then $1000/month, and doing some back of the envelope math to say if they had a fleet of H100s at their disposal, that it would take six people running it flat out, for the $200/month plan to be profitable.
- WXLCKNO 1y agoRight but it probably takes like 8-10 H100s to run Claude Opus for inference just memory wise? I'm far from an expert just asking. Does "one" Claude Opus instance count as the full model being loaded onto however many GPUs it takes ?