3 ms·
So, does that mean that in general for the most modern high end LLM tools, to generate ~1500 tokens per seconds you need around $500k in hardware? Checking: An
by stingraycharles 1y ago
So, does that mean that in general for the most modern high end LLM tools, to generate ~1500 tokens per seconds you need around $500k in hardware?
Checking: Anthropic charges $70 per 1 million output tokens. @1500 tokens per second that would be around 10 cents per second, or around $8k per day.
The $500k sounds about right then, unless I’m mistaken.
- andruby 1y ago62 days to break even, that would be a great investment