23 ms·

>Each one costs ~$2 million, so that is $40 million. Pricing for exotic hardware that is not manufactured at scale is quite meaningless. They are selling token

>Each one costs ~$2 million, so that is $40 million. Pricing for exotic hardware that is not manufactured at scale is quite meaningless. They are selling tokens over an API. The token pricing is competitive with other token APIs.

jenny91 1y ago
I agree on the first. On the second: I would bet a lot of money that they aren't actually breaking even on their API (or even close to). They don't have a "pay as you go" per-token tier, it's all geared up to demonstrate use of their API as a novelty. They're probably burning cash on every single token. But their valuation and hype has surely gone way up since they got onto LLMs.
- bubblethink 1y ago
  They seem to have dev tier pricing (https://inference-docs.cerebras.ai/support/pricing https://inference-docs.cerebras.ai/support/pricing). It's likely that they don't make much money on this and only make money on large enterprise contracts.
ryao 1y ago
Last year, I took the time to read through public documents and estimated that their annual production was limited to ~300 wafers per year from TSMC. That is not Nvidia level scale, but it is scale. There are many companies that sell tokens from an API and many more that need hardware to compute tokens. Cerebras posted a comparison of hardware options for these companies, so evaluating it as such is meaningful. It is perhaps less meaningful to the average person who cannot afford the barrier to entry to afford this hardware, but there are plenty of people curious what the options are for the companies that sell tokens through APIs, as those impact available capacity.
- latchkey 1y ago
  > There are many companies that sell tokens from an API I was just at Dell Tech World and they proudly displayed a slide during the CTO keynote that said: "Cost per token decreased 4 orders of magnitude" Personally speaking, not a business I'd want to get into.
  - ryao 1y ago
    Some context is needed for this. The only way to get a 4 orders of magnitude difference would be to compare incomparable things, like OpenAI’s most expensive model versus llama 3.1 8B.

jenny91 1y ago

I agree on the first. On the second: I would bet a lot of money that they aren't actually breaking even on their API (or even close to). They don't have a "pay as you go" per-token tier, it's all geared up to demonstrate use of their API as a novelty. They're probably burning cash on every single token. But their valuation and hype has surely gone way up since they got onto LLMs.

bubblethink 1y ago
They seem to have dev tier pricing (https://inference-docs.cerebras.ai/support/pricing https://inference-docs.cerebras.ai/support/pricing). It's likely that they don't make much money on this and only make money on large enterprise contracts.

ryao 1y ago

Last year, I took the time to read through public documents and estimated that their annual production was limited to ~300 wafers per year from TSMC. That is not Nvidia level scale, but it is scale. There are many companies that sell tokens from an API and many more that need hardware to compute tokens. Cerebras posted a comparison of hardware options for these companies, so evaluating it as such is meaningful. It is perhaps less meaningful to the average person who cannot afford the barrier to entry to afford this hardware, but there are plenty of people curious what the options are for the companies that sell tokens through APIs, as those impact available capacity.

latchkey 1y ago
> There are many companies that sell tokens from an API I was just at Dell Tech World and they proudly displayed a slide during the CTO keynote that said: "Cost per token decreased 4 orders of magnitude" Personally speaking, not a business I'd want to get into.
- ryao 1y ago
  Some context is needed for this. The only way to get a 4 orders of magnitude difference would be to compare incomparable things, like OpenAI’s most expensive model versus llama 3.1 8B.