3 ms·
Pretty sure this "cost" is based on their retail price instead of actual inference cost.
by rfoo 2y ago
Pretty sure this "cost" is based on their retail price instead of actual inference cost.
- neuroelectron 2y agoYes that's correct and there's a bit of "pixel math" as well so take these numbers with a pinch of salt. Preliminary model sizes from the temporarily public HF repository puts the full model size at 8tb or roughly 80 H100s
- az226 2y agoI thought that was a fake.
- neuroelectron 2y agoI didn't hear that but it could be. But it doesn't matter really because there's so much more to consider in the cost, R&D, including all the supporting functions of a model like censorship and data capture and so on.
- ec109685 2y agoYeah and can run off peak, etc. Does seem to show an absolutely massive market for inference compute…