4 ms·
...for all those users with H100s.
by snovv_crash 2y ago
...for all those users with H100s.
- rfoo 2y ago... which is currently the most cost-efficient and environment-friendly way to do LLM inference [0]. [0] Small footprint time: before B100 ships; for actually large language models; for prefill only; may cause cancer in California.
- stale2002 2y agoIndeed. Anyone who is doing anything important or at scale would be at least renting those, or even using an abstracted service that is on top of another service. Those cost savings allow people to train things for cheaper, causing those cost savings to benefit almost everyone doing important stuff in the space.