11 ms·
How 'inference' is driving competition to Nvidia's AI chip dominance
- bookofjoe 2y agohttps://archive.ph/yAUpD https://archive.ph/yAUpD
- jsemrau 2y agoI think the key takeaway quotes are these: “The amount of inference compute needed is already 100x more” than it was when large language models started out, Huang said on last month’s earnings call. “And that’s just the beginning.” The cost of serving up responses from LLMs has fallen rapidly over the past two years, driven by a combination of more powerful chips, more efficient AI systems and intense competition between AI developers such as Google, OpenAI and Anthropic.
- deleted 2y ago[deleted]