3 ms·
Google has figured out how to scale search to the current number of requests per second Scaling a ChatGPT like product to that level would be challenging I ass
by fareesh 4y ago
Google has figured out how to scale search to the current number of requests per second
Scaling a ChatGPT like product to that level would be challenging I assume - and extremely expensive. Is that correct?
- freediver 4y agoGoing rate is about 3 cents per LLM query, so at Google scale of 100k qps, it would be ~$300MM/day. However you probably need to send only about 1/10 of current queries to LLM (as most are still not suitrable for LLM to asnwer) so that would cost only $30MM/day or about $9B/year which is peanuts for Google.
- fareesh 4y agoFrom what I have read, some people are of the view that it will affect their margins and ability to compete in other businesses. This is based on something Satya Nadella mentioned in an interview