2 ms·
I'd estimate that running a large language model like GPT-3.5 or Lambda is currently 100x-1000x both more expensive and slower to run inference through than a l
by freediver 4y ago
I'd estimate that running a large language model like GPT-3.5 or Lambda is currently 100x-1000x both more expensive and slower to run inference through than a language model like BERT.
So deploying it at Google scale is not viable (yet).
There is also the question of incentives. If an LLM model can return exactly what the user asked for, where do you put ads? Before the answer, after the answer or inside the answer? Any of those is a bad outcome for the user wanting just the answer.
We already witnessed the failure to monetize Alexa with ads - in a purely question answering setting, users will not tolerate anything but the answer itself. Thus, the business model for this needs to be paid AI/search, and Google would be facing innovator's dillema. If I was writing a book about Google, I would love to witness the VP level meetings at Google at this moment.
- ramraj07 4y agoI can easily see how to monetize it in a minute - whenever the answer involves a recommendation that has monetary value (a product or service) you get the option of choosing what product label to recommend and there’s the monetization angle.
- freediver 4y agoProblem with this is a) product or service recommendations are a tiny fraction of queries and b) gpt3 and any llm would be pretty bad at doing this, because they would exclude any recent product/services (they are trained on data from 1-2 years ago) and c) if the output of llm is changed in a way to replace llm’s recommendation with a paid one, why bother with having a llm in the first place?