4 ms·

On the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard

On the low end, google can sprinkle in well integrated cost-optimized 'pretty good' LLMs into all their products which already have billions of DAU. It's hard for OpenAI to compete here, their models have to be significantly better to encourage people to copy and paste into ChatGPT and yet still cost effective. The high end is more interesting in that I'm not sure Google has a response to Gpt4. Specialized Palm2 models are allegedly SOTA in certain verticals but they're not generally available so it's hard to say. It's possible Google could offer a beefy model to Google One subscribers or something but they seem to be focused on the low end for now at least in the B2C arena with specialized models for B2B. I honestly don't think OSS is a big factor business-wise but it's nice to see and certainly enables more participation in the ecosystem.

chaxor 3y ago
This idea that ChatGPT is better than the models that Google has is false; but I understand the perception. Deepmind has far better models than OpenAI, and the models OpenAI have made are based on the research done at Deepmind. For example, ChatGPT was not the first model trained by RLHF - that was Sparrow by Deepmind. The difference is that Google has been very disinterested in making products out of these systems, and didn't spend any effort to make them publicly accessible. Deepmind is still the leader in AI, and they have much better models - it's just going to take a while for people who don't study ML to realize this because Google hasn't released any of their good models for public use yet (and yeah, I'm not counting Bard, etc because the real model was probably put together by one guy in an afternoon at Google, and the rest of the time working was marketing).
visarga 3y ago
> I honestly don't think OSS is a big factor business-wise Lots of companies are evaluating fine-tuning strategies for <60B LLMs. Such models can be efficient, private, cheaper and adapted to the use case. They may not be generalists like GPT-4, but you need them for scaling up usage on high volume calls on the same task if you don't want to have to pay a fortune to OpenAI, or for processing confidential information such as in medical, B2B and other cases. Not to mention the fear that OpenAI might snoop on their private data and use it in their corpus.

chaxor 3y ago

This idea that ChatGPT is better than the models that Google has is false; but I understand the perception. Deepmind has far better models than OpenAI, and the models OpenAI have made are based on the research done at Deepmind. For example, ChatGPT was not the first model trained by RLHF - that was Sparrow by Deepmind. The difference is that Google has been very disinterested in making products out of these systems, and didn't spend any effort to make them publicly accessible. Deepmind is still the leader in AI, and they have much better models - it's just going to take a while for people who don't study ML to realize this because Google hasn't released any of their good models for public use yet (and yeah, I'm not counting Bard, etc because the real model was probably put together by one guy in an afternoon at Google, and the rest of the time working was marketing).

visarga 3y ago

> I honestly don't think OSS is a big factor business-wise Lots of companies are evaluating fine-tuning strategies for <60B LLMs. Such models can be efficient, private, cheaper and adapted to the use case. They may not be generalists like GPT-4, but you need them for scaling up usage on high volume calls on the same task if you don't want to have to pay a fortune to OpenAI, or for processing confidential information such as in medical, B2B and other cases. Not to mention the fear that OpenAI might snoop on their private data and use it in their corpus.