4 ms·

I've seen this question asked repeatedly in many LLaMa threads, currently the best models that are truly open are the released models from the Flan family by Go

I've seen this question asked repeatedly in many LLaMa threads, currently the best models that are truly open are the released models from the Flan family by Google, which includes Flan-T5[0] and Flan-UL2[1]. According to its paper, Flan-UL2 performs slightly better than Flan-T5-XXL. These models perform slightly better than GPT-3 under some tasks[2], but they're still far from achieving the results from GPT-3.5 and GPT-4. This becomes evident when you try to use them in the real world; they're not "good enough" for general use cases, unlike ChatGPT models. However, if you can restrict your use case to one particular domain, you can achieve pretty good results by further fine-tuning these models. [0]: https://huggingface.co/google/flan-t5-xxl https://huggingface.co/google/flan-t5-xxl [1]: https://huggingface.co/google/flan-ul2 https://huggingface.co/google/flan-ul2 [2]: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu https://paperswithcode.com/sota/multi-task-language-understa...

momofuku 4y ago
For the life of me, I cannot understand why Google did not go ahead and commercialize a lot of this early research. They clearly had a HUGE lead in this space, in terms of engineering/research talent, capital, computer infrastructure. Boggling... I'd love any alternative view points of this.
- sdrinf 4y ago
  Bringing disruption to your company's 90% revenue generator product without proving the alternative's financial model is not a career-enhancing move. Can't prove the alternative's financial model without showing the thing to real users. Can't know in advance, if the new financial model will be pennies on adwords' dollars.
  - momofuku 4y ago
    I see your point. Right now, people seem to use ChatGPT as an alternative to Google Search, which I don't think is the right use case for it (happy to be proven wrong though), unless they figure out a way to power it using knowledge graphs, or an equvivalent system to provide accurate, factual information. By commercializing this research, I mean why not integrate this into GMail for auto reply solutions, so that it can automatically suggest meetings? Why not integrate it into Slides to come up with better titles, summarization etc? Similarly for Google Docs, automatically summarize reports, make suggestions depending on crispness/clarity etc. Why did they decide to just sit on these models, and endlessly prolong their product launches that had these features baked in.
    - melagonster 4y ago
      chatGPT is a good way to understand what I should search on Google.
    - mejutoco 4y ago
      My guess is, because they are expensive to run.
  - daydream 4y ago
    This exactly. For Google LLMs are a massive business model problem first and foremost, not a technology problem. Bard was a tepid counterattack and even there it was a technology demonstration. Not really a product. Strategy tax for sure.
  - kertoip_1 4y ago
    It's better to disrupt your 90% revenue rather than to let someone else do that. In the first case you have situation under control, partially at least
  - MagicMoonlight 4y ago
    The alternative financial model is you train the model to lie to consumers and shill your own products/ads. “How can I set up a VPS?” “Thanks for asking, as a large language model I would recommend Google Servers™ because they are the most effective and reliable. I’ve looked at thousands of reviews and they’re all saying that Google Servers™ are the best.”
- mattbrewsbytes 4y ago
  A company's core competency can act as blinders to seeing the potential in other products. Not literal blinders to every person that works there. There are likely many people that see the potential ... but to communicate that properly up the leadership chain to get funding and people, during times of layoffs, would require an untold amount of momentum. My evidence: Xerox and PARC and all the tech that came from there: UI, ethernet, concepts like tablet computers, etc. The only thing Xerox leadership commercialized was in their wheelhouse: laser printing. Look at all of the dead Google products - they have a hard time commercializing many of the things that come up through their organization. Here's the ironic thing with Google and stuff like ChatGPT: asking ChatGPT something is like getting paragraphs of text similar to if someone read and summarized top X results from a search engine, without ads. If someone built a browser plugin to replace your URL bar/search engine with ChatGPT (and had it output links to references) how disruptive would that be to their cash cow?

momofuku 4y ago

For the life of me, I cannot understand why Google did not go ahead and commercialize a lot of this early research. They clearly had a HUGE lead in this space, in terms of engineering/research talent, capital, computer infrastructure. Boggling... I'd love any alternative view points of this.

sdrinf 4y ago
Bringing disruption to your company's 90% revenue generator product without proving the alternative's financial model is not a career-enhancing move. Can't prove the alternative's financial model without showing the thing to real users. Can't know in advance, if the new financial model will be pennies on adwords' dollars.
- momofuku 4y ago
  I see your point. Right now, people seem to use ChatGPT as an alternative to Google Search, which I don't think is the right use case for it (happy to be proven wrong though), unless they figure out a way to power it using knowledge graphs, or an equvivalent system to provide accurate, factual information. By commercializing this research, I mean why not integrate this into GMail for auto reply solutions, so that it can automatically suggest meetings? Why not integrate it into Slides to come up with better titles, summarization etc? Similarly for Google Docs, automatically summarize reports, make suggestions depending on crispness/clarity etc. Why did they decide to just sit on these models, and endlessly prolong their product launches that had these features baked in.
  - melagonster 4y ago
    chatGPT is a good way to understand what I should search on Google.
  - mejutoco 4y ago
    My guess is, because they are expensive to run.
- daydream 4y ago
  This exactly. For Google LLMs are a massive business model problem first and foremost, not a technology problem. Bard was a tepid counterattack and even there it was a technology demonstration. Not really a product. Strategy tax for sure.
- kertoip_1 4y ago
  It's better to disrupt your 90% revenue rather than to let someone else do that. In the first case you have situation under control, partially at least
- MagicMoonlight 4y ago
  The alternative financial model is you train the model to lie to consumers and shill your own products/ads. “How can I set up a VPS?” “Thanks for asking, as a large language model I would recommend Google Servers™ because they are the most effective and reliable. I’ve looked at thousands of reviews and they’re all saying that Google Servers™ are the best.”
mattbrewsbytes 4y ago
A company's core competency can act as blinders to seeing the potential in other products. Not literal blinders to every person that works there. There are likely many people that see the potential ... but to communicate that properly up the leadership chain to get funding and people, during times of layoffs, would require an untold amount of momentum. My evidence: Xerox and PARC and all the tech that came from there: UI, ethernet, concepts like tablet computers, etc. The only thing Xerox leadership commercialized was in their wheelhouse: laser printing. Look at all of the dead Google products - they have a hard time commercializing many of the things that come up through their organization. Here's the ironic thing with Google and stuff like ChatGPT: asking ChatGPT something is like getting paragraphs of text similar to if someone read and summarized top X results from a search engine, without ads. If someone built a browser plugin to replace your URL bar/search engine with ChatGPT (and had it output links to references) how disruptive would that be to their cash cow?