3 ms·

Based on the history of Google, it won't matter. The only good LLM they seem to put out is the LLM that isn't yet available to test. Then, when they do get ove

by iinnPP 3y ago

Based on the history of Google, it won't matter. The only good LLM they seem to put out is the LLM that isn't yet available to test. Then, when they do get over the 3.5 line, they will just graveyard it anyway. They've fully earned a reputation as a marketing parasite.

dartos 3y ago
Not sure why you’re being down voted. This is the impression I have of google as well. Especially in the AI space. Remember that “AI” scheduling a haircut or something that google presented in 2017?
- cubefox 3y ago
  Ah yes, 2017, when DeepMind, an Alphabet subsidiary, invented the T in GPT. Merely 100.000 citations so far.
  - psb217 3y ago
    The Transformer paper, ie "Attention is All You Need", was a Google Brain/Research paper, not a Deepmind paper.
    - cubefox 3y ago
      Thanks, I stand corrected.
  - dartos 3y ago
    You can invent a car tire. That doesn’t mean you’re any good at building cars.
itslennysfault 3y ago
It's honestly sad to watch. Google has the talent and research to be smoking everyone in this space, but it seems like they have zero product or marketing leadership/vision. For years they've been shipping half baked products built on killer tech that are destined to fail.
cubefox 3y ago
In the Gemini paper they already demonstrated that they beat GPT-4 on most important benchmarks, HellaSwag being an exception. See https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf#page=7 https://storage.googleapis.com/deepmind-media/gemini/gemini_... By the way, I linked to this report when it came out, but was labeled a "dupe" because someone else had already posted a link to the flashy website without any technical details.
- pants2 3y ago
  Google's misleading demonstration video shows they can't be trusted to honestly evaluate their own model. Until we can actually get our hands on Ultra and try for ourselves, I wouldn't give too much credit to their numbers.
  - replwoacause 3y ago
    It’s like Big Tobacco doing safety research into their cigarettes, cannot be trusted. Google is untrustworthy and has been for a long time.
  - sumedh 3y ago
    > Google's misleading demonstration video I cannot believe someone in Google approved that video but then again I cannot believe that a company would be launching 10 different chat apps.
- ianbicking 3y ago
  "Important benchmarks" aren't the same as being useful and reliable. I tried using Gemini Pro for tasks that GPT-3.5 does very well on and it fell apart completely. From reading the reviews of people using it, I get the impression Gemini Pro is good at writing marketing copy and other stuff that GPT-3 was good at, and I suspect a lot of benchmarks are over focused on that same text completion modality. Honestly Gemini Pro is so unimpressive I expect even GPT-3.5 will beat Gemini Ultra in terms of utility, at least for the next 6 months.

by iinnPP 3y ago

dartos 3y ago

Not sure why you’re being down voted. This is the impression I have of google as well. Especially in the AI space. Remember that “AI” scheduling a haircut or something that google presented in 2017?

cubefox 3y ago
Ah yes, 2017, when DeepMind, an Alphabet subsidiary, invented the T in GPT. Merely 100.000 citations so far.
- psb217 3y ago
  The Transformer paper, ie "Attention is All You Need", was a Google Brain/Research paper, not a Deepmind paper.
  - cubefox 3y ago
    Thanks, I stand corrected.
- dartos 3y ago
  You can invent a car tire. That doesn’t mean you’re any good at building cars.

itslennysfault 3y ago

It's honestly sad to watch. Google has the talent and research to be smoking everyone in this space, but it seems like they have zero product or marketing leadership/vision. For years they've been shipping half baked products built on killer tech that are destined to fail.

cubefox 3y ago

In the Gemini paper they already demonstrated that they beat GPT-4 on most important benchmarks, HellaSwag being an exception. See https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf#page=7 https://storage.googleapis.com/deepmind-media/gemini/gemini_... By the way, I linked to this report when it came out, but was labeled a "dupe" because someone else had already posted a link to the flashy website without any technical details.

pants2 3y ago
Google's misleading demonstration video shows they can't be trusted to honestly evaluate their own model. Until we can actually get our hands on Ultra and try for ourselves, I wouldn't give too much credit to their numbers.
- replwoacause 3y ago
  It’s like Big Tobacco doing safety research into their cigarettes, cannot be trusted. Google is untrustworthy and has been for a long time.
- sumedh 3y ago
  > Google's misleading demonstration video I cannot believe someone in Google approved that video but then again I cannot believe that a company would be launching 10 different chat apps.
ianbicking 3y ago
"Important benchmarks" aren't the same as being useful and reliable. I tried using Gemini Pro for tasks that GPT-3.5 does very well on and it fell apart completely. From reading the reviews of people using it, I get the impression Gemini Pro is good at writing marketing copy and other stuff that GPT-3 was good at, and I suspect a lot of benchmarks are over focused on that same text completion modality. Honestly Gemini Pro is so unimpressive I expect even GPT-3.5 will beat Gemini Ultra in terms of utility, at least for the next 6 months.