5 ms·

Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well.

Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.

dooraven 3y ago
> Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
- whimsicalism 3y ago
  I have no idea how you are so certain of that. Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
  - dooraven 3y ago
    We still don't have access to Imagen last I checked, it's still in restricted access. We don't have access to SoundStorm or MusicLM https://imagen.research.google/ https://imagen.research.google/ https://google-research.github.io/seanet/soundstorm/examples/ https://google-research.github.io/seanet/soundstorm/examples... https://google-research.github.io/seanet/musiclm/examples/ https://google-research.github.io/seanet/musiclm/examples/ Why would it be surprising that they have better models for resarch that they don't want to give out yet?
    - whimsicalism 3y ago
      Because I work in NLP so I have a good sense of the different capabilities of different firms and for the Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. The other things you are describing are just standard for research paper releases.
      - dooraven 3y ago
        > Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz. Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype. Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
        whimsicalism 3y ago
        I can already tell you that PaLM is not anywhere near as good and PaLM-2 is at least not as good before RLHF. Not going to keep replying, believe what you want about Google's capabilities
        dooraven 3y ago
        ok now I am confused, as Meta themselves say Palm-2 is better than Llama 2? > Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L. https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000... If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already. Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
        neonbjb 3y ago
        @dooraven - I also work in ML (including recently working at Google) and I agree with @whimsicalism. You seem to be under the mistaken belief that: 1. Google has competent high-level organization that effectively sets and pursues long term goals. 2. There is some advantage to developing a highly capable LLM but not releasing it. (2) could be the case if Google had built an extremely large model which was too expensive to deploy. Having been privy to what they had been working on up until mid-2022 and knowing how much work, compute and planning goes into extremely large models, this would very much surprise me. Note: I did not have much visibility into what deepmind was up to. Maybe they had something.
  - flangola7 3y ago
    > I have no idea how you are so certain of that. Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
- WastingMyTime89 3y ago
  Do they? Considering how much was at stack in term of PR when OpenAI released ChatGPT, I would be surprised that Google didn’t put out the best they could.
  - freedomben 3y ago
    The other end of the PR stake was safety/alignment. If Google released a well functioning model, but it said some unsavory things or carried out requests that the public doesn't find agreeable, it could make Google look bad.
- matt_holden 3y ago
  Comments like this remind me of the old-timers from IBM saying "but wait, we invented the PC! and the cloud! and..." Gotta put products in the market, or it didn't happen...
  - jefftk 3y ago
    It's fine not to give them public credit for in-house only things, but in this subthread we're speculating about whether Llama 2 would be useful to them, which does depend heavily on the quality of their internal models.
  - foobiekr 3y ago
    bringing back PLOSTFU culture might not actually be a bad thing.
  - cma 3y ago
    OpenAI seemingly downgraded ChatGPT 4 due to the expense of running it for pro customers (unless you run it through the API).
chaxor 3y ago
Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.
- whimsicalism 3y ago
  I work in this field. I would love to see what you are basing these assertions off of. > they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc) Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
- renewiltord 3y ago
  Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys. All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
  - jll29 3y ago
    I can assure you Google BERT isn't vaporware. It was probably a challenge to integrate it into search, but they did that. So your assertion has been refuted based on your use of "all", at the very least.
    - renewiltord 3y ago
      Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.
galaxyLogic 3y ago
I just googled "What is the order of object-fields in JavaScript" and the bard-answer said nothing about the differences between ES5 and ES6 and ES2020 how by now the order of object-fields in fact is deterministic. It seems it is not aware of the notion of historic development, perhaps its world-model is "static"? Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News". REF: https://www.stefanjudis.com/today-i-learned/property-order-is-predictable-in-javascript-objects-since-es2015/ https://www.stefanjudis.com/today-i-learned/property-order-i...
deleted 3y ago
[deleted]
ankeshanand 3y ago
Has anyone in this subthread actually read the papers and compared the benchmarks? LLama2 is behind PALM-2 on all major benchmarks, I mean they spell this out in the paper explicitly.

dooraven 3y ago

> Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. Google's publically available model isn't as capable. But they certainly have models that are far better already in house.

whimsicalism 3y ago
I have no idea how you are so certain of that. Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
- dooraven 3y ago
  We still don't have access to Imagen last I checked, it's still in restricted access. We don't have access to SoundStorm or MusicLM https://imagen.research.google/ https://imagen.research.google/ https://google-research.github.io/seanet/soundstorm/examples/ https://google-research.github.io/seanet/soundstorm/examples... https://google-research.github.io/seanet/musiclm/examples/ https://google-research.github.io/seanet/musiclm/examples/ Why would it be surprising that they have better models for resarch that they don't want to give out yet?
  - whimsicalism 3y ago
    Because I work in NLP so I have a good sense of the different capabilities of different firms and for the Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. The other things you are describing are just standard for research paper releases.
    - dooraven 3y ago
      > Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz. Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype. Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
      - whimsicalism 3y ago
        I can already tell you that PaLM is not anywhere near as good and PaLM-2 is at least not as good before RLHF. Not going to keep replying, believe what you want about Google's capabilities
        dooraven 3y ago
        ok now I am confused, as Meta themselves say Palm-2 is better than Llama 2? > Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L. https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000... If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already. Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
        neonbjb 3y ago
        @dooraven - I also work in ML (including recently working at Google) and I agree with @whimsicalism. You seem to be under the mistaken belief that: 1. Google has competent high-level organization that effectively sets and pursues long term goals. 2. There is some advantage to developing a highly capable LLM but not releasing it. (2) could be the case if Google had built an extremely large model which was too expensive to deploy. Having been privy to what they had been working on up until mid-2022 and knowing how much work, compute and planning goes into extremely large models, this would very much surprise me. Note: I did not have much visibility into what deepmind was up to. Maybe they had something.
- flangola7 3y ago
  > I have no idea how you are so certain of that. Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
WastingMyTime89 3y ago
Do they? Considering how much was at stack in term of PR when OpenAI released ChatGPT, I would be surprised that Google didn’t put out the best they could.
- freedomben 3y ago
  The other end of the PR stake was safety/alignment. If Google released a well functioning model, but it said some unsavory things or carried out requests that the public doesn't find agreeable, it could make Google look bad.
matt_holden 3y ago
Comments like this remind me of the old-timers from IBM saying "but wait, we invented the PC! and the cloud! and..." Gotta put products in the market, or it didn't happen...
- jefftk 3y ago
  It's fine not to give them public credit for in-house only things, but in this subthread we're speculating about whether Llama 2 would be useful to them, which does depend heavily on the quality of their internal models.
- foobiekr 3y ago
  bringing back PLOSTFU culture might not actually be a bad thing.
- cma 3y ago
  OpenAI seemingly downgraded ChatGPT 4 due to the expense of running it for pro customers (unless you run it through the API).

chaxor 3y ago

Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.

whimsicalism 3y ago
I work in this field. I would love to see what you are basing these assertions off of. > they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc) Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
renewiltord 3y ago
Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys. All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
- jll29 3y ago
  I can assure you Google BERT isn't vaporware. It was probably a challenge to integrate it into search, but they did that. So your assertion has been refuted based on your use of "all", at the very least.
  - renewiltord 3y ago
    Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.

galaxyLogic 3y ago

I just googled "What is the order of object-fields in JavaScript" and the bard-answer said nothing about the differences between ES5 and ES6 and ES2020 how by now the order of object-fields in fact is deterministic. It seems it is not aware of the notion of historic development, perhaps its world-model is "static"? Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News". REF: https://www.stefanjudis.com/today-i-learned/property-order-is-predictable-in-javascript-objects-since-es2015/ https://www.stefanjudis.com/today-i-learned/property-order-i...

deleted 3y ago

[deleted]

ankeshanand 3y ago

Has anyone in this subthread actually read the papers and compared the benchmarks? LLama2 is behind PALM-2 on all major benchmarks, I mean they spell this out in the paper explicitly.