10 ms·

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon

I think more Apple. It's not like Google or Microsoft would want to use LLaMA when they have fully capable models themselves. I wouldn't be surprised if Amazon does as well. Apple is the big laggard in terms of big tech and complex neural network models.

whimsicalism 3y ago
Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.
- dooraven 3y ago
  > Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
  - whimsicalism 3y ago
    I have no idea how you are so certain of that. Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
    - dooraven 3y ago
      We still don't have access to Imagen last I checked, it's still in restricted access. We don't have access to SoundStorm or MusicLM https://imagen.research.google/ https://imagen.research.google/ https://google-research.github.io/seanet/soundstorm/examples/ https://google-research.github.io/seanet/soundstorm/examples... https://google-research.github.io/seanet/musiclm/examples/ https://google-research.github.io/seanet/musiclm/examples/ Why would it be surprising that they have better models for resarch that they don't want to give out yet?
      - whimsicalism 3y ago
        Because I work in NLP so I have a good sense of the different capabilities of different firms and for the Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. The other things you are describing are just standard for research paper releases.
        dooraven 3y ago
        > Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz. Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype. Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
        whimsicalism 3y ago
        I can already tell you that PaLM is not anywhere near as good and PaLM-2 is at least not as good before RLHF. Not going to keep replying, believe what you want about Google's capabilities
        dooraven 3y ago
        ok now I am confused, as Meta themselves say Palm-2 is better than Llama 2? > Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L. https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000... If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already. Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
    - flangola7 3y ago
      > I have no idea how you are so certain of that. Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
  - WastingMyTime89 3y ago
    Do they? Considering how much was at stack in term of PR when OpenAI released ChatGPT, I would be surprised that Google didn’t put out the best they could.
    - freedomben 3y ago
      The other end of the PR stake was safety/alignment. If Google released a well functioning model, but it said some unsavory things or carried out requests that the public doesn't find agreeable, it could make Google look bad.
  - matt_holden 3y ago
    Comments like this remind me of the old-timers from IBM saying "but wait, we invented the PC! and the cloud! and..." Gotta put products in the market, or it didn't happen...
    - jefftk 3y ago
      It's fine not to give them public credit for in-house only things, but in this subthread we're speculating about whether Llama 2 would be useful to them, which does depend heavily on the quality of their internal models.
    - foobiekr 3y ago
      bringing back PLOSTFU culture might not actually be a bad thing.
    - cma 3y ago
      OpenAI seemingly downgraded ChatGPT 4 due to the expense of running it for pro customers (unless you run it through the API).
- chaxor 3y ago
  Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.
  - whimsicalism 3y ago
    I work in this field. I would love to see what you are basing these assertions off of. > they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc) Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
  - renewiltord 3y ago
    Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys. All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
    - jll29 3y ago
      I can assure you Google BERT isn't vaporware. It was probably a challenge to integrate it into search, but they did that. So your assertion has been refuted based on your use of "all", at the very least.
      - renewiltord 3y ago
        Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.
- galaxyLogic 3y ago
  I just googled "What is the order of object-fields in JavaScript" and the bard-answer said nothing about the differences between ES5 and ES6 and ES2020 how by now the order of object-fields in fact is deterministic. It seems it is not aware of the notion of historic development, perhaps its world-model is "static"? Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News". REF: https://www.stefanjudis.com/today-i-learned/property-order-is-predictable-in-javascript-objects-since-es2015/ https://www.stefanjudis.com/today-i-learned/property-order-i...
- deleted 3y ago
  [deleted]
- ankeshanand 3y ago
  Has anyone in this subthread actually read the papers and compared the benchmarks? LLama2 is behind PALM-2 on all major benchmarks, I mean they spell this out in the paper explicitly.
lacker 3y ago
I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license: v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE Just like Google scrapes the internet to improve their models, it might make sense to ingest outputs from other models to improve their models. This licensing prevents them from doing that. Using Llama to improve other LLMs is specifically forbidden, but Google will also be forbidden from using Llama to improve any other AI products they might be building.
- galaxyLogic 3y ago
  I can see their business logic but isn't it a bit like do not allow people (or bots) talk to each other, they might all get smarter. I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.
  - peddling-brink 3y ago
    > if the goal is to build better AI to serve the humanity It’s not.
  - DeathArrow 3y ago
    >but if the goal is to build better AI to serve the humanity Whose goal is that?
    - galaxyLogic 3y ago
      Google's. Do no evil they say
      - CamperBob2 3y ago
samwillis 3y ago
Apple would absolutely not want to use a competitors, or any other, public LLM. They want to own the whole stack, and will want to have their own secret source as part of it. It's not like they don't have the capital to invest in training...
- whimsicalism 3y ago
  Apple does not have the capability to train a LLM currently.
  - samwillis 3y ago
    I very much doubt that.
    - smoldesu 3y ago
      If they want to own the whole stack, I don't think they have much to work with. Their highest-end server chip is a duplex laptop SOC, with maxed-out memory that doesn't even match the lowest-end Grace CPU you can buy (nevermind a fully-networked GH200). Their consumer offerings are competitive, but I don't think Apple Silicon or CoreML is ready to seriously compete with Grace and CUDA.
      - samwillis 3y ago
        While Apple silicone may not be there for training, I think it's probably there for inference. I expect next years device models to launch with exclusive support for Apples own LLM based Siri.
        smoldesu 3y ago
        Sure. Haswell CPUs from 2014 are "there" for inference if they have AVX support and 8gb of RAM. Inferencing isn't the problem though, not on M1 or Macbooks from 2016. Scaling a desirable (and hopefully open) GPGPU programming interface is. This is bottlenecked by both hardware and software decisions Apple has made, making a "home grown" competitive model much more unlikely in my eyes. I agree that there is an incentive to put AI models on your OS. I just don't think Apple can own the whole stack if they want to play ball right now.
xbmcuser 3y ago
What makes you think that. Apple is the company that would be most successful at hiding something like this then introduce it as siri ai or something. Not that they are I am just saying Apple keeps everything close to its chest when it comes to products it might introduce in the future.
- whimsicalism 3y ago
  I work in the field and they just are not hiring the people they need to be hiring.
  - kossTKR 3y ago
    Interesting. The very early adoption of the neural engines in all Apple products would make you think that they had something brewing. Same with the relatively capable m1/2 GPU's. Various models and stable diffusion runs suprisingly fast on these devices and could be optimised to run much, much faster if Apple actually cared, but they weirdly seem not to.
reacharavindh 3y ago
Considering how much Apple likes to retain control, I’m almost sure they won’t want to use someone else’s model even if it were free in every sense of the word.

whimsicalism 3y ago

Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.

dooraven 3y ago
> Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
- whimsicalism 3y ago
  I have no idea how you are so certain of that. Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
  - dooraven 3y ago
    We still don't have access to Imagen last I checked, it's still in restricted access. We don't have access to SoundStorm or MusicLM https://imagen.research.google/ https://imagen.research.google/ https://google-research.github.io/seanet/soundstorm/examples/ https://google-research.github.io/seanet/soundstorm/examples... https://google-research.github.io/seanet/musiclm/examples/ https://google-research.github.io/seanet/musiclm/examples/ Why would it be surprising that they have better models for resarch that they don't want to give out yet?
    - whimsicalism 3y ago
      Because I work in NLP so I have a good sense of the different capabilities of different firms and for the Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. The other things you are describing are just standard for research paper releases.
      - dooraven 3y ago
        > Bard release, it would have made more sense for them to have a more limited release of a better model for PR reasons than what actually happened. Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz. Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype. Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
        whimsicalism 3y ago
        I can already tell you that PaLM is not anywhere near as good and PaLM-2 is at least not as good before RLHF. Not going to keep replying, believe what you want about Google's capabilities
        dooraven 3y ago
        ok now I am confused, as Meta themselves say Palm-2 is better than Llama 2? > Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L. https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000... If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already. Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
  - flangola7 3y ago
    > I have no idea how you are so certain of that. Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
- WastingMyTime89 3y ago
  Do they? Considering how much was at stack in term of PR when OpenAI released ChatGPT, I would be surprised that Google didn’t put out the best they could.
  - freedomben 3y ago
    The other end of the PR stake was safety/alignment. If Google released a well functioning model, but it said some unsavory things or carried out requests that the public doesn't find agreeable, it could make Google look bad.
- matt_holden 3y ago
  Comments like this remind me of the old-timers from IBM saying "but wait, we invented the PC! and the cloud! and..." Gotta put products in the market, or it didn't happen...
  - jefftk 3y ago
    It's fine not to give them public credit for in-house only things, but in this subthread we're speculating about whether Llama 2 would be useful to them, which does depend heavily on the quality of their internal models.
  - foobiekr 3y ago
    bringing back PLOSTFU culture might not actually be a bad thing.
  - cma 3y ago
    OpenAI seemingly downgraded ChatGPT 4 due to the expense of running it for pro customers (unless you run it through the API).
chaxor 3y ago
Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.
- whimsicalism 3y ago
  I work in this field. I would love to see what you are basing these assertions off of. > they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc) Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
- renewiltord 3y ago
  Google's LLMs are all vaporware. No one's ever seen them. They're supposedly mind-blowing but when they are released they always sound like lobotomized monkeys. All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
  - jll29 3y ago
    I can assure you Google BERT isn't vaporware. It was probably a challenge to integrate it into search, but they did that. So your assertion has been refuted based on your use of "all", at the very least.
    - renewiltord 3y ago
      Haha, that's right. Google has BERT. Their AI stuff isn't all vaporware. There's always BERT.
galaxyLogic 3y ago
I just googled "What is the order of object-fields in JavaScript" and the bard-answer said nothing about the differences between ES5 and ES6 and ES2020 how by now the order of object-fields in fact is deterministic. It seems it is not aware of the notion of historic development, perhaps its world-model is "static"? Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News". REF: https://www.stefanjudis.com/today-i-learned/property-order-is-predictable-in-javascript-objects-since-es2015/ https://www.stefanjudis.com/today-i-learned/property-order-i...
deleted 3y ago
[deleted]
ankeshanand 3y ago
Has anyone in this subthread actually read the papers and compared the benchmarks? LLama2 is behind PALM-2 on all major benchmarks, I mean they spell this out in the paper explicitly.

lacker 3y ago

I think Google or Microsoft probably would want to use LLaMa for various purposes like benchmarking and improving their own products. Check out this other condition from the license: v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof). https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE Just like Google scrapes the internet to improve their models, it might make sense to ingest outputs from other models to improve their models. This licensing prevents them from doing that. Using Llama to improve other LLMs is specifically forbidden, but Google will also be forbidden from using Llama to improve any other AI products they might be building.

galaxyLogic 3y ago
I can see their business logic but isn't it a bit like do not allow people (or bots) talk to each other, they might all get smarter. I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.
- peddling-brink 3y ago
  > if the goal is to build better AI to serve the humanity It’s not.
- DeathArrow 3y ago
  >but if the goal is to build better AI to serve the humanity Whose goal is that?
  - galaxyLogic 3y ago
    Google's. Do no evil they say
    - CamperBob2 3y ago

samwillis 3y ago

Apple would absolutely not want to use a competitors, or any other, public LLM. They want to own the whole stack, and will want to have their own secret source as part of it. It's not like they don't have the capital to invest in training...

whimsicalism 3y ago
Apple does not have the capability to train a LLM currently.
- samwillis 3y ago
  I very much doubt that.
  - smoldesu 3y ago
    If they want to own the whole stack, I don't think they have much to work with. Their highest-end server chip is a duplex laptop SOC, with maxed-out memory that doesn't even match the lowest-end Grace CPU you can buy (nevermind a fully-networked GH200). Their consumer offerings are competitive, but I don't think Apple Silicon or CoreML is ready to seriously compete with Grace and CUDA.
    - samwillis 3y ago
      While Apple silicone may not be there for training, I think it's probably there for inference. I expect next years device models to launch with exclusive support for Apples own LLM based Siri.
      - smoldesu 3y ago
        Sure. Haswell CPUs from 2014 are "there" for inference if they have AVX support and 8gb of RAM. Inferencing isn't the problem though, not on M1 or Macbooks from 2016. Scaling a desirable (and hopefully open) GPGPU programming interface is. This is bottlenecked by both hardware and software decisions Apple has made, making a "home grown" competitive model much more unlikely in my eyes. I agree that there is an incentive to put AI models on your OS. I just don't think Apple can own the whole stack if they want to play ball right now.

xbmcuser 3y ago

What makes you think that. Apple is the company that would be most successful at hiding something like this then introduce it as siri ai or something. Not that they are I am just saying Apple keeps everything close to its chest when it comes to products it might introduce in the future.

whimsicalism 3y ago
I work in the field and they just are not hiring the people they need to be hiring.
- kossTKR 3y ago
  Interesting. The very early adoption of the neural engines in all Apple products would make you think that they had something brewing. Same with the relatively capable m1/2 GPU's. Various models and stable diffusion runs suprisingly fast on these devices and could be optimised to run much, much faster if Apple actually cared, but they weirdly seem not to.

reacharavindh 3y ago

Considering how much Apple likes to retain control, I’m almost sure they won’t want to use someone else’s model even if it were free in every sense of the word.