5 ms·

I’ve been hearing “Google has secret better models” for 7 months now. Maybe some UFOs in the hangers at Moffett Field too?

PeterStuer 3y ago
Would you believe OpenAI has vastly better models that they are not releasing publicly?
- whimsicalism 3y ago
  No
  - mirekrusin 3y ago
    GPT models were internally available 6-12 months before they've seen public beta, of course OpenAI has more capable internal models.
    - foobiekr 3y ago
      There's no reason to believe this. The training time and cost is so substantial that they are almost certainly building their next release, but it isn't sitting there rotting.
      - int_19h 3y ago
        Much of that training time is RLHF, the absence of which does not make the model less capable of carrying out useful tasks (indeed, in case of GPT-4, it actually made the model slightly less capable).
      - PeterStuer 3y ago
        OpenAI themselves have said they hat GPT-4 internally before they ever released the first version of ChatGPT.
airgapstopgap 3y ago
Do you realize that LLaMA-1 is just a very slightly smaller, comparably performing replication of Chinchilla [1], which DeepMind had completed a year prior to LLaMA's release? And has RLHF-ed into a suitable chatbot "Sparrow" [2] months earlier than ChatGPT was launched? To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models but they've arrived at those not through training and benchmarking but by divination and bullshitting. This, let us say, does not sound plausible. Then again, Microsoft uses LLaMA for research, and they should theoretically have some ability to get stuff from OpenAI. Evidently this isn't how any of this works, huh. 1. https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 2. https://en.wikipedia.org/wiki/Sparrow_(bot) https://en.wikipedia.org/wiki/Sparrow_(bot)
- foobiekr 3y ago
  Google _internally_ feels that they are way behind. Forget commenters on HN, literally all of the google employees that I know believe that the company is failing here.
  - airgapstopgap 3y ago
    This is not responsive to my arguments. Google can be arbitrarily far behind OpenAI or Anthropic, OP's idea that they feel threatened by LLaMA when they (well, Deepmind) have reached LLaMA level 18-10 months ago is still wrong.
deleted 3y ago
[deleted]

PeterStuer 3y ago

Would you believe OpenAI has vastly better models that they are not releasing publicly?

whimsicalism 3y ago
No
- mirekrusin 3y ago
  GPT models were internally available 6-12 months before they've seen public beta, of course OpenAI has more capable internal models.
  - foobiekr 3y ago
    There's no reason to believe this. The training time and cost is so substantial that they are almost certainly building their next release, but it isn't sitting there rotting.
    - int_19h 3y ago
      Much of that training time is RLHF, the absence of which does not make the model less capable of carrying out useful tasks (indeed, in case of GPT-4, it actually made the model slightly less capable).
    - PeterStuer 3y ago
      OpenAI themselves have said they hat GPT-4 internally before they ever released the first version of ChatGPT.

airgapstopgap 3y ago

Do you realize that LLaMA-1 is just a very slightly smaller, comparably performing replication of Chinchilla [1], which DeepMind had completed a year prior to LLaMA's release? And has RLHF-ed into a suitable chatbot "Sparrow" [2] months earlier than ChatGPT was launched? To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models but they've arrived at those not through training and benchmarking but by divination and bullshitting. This, let us say, does not sound plausible. Then again, Microsoft uses LLaMA for research, and they should theoretically have some ability to get stuff from OpenAI. Evidently this isn't how any of this works, huh. 1. https://arxiv.org/abs/2203.15556 https://arxiv.org/abs/2203.15556 2. https://en.wikipedia.org/wiki/Sparrow_(bot) https://en.wikipedia.org/wiki/Sparrow_(bot)

foobiekr 3y ago
Google _internally_ feels that they are way behind. Forget commenters on HN, literally all of the google employees that I know believe that the company is failing here.
- airgapstopgap 3y ago
  This is not responsive to my arguments. Google can be arbitrarily far behind OpenAI or Anthropic, OP's idea that they feel threatened by LLaMA when they (well, Deepmind) have reached LLaMA level 18-10 months ago is still wrong.

deleted 3y ago

[deleted]