3 ms·

IMO this is exactly apples-to-apples comparison. They both represent SOTA of two firms trying for technically the same thing. Just because the models or the in

IMO this is exactly apples-to-apples comparison. They both represent SOTA of two firms trying for technically the same thing. Just because the models or the infrastructure aren't identical doesn't mean we should not be comparing those to the same standards. Where Bard gains in speed and accessibility, it looses in reasoning and response quality.

danpalmer 3y ago
On the contrary, Bard is a product not a model. If you want to see the cutting edge capabilities then comparing the GPT-4 API to the bigger PaLM2 APIs available on GCP is probably a more apples to apples comparison. Bard is more directly comparable to ChatGPT as a product in general, and since it doesn’t have swappable models, comparing it to the opt-in paid-only model isn’t really a direct comparison.
scarmig 3y ago
Bard represents SOTA in terms of optimizing for low cost; ChatGPT represents SOTA in terms of optimizing for accuracy. On the SOTA frontier, these two goals represent a tradeoff. ChatGPT could choose to go for lower accuracy for lower cost, while Google could for higher accuracy at higher cost. It's like comparing a buffet to a high end restaurant. Even if Bard were targeting accuracy, it'd still fall short of ChatGPT, but much less so than it does now. (That said, as a product strategy it's questionable: at some point, which I think Bard reaches, the loss in quality makes it more trouble than it's worth.)
- cfeduke 3y ago
  Is this state of the art in terms of fast, incorrect answers? An incorrect answer is often less valuable than no answer at all! The OpenAI strategy here then seems like a no brainer.
  - scarmig 3y ago
    I think there are two viable strategies here: make a model that is useful at the lowest possible cost and make a model that is maximally useful at high costs. Probably some spots in between them as well. Google's mistake is in thinking that ChatGPT was a maximally useful product at high cost. Right now, ChatGPT is a useful product at a high cost which is nonetheless the lowest possible cost for a useful model.
  - verdverm 3y ago
    I cancelled my OpenAI plus because why pay for something you cannot use because it is always slow, down, busy, or returning errors. You cannot build a reliable business on OpenAI APIs either ChatGPT also spouts falsehoods and makes mistakes on non-trivial problems, there is not much difference here. Both have enough issues that you have to be very careful with them, especially when building a product that will be user facing
    - BlackSwanMan 3y ago
      [dead]

danpalmer 3y ago

On the contrary, Bard is a product not a model. If you want to see the cutting edge capabilities then comparing the GPT-4 API to the bigger PaLM2 APIs available on GCP is probably a more apples to apples comparison. Bard is more directly comparable to ChatGPT as a product in general, and since it doesn’t have swappable models, comparing it to the opt-in paid-only model isn’t really a direct comparison.

scarmig 3y ago

Bard represents SOTA in terms of optimizing for low cost; ChatGPT represents SOTA in terms of optimizing for accuracy. On the SOTA frontier, these two goals represent a tradeoff. ChatGPT could choose to go for lower accuracy for lower cost, while Google could for higher accuracy at higher cost. It's like comparing a buffet to a high end restaurant. Even if Bard were targeting accuracy, it'd still fall short of ChatGPT, but much less so than it does now. (That said, as a product strategy it's questionable: at some point, which I think Bard reaches, the loss in quality makes it more trouble than it's worth.)

cfeduke 3y ago
Is this state of the art in terms of fast, incorrect answers? An incorrect answer is often less valuable than no answer at all! The OpenAI strategy here then seems like a no brainer.
- scarmig 3y ago
  I think there are two viable strategies here: make a model that is useful at the lowest possible cost and make a model that is maximally useful at high costs. Probably some spots in between them as well. Google's mistake is in thinking that ChatGPT was a maximally useful product at high cost. Right now, ChatGPT is a useful product at a high cost which is nonetheless the lowest possible cost for a useful model.
- verdverm 3y ago
  I cancelled my OpenAI plus because why pay for something you cannot use because it is always slow, down, busy, or returning errors. You cannot build a reliable business on OpenAI APIs either ChatGPT also spouts falsehoods and makes mistakes on non-trivial problems, there is not much difference here. Both have enough issues that you have to be very careful with them, especially when building a product that will be user facing
  - BlackSwanMan 3y ago
    [dead]