4 ms·

Seems like Bard is still way behind GPT-4 though. GPT-4 gives far superior results in most questions I've tried. I'm interested in comparing Google's Duet AI w

by umvi 3y ago

Seems like Bard is still way behind GPT-4 though. GPT-4 gives far superior results in most questions I've tried. I'm interested in comparing Google's Duet AI with GitHub Copilot but so far seems like the waiting list is taking forever.

danpalmer 3y ago
I'm not sure Bard and GPT-4 are quite an apples-to-apples comparison though. GPT-4 is restricted to paying users, and is notable for how slow it is, whereas Bard is free to use, widely available (and becoming more so), and relatively fast. In other words, if Google had a GPT-4 quality model I'm not sure they would ship it for Bard as I think the cost would be too high for free use and the UX debatable.
- timthelion 3y ago
  How is Bard widely available. ChatGPT is available worldwide, Bard isn't in Europe yet.
  - telotortium 3y ago
    There's a good chance ChatGPT gets banned from Europe, whereas Google, despite its fines by EU authorities (most of which are for antitrust), can at least demonstrate that it's set up and continues to maintain GDPR compliance.
  - danpalmer 3y ago
    Bard is available in 180 countries. https://support.google.com/bard/answer/13575153?hl=en https://support.google.com/bard/answer/13575153?hl=en
    - acatton 3y ago
      Why is basically almost all the countries in the world except the EU countries. GP comment about "bard is still not available in europe" still stands. (Snapshot of the page at the time this comment was written: https://archive.is/hScBl https://archive.is/hScBl )
      - danpalmer 3y ago
        If we're going to be pedantic, then "bard is still not available in europe" is not true as it's available in the UK which is in Europe. I get the general point, but I would say that "everywhere but the EU" is very much "widely available".
    - progbits 3y ago
      Yes, basically everywhere except europe, likely due to regulatory concerns. (Would be interested to know what precisely, but the page doesn't say. Any guesses?)
- MaxikCZ 3y ago
  IMO this is exactly apples-to-apples comparison. They both represent SOTA of two firms trying for technically the same thing. Just because the models or the infrastructure aren't identical doesn't mean we should not be comparing those to the same standards. Where Bard gains in speed and accessibility, it looses in reasoning and response quality.
  - danpalmer 3y ago
    On the contrary, Bard is a product not a model. If you want to see the cutting edge capabilities then comparing the GPT-4 API to the bigger PaLM2 APIs available on GCP is probably a more apples to apples comparison. Bard is more directly comparable to ChatGPT as a product in general, and since it doesn’t have swappable models, comparing it to the opt-in paid-only model isn’t really a direct comparison.
  - scarmig 3y ago
    Bard represents SOTA in terms of optimizing for low cost; ChatGPT represents SOTA in terms of optimizing for accuracy. On the SOTA frontier, these two goals represent a tradeoff. ChatGPT could choose to go for lower accuracy for lower cost, while Google could for higher accuracy at higher cost. It's like comparing a buffet to a high end restaurant. Even if Bard were targeting accuracy, it'd still fall short of ChatGPT, but much less so than it does now. (That said, as a product strategy it's questionable: at some point, which I think Bard reaches, the loss in quality makes it more trouble than it's worth.)
    - cfeduke 3y ago
      Is this state of the art in terms of fast, incorrect answers? An incorrect answer is often less valuable than no answer at all! The OpenAI strategy here then seems like a no brainer.
      - scarmig 3y ago
        I think there are two viable strategies here: make a model that is useful at the lowest possible cost and make a model that is maximally useful at high costs. Probably some spots in between them as well. Google's mistake is in thinking that ChatGPT was a maximally useful product at high cost. Right now, ChatGPT is a useful product at a high cost which is nonetheless the lowest possible cost for a useful model.
- deleted 3y ago
  [deleted]

by umvi 3y ago

danpalmer 3y ago

I'm not sure Bard and GPT-4 are quite an apples-to-apples comparison though. GPT-4 is restricted to paying users, and is notable for how slow it is, whereas Bard is free to use, widely available (and becoming more so), and relatively fast. In other words, if Google had a GPT-4 quality model I'm not sure they would ship it for Bard as I think the cost would be too high for free use and the UX debatable.

timthelion 3y ago
How is Bard widely available. ChatGPT is available worldwide, Bard isn't in Europe yet.
- telotortium 3y ago
  There's a good chance ChatGPT gets banned from Europe, whereas Google, despite its fines by EU authorities (most of which are for antitrust), can at least demonstrate that it's set up and continues to maintain GDPR compliance.
- danpalmer 3y ago
  Bard is available in 180 countries. https://support.google.com/bard/answer/13575153?hl=en https://support.google.com/bard/answer/13575153?hl=en
  - acatton 3y ago
    Why is basically almost all the countries in the world except the EU countries. GP comment about "bard is still not available in europe" still stands. (Snapshot of the page at the time this comment was written: https://archive.is/hScBl https://archive.is/hScBl )
    - danpalmer 3y ago
      If we're going to be pedantic, then "bard is still not available in europe" is not true as it's available in the UK which is in Europe. I get the general point, but I would say that "everywhere but the EU" is very much "widely available".
  - progbits 3y ago
    Yes, basically everywhere except europe, likely due to regulatory concerns. (Would be interested to know what precisely, but the page doesn't say. Any guesses?)
MaxikCZ 3y ago
IMO this is exactly apples-to-apples comparison. They both represent SOTA of two firms trying for technically the same thing. Just because the models or the infrastructure aren't identical doesn't mean we should not be comparing those to the same standards. Where Bard gains in speed and accessibility, it looses in reasoning and response quality.
- danpalmer 3y ago
  On the contrary, Bard is a product not a model. If you want to see the cutting edge capabilities then comparing the GPT-4 API to the bigger PaLM2 APIs available on GCP is probably a more apples to apples comparison. Bard is more directly comparable to ChatGPT as a product in general, and since it doesn’t have swappable models, comparing it to the opt-in paid-only model isn’t really a direct comparison.
- scarmig 3y ago
  Bard represents SOTA in terms of optimizing for low cost; ChatGPT represents SOTA in terms of optimizing for accuracy. On the SOTA frontier, these two goals represent a tradeoff. ChatGPT could choose to go for lower accuracy for lower cost, while Google could for higher accuracy at higher cost. It's like comparing a buffet to a high end restaurant. Even if Bard were targeting accuracy, it'd still fall short of ChatGPT, but much less so than it does now. (That said, as a product strategy it's questionable: at some point, which I think Bard reaches, the loss in quality makes it more trouble than it's worth.)
  - cfeduke 3y ago
    Is this state of the art in terms of fast, incorrect answers? An incorrect answer is often less valuable than no answer at all! The OpenAI strategy here then seems like a no brainer.
    - scarmig 3y ago
      I think there are two viable strategies here: make a model that is useful at the lowest possible cost and make a model that is maximally useful at high costs. Probably some spots in between them as well. Google's mistake is in thinking that ChatGPT was a maximally useful product at high cost. Right now, ChatGPT is a useful product at a high cost which is nonetheless the lowest possible cost for a useful model.
deleted 3y ago
[deleted]