4 ms·

I guess Pro is not supposed to be on par with GPT4. That would be Ultra coming out sometime in the first quarter. I’m going to reserve judgement till that is re

I guess Pro is not supposed to be on par with GPT4. That would be Ultra coming out sometime in the first quarter. I’m going to reserve judgement till that is released.

YetAnotherNick 3y ago
In my experience, Bard is not comparable to GPT-3.5 in terms of instruction following and it sometimes gets lost in complex situations and then the response quality drops significantly. While GPT-3.5 is a much better feel, if that is a word for evaluating LLMs. And Bard is just annoying if it can't complete a task. Also hallucinations are wild in Gemini pro compared to GPT-3.5.
- AuthConnectFail 3y ago
  any examples of that? my experience has been other way round (i don't have gpt-4 access so i am comparing chatgpt-3.5 with bard)
- ipsum2 3y ago
  I don't know why you were down voted for sharing your opinion on bard. I agree with you that bard is significantly worse than gpt 3.5.
nycdatasci 3y ago
Per LLM leaderboard, Bard (jan 24 - Gemini Pro) is on par with GPT 4: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... I think there’s bias in the types of prompts they’re getting. In my personal experience, Bard is useful for creative use cases but not good with reasoning or facts.
- EvgeniyZh 3y ago
  They are getting whatever you give them ;)
- skissane 3y ago
  Here is a simple maths problem that GPT-4 gets right but Bard (even the Gemini Pro version) consistently gets wrong: “What is one (short scale) centillion divided by the cube of a googol?” But you are right, we don’t know the types of prompts Chatbot Arena users are submitting. Maths problems like that are probably a small minority of usage. One other thing I notice: if you ask about controversial issues, both GPT-3.5/4 and Bard can get a bit “preachy” from a progressive perspective - but I personally find Bard to be noticeably more “preachy” than OpenAI at this (while still not reaching Llama levels)

I guess Pro is not supposed to be on par with GPT4. That would be Ultra coming out sometime in the first quarter. I’m going to reserve judgement till that is released.

YetAnotherNick 3y ago

In my experience, Bard is not comparable to GPT-3.5 in terms of instruction following and it sometimes gets lost in complex situations and then the response quality drops significantly. While GPT-3.5 is a much better feel, if that is a word for evaluating LLMs. And Bard is just annoying if it can't complete a task. Also hallucinations are wild in Gemini pro compared to GPT-3.5.

AuthConnectFail 3y ago
any examples of that? my experience has been other way round (i don't have gpt-4 access so i am comparing chatgpt-3.5 with bard)
ipsum2 3y ago
I don't know why you were down voted for sharing your opinion on bard. I agree with you that bard is significantly worse than gpt 3.5.

nycdatasci 3y ago

Per LLM leaderboard, Bard (jan 24 - Gemini Pro) is on par with GPT 4: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... I think there’s bias in the types of prompts they’re getting. In my personal experience, Bard is useful for creative use cases but not good with reasoning or facts.

EvgeniyZh 3y ago
They are getting whatever you give them ;)
skissane 3y ago
Here is a simple maths problem that GPT-4 gets right but Bard (even the Gemini Pro version) consistently gets wrong: “What is one (short scale) centillion divided by the cube of a googol?” But you are right, we don’t know the types of prompts Chatbot Arena users are submitting. Maths problems like that are probably a small minority of usage. One other thing I notice: if you ask about controversial issues, both GPT-3.5/4 and Bard can get a bit “preachy” from a progressive perspective - but I personally find Bard to be noticeably more “preachy” than OpenAI at this (while still not reaching Llama levels)