4 ms·

I have a Gemini Pro trial, have been paying for ChatGPT since premium launched and as of last month started paying for Claude Opus. Claude’s context window for

by _fw 2y ago

I have a Gemini Pro trial, have been paying for ChatGPT since premium launched and as of last month started paying for Claude Opus. Claude’s context window for Opus is remarkable and I’d say that GPT4 is vastly inferior as a result. The second Anthropic ship Claude with a sandbox that can use python (or equiv) to take care of tasks, I’d happily end my ChatGPT subscription. GPT4 has been here for over a year and it shows. But until other assistants can manipulate Excel data as easily… I’m stuck paying for it. Claude is amazing for summarisation, copywriting and reasoned thinking. I love it.

free_energy_min 2y ago
Kagi (paid privacy focused search engine) allows access to all of these for $25 a month. Only downside I’ve found is it doesn’t save chat history for privacy reasons, but you can download them. https://help.kagi.com/kagi/ai/assistant.html https://help.kagi.com/kagi/ai/assistant.html
- siamese_puff 2y ago
  Is the context window reduced?
  - ukuina 2y ago
    Yes, input context window is reduced to 16k characters (not tokens). https://help.kagi.com/kagi/ai/assistant.html#tips-and-tricks-for-using-kagi-assistant https://help.kagi.com/kagi/ai/assistant.html#tips-and-tricks...
teamspirit 2y ago
That’s interesting, I found Opus to be underwhelming compared to GPT4. Hallucinations occurred as often, if not slightly more, and overall I found Opus to be not as “creative” when writing revisions. That and the rate limit resetting at midnight as compared to GPT4 being something like an hour or two made me return to old faithful. Maybe I should give it another try.
- whimsicalism 2y ago
  I enjoy chatting with Claude (opus) more, but still find GPT4 is better for coding. I also frankly will not extensively use a model that I can't turn off data storage for.
CSMastermind 2y ago
I got into the habit of running every query I do through all of the SOTA models so I can directly compare the results. This was originally GPT-4, Claude 3 Opus, and Gemini Advanced. I recently added Meta AI when they launched. Right now I've sent 486 queries through the first three systems. The clearest pattern to emerge is that Gemini is terrible, not on par with the other two. There hasn't been a single query that it was the only model who did well. Around 1/4 of the time it gives a clearly inferior answer to the others. But between GPT-4 and Claude it's less clear. 31 of the 486 queries Claude provided a significantly better answer than the other two but 20 times GPT-4 provided the significantly better answer. I do think that Claude is a slightly better model but right now it's not a clear enough advantage that I'd recommend it generally. I will say you can probably cancel you Gemini subscription if you're using it though.
- theturtletalks 2y ago
  One thing I would say about Opus over ChatGPT is that it can return code without placeholders whereas ChatGPT will always have placeholders. It wasn't always that way though and it seems like cutting corners.
  - whimsicalism 2y ago
    interesting, this was a problem around the turn of the year for me but chatgpt has stopped doing it now
- tomComb 2y ago
  Why would you compare Gemini pro with the other two - they are not in the same category. Did you mean to say advanced?
  - chatmasta 2y ago
    Why isn’t the model offered by Google in the same category? Or did I misunderstand their latest naming fiasco and “Gemini Pro” is not actually their latest model? EDIT: It looks like Gemini Ultra 1.0 > Gemini Pro 1.5 > Gemini Pro 1.0… is that true? What a horrible versioning scheme.

by _fw 2y ago

free_energy_min 2y ago

Kagi (paid privacy focused search engine) allows access to all of these for $25 a month. Only downside I’ve found is it doesn’t save chat history for privacy reasons, but you can download them. https://help.kagi.com/kagi/ai/assistant.html https://help.kagi.com/kagi/ai/assistant.html

siamese_puff 2y ago
Is the context window reduced?
- ukuina 2y ago
  Yes, input context window is reduced to 16k characters (not tokens). https://help.kagi.com/kagi/ai/assistant.html#tips-and-tricks-for-using-kagi-assistant https://help.kagi.com/kagi/ai/assistant.html#tips-and-tricks...

teamspirit 2y ago

That’s interesting, I found Opus to be underwhelming compared to GPT4. Hallucinations occurred as often, if not slightly more, and overall I found Opus to be not as “creative” when writing revisions. That and the rate limit resetting at midnight as compared to GPT4 being something like an hour or two made me return to old faithful. Maybe I should give it another try.

whimsicalism 2y ago
I enjoy chatting with Claude (opus) more, but still find GPT4 is better for coding. I also frankly will not extensively use a model that I can't turn off data storage for.

CSMastermind 2y ago

I got into the habit of running every query I do through all of the SOTA models so I can directly compare the results. This was originally GPT-4, Claude 3 Opus, and Gemini Advanced. I recently added Meta AI when they launched. Right now I've sent 486 queries through the first three systems. The clearest pattern to emerge is that Gemini is terrible, not on par with the other two. There hasn't been a single query that it was the only model who did well. Around 1/4 of the time it gives a clearly inferior answer to the others. But between GPT-4 and Claude it's less clear. 31 of the 486 queries Claude provided a significantly better answer than the other two but 20 times GPT-4 provided the significantly better answer. I do think that Claude is a slightly better model but right now it's not a clear enough advantage that I'd recommend it generally. I will say you can probably cancel you Gemini subscription if you're using it though.

theturtletalks 2y ago
One thing I would say about Opus over ChatGPT is that it can return code without placeholders whereas ChatGPT will always have placeholders. It wasn't always that way though and it seems like cutting corners.
- whimsicalism 2y ago
  interesting, this was a problem around the turn of the year for me but chatgpt has stopped doing it now
tomComb 2y ago
Why would you compare Gemini pro with the other two - they are not in the same category. Did you mean to say advanced?
- chatmasta 2y ago
  Why isn’t the model offered by Google in the same category? Or did I misunderstand their latest naming fiasco and “Gemini Pro” is not actually their latest model? EDIT: It looks like Gemini Ultra 1.0 > Gemini Pro 1.5 > Gemini Pro 1.0… is that true? What a horrible versioning scheme.