7 ms·

Have you actually tried using Opus side by side with GPT4 on “work” related stuff? GPT4 is way better in my experience, to the point where I cancelled my Opus s

by ein0p 2y ago

Have you actually tried using Opus side by side with GPT4 on “work” related stuff? GPT4 is way better in my experience, to the point where I cancelled my Opus subscription after just a couple of months.

deleted 2y ago
[deleted]
benreesman 2y ago
I make an effort to use both and several high-capability open tunes every day (it’s not literally every day but I have keyboard shortcuts for all of them). Opus historically had issues with minor typographical errors, though recently that seems to not happen often, lots of very sharp people at Anthropic. So a month ago if I wanted something from Opus I’d run it through a cleanup pass courtesy of one of the other ones, but even my old standby dolphin-8x7 can clean up typos. 1106 can as well, but all else equal I don’t want to be sending my stuff to any black box data warehouse and I’m always surprised so many other sophisticated people don’t share the preference. My personal eyeball capability check is to posit a gauge symmetry and ask what it thinks the implied conserved quantity is, and I’ve yet to see Opus not crush that relative to anything else, including real footnotes. On coding I usually hand it a Prisma schema and ask for a proto3/gRPC definition that is a good way to interact with it, Opus in my personal experience also dominates there. If you have an example of a task that represents a counter example I’d be grateful for another little integration test for my personal ad-hoc model card. I want to know the best tool for every job.
- refulgentis 2y ago
  Opus hasn't changed. I really don't like how condescending your root comment is, I don't have any idea how all these extra things you're disappointed in are relevant at all to the actual topic.
virgildotcodes 2y ago
I feel like Opus is much better at creative writing in terms of sounding more natural and less formulaic, but GPT4 does beat it on just about everything else.
- benreesman 2y ago
  I replied to a sibling with a few of my ad-hoc tasks. I’d likewise be grateful if you’d lend me an example or two that should be on my little ad-hoc task set.
  - CSMastermind 2y ago
    Which direction are you looking for? Ones where GPT-4o does better or ones where Opus does better? Assuming the former, here's an example I had yesterday where Opus was never able to formulate a correct response even with multiple follow-ups but GPT-4o got it with only two follow ups: > I have a postgres database with a table called projects. On that table is a column called designs that is of json type and is nullable as well as a column called id that is of type text. The data in the designs column will be an array of objects (assuming it's not null). Each object in the array will have a field called price that will be a string. Most of these prices will be in the format $#,### where # is a number. However some will be in the format # ### US$. This second format where the numbers are separated by spaces instead of commas and the dollar sign appears at the end of the string instead of the beginning with an extraneous US is incorrect. What SQL can I run to migrate all prices that are incorrectly formatted to the correct format? Alternatively, this is a non-tech related one that GPT-4 gets correct that no other model does: > In 388 BC, the boxer Eupolus of Thessaly defeated three opponents at the ancient Olympics. Funded by the men involved, several bronze statues of Zeus were erected, with inscriptions detailing the events. Why did the four men prefer not to have the statues around? I like that one because Opus will tell you that you're wrong and mistaken, which is pretty funny while GPT-4 answer correctly with the actual context.
    - ppsreejith 2y ago
      Very interesting prompt regarding the boxer Eupolus of Thessaly. Gemini Advanced gets this wrong as well as Llama 3 70b (as run on groq.com). However, if I start with: "What is the earliest record of cheating in the olympic games?" Then all models get the question right. It's surprising that GPT-4 gets it right on the first go.
  - 2y ago
Zambyte 2y ago
That's really interesting to me. I've been using both through my Kagi subscription, but I always find myself favoring the quality of Opus. I generally use GPT 4o if I don't want to wait for a slow response from Opus, and I use Opus when I want the highest quality.
- benreesman 2y ago
  It’s likewise my experience that on e.g. TTFT is higher on Anthropic’s API. But it’s not by a ton, and it could be just less hyper-scale network infrastructure. I don’t recall what Anthropic has raised but IIRC it wasn’t “peer with everyone” levels.
  - seanhunter 2y ago
    I think they've raised just over $5B, consisting of $1B from everyone other than amazon (750m at [1] and 450m at [2]) and then $4B from amazon[3]. The reason I seperate them is I think a lot of the $4B from amazon is in the form of AWS credits rather than cash. [1] https://www.bloomberg.com/news/articles/2023-12-21/anthropic-in-talks-to-raise-750-million-led-by-menlo-ventures https://www.bloomberg.com/news/articles/2023-12-21/anthropic... [2] https://www.anthropic.com/news/anthropic-series-c https://www.anthropic.com/news/anthropic-series-c [3] https://www.aboutamazon.com/news/company-news/amazon-anthropic-ai-investment https://www.aboutamazon.com/news/company-news/amazon-anthrop...

by ein0p 2y ago

deleted 2y ago

[deleted]

benreesman 2y ago

I make an effort to use both and several high-capability open tunes every day (it’s not literally every day but I have keyboard shortcuts for all of them). Opus historically had issues with minor typographical errors, though recently that seems to not happen often, lots of very sharp people at Anthropic. So a month ago if I wanted something from Opus I’d run it through a cleanup pass courtesy of one of the other ones, but even my old standby dolphin-8x7 can clean up typos. 1106 can as well, but all else equal I don’t want to be sending my stuff to any black box data warehouse and I’m always surprised so many other sophisticated people don’t share the preference. My personal eyeball capability check is to posit a gauge symmetry and ask what it thinks the implied conserved quantity is, and I’ve yet to see Opus not crush that relative to anything else, including real footnotes. On coding I usually hand it a Prisma schema and ask for a proto3/gRPC definition that is a good way to interact with it, Opus in my personal experience also dominates there. If you have an example of a task that represents a counter example I’d be grateful for another little integration test for my personal ad-hoc model card. I want to know the best tool for every job.

refulgentis 2y ago
Opus hasn't changed. I really don't like how condescending your root comment is, I don't have any idea how all these extra things you're disappointed in are relevant at all to the actual topic.

virgildotcodes 2y ago

I feel like Opus is much better at creative writing in terms of sounding more natural and less formulaic, but GPT4 does beat it on just about everything else.

benreesman 2y ago
I replied to a sibling with a few of my ad-hoc tasks. I’d likewise be grateful if you’d lend me an example or two that should be on my little ad-hoc task set.
- CSMastermind 2y ago
  Which direction are you looking for? Ones where GPT-4o does better or ones where Opus does better? Assuming the former, here's an example I had yesterday where Opus was never able to formulate a correct response even with multiple follow-ups but GPT-4o got it with only two follow ups: > I have a postgres database with a table called projects. On that table is a column called designs that is of json type and is nullable as well as a column called id that is of type text. The data in the designs column will be an array of objects (assuming it's not null). Each object in the array will have a field called price that will be a string. Most of these prices will be in the format $#,### where # is a number. However some will be in the format # ### US$. This second format where the numbers are separated by spaces instead of commas and the dollar sign appears at the end of the string instead of the beginning with an extraneous US is incorrect. What SQL can I run to migrate all prices that are incorrectly formatted to the correct format? Alternatively, this is a non-tech related one that GPT-4 gets correct that no other model does: > In 388 BC, the boxer Eupolus of Thessaly defeated three opponents at the ancient Olympics. Funded by the men involved, several bronze statues of Zeus were erected, with inscriptions detailing the events. Why did the four men prefer not to have the statues around? I like that one because Opus will tell you that you're wrong and mistaken, which is pretty funny while GPT-4 answer correctly with the actual context.
  - ppsreejith 2y ago
    Very interesting prompt regarding the boxer Eupolus of Thessaly. Gemini Advanced gets this wrong as well as Llama 3 70b (as run on groq.com). However, if I start with: "What is the earliest record of cheating in the olympic games?" Then all models get the question right. It's surprising that GPT-4 gets it right on the first go.
- 2y ago

Zambyte 2y ago

That's really interesting to me. I've been using both through my Kagi subscription, but I always find myself favoring the quality of Opus. I generally use GPT 4o if I don't want to wait for a slow response from Opus, and I use Opus when I want the highest quality.

benreesman 2y ago
It’s likewise my experience that on e.g. TTFT is higher on Anthropic’s API. But it’s not by a ton, and it could be just less hyper-scale network infrastructure. I don’t recall what Anthropic has raised but IIRC it wasn’t “peer with everyone” levels.
- seanhunter 2y ago
  I think they've raised just over $5B, consisting of $1B from everyone other than amazon (750m at [1] and 450m at [2]) and then $4B from amazon[3]. The reason I seperate them is I think a lot of the $4B from amazon is in the form of AWS credits rather than cash. [1] https://www.bloomberg.com/news/articles/2023-12-21/anthropic-in-talks-to-raise-750-million-led-by-menlo-ventures https://www.bloomberg.com/news/articles/2023-12-21/anthropic... [2] https://www.anthropic.com/news/anthropic-series-c https://www.anthropic.com/news/anthropic-series-c [3] https://www.aboutamazon.com/news/company-news/amazon-anthropic-ai-investment https://www.aboutamazon.com/news/company-news/amazon-anthrop...