3 ms·

It seems more and more that the solution to AI's quality problems is... more AI. Does Anthropic do something like this as well, or is there another reason Clau

by mkmk 2y ago

It seems more and more that the solution to AI's quality problems is... more AI. Does Anthropic do something like this as well, or is there another reason Claude Sonnet 3.5 is so much better at coding than GPT-4o?

GaggiX 2y ago
>or is there another reason Claude Sonnet 3.5 is so much better at coding than GPT-4o? It's impossible to say because these models are proprietary.
- mkmk 2y ago
  Isn't the very article we're commenting on an indication that you can form a basic opinion on what makes one proprietary model different from another?
  - GaggiX 2y ago
    Not really, we know absolutely nothing about Claude 3.5 Sonnet, except that it's an LLM.
ru552 2y ago
Anthropic has attributed Sonnet 3.5's model improvement to better training data. "Which data specifically? Gerstenhaber wouldn’t disclose, but he implied that Claude 3.5 Sonnet draws much of its strength from these training sets."[0] [0]https://techcrunch.com/2024/06/20/anthropic-claims-its-latest-model-is-best-in-class/ https://techcrunch.com/2024/06/20/anthropic-claims-its-lates...
- swyx 2y ago
  and water is wet
jasonjmcghee 2y ago
My guess, which could be completely wrong, Anthropic spent more resources on interpretability and it's paying off. I remember when I first started using activation maps when building image classification models and it was like what on earth was I doing before this... just blindly trusting the loss. How do you discover biases and issues with training data without interpretability?
Kiro 2y ago
Is it really that much better? I'm really happy with GPT-4o's coding capabilities and very seldom experience problems with hallucinations or incorrect responses, so I'm intrigued by how much better it can actually be.
p1esk 2y ago
In my experience Sonnet 3.5 is about the same as 4o for coding. Sometimes one provides a better solution, sometimes the other. Both are pretty good.
surfingdino 2y ago
> It seems more and more that the solution to AI's quality problems is... more AI. This reminds me of the passage found in the description of the fuckitpy module: "This module is like violence: if it doesn't work, you just need more of it."

by mkmk 2y ago

GaggiX 2y ago

>or is there another reason Claude Sonnet 3.5 is so much better at coding than GPT-4o? It's impossible to say because these models are proprietary.

mkmk 2y ago
Isn't the very article we're commenting on an indication that you can form a basic opinion on what makes one proprietary model different from another?
- GaggiX 2y ago
  Not really, we know absolutely nothing about Claude 3.5 Sonnet, except that it's an LLM.

ru552 2y ago

Anthropic has attributed Sonnet 3.5's model improvement to better training data. "Which data specifically? Gerstenhaber wouldn’t disclose, but he implied that Claude 3.5 Sonnet draws much of its strength from these training sets."[0] [0]https://techcrunch.com/2024/06/20/anthropic-claims-its-latest-model-is-best-in-class/ https://techcrunch.com/2024/06/20/anthropic-claims-its-lates...

swyx 2y ago
and water is wet

jasonjmcghee 2y ago

My guess, which could be completely wrong, Anthropic spent more resources on interpretability and it's paying off. I remember when I first started using activation maps when building image classification models and it was like what on earth was I doing before this... just blindly trusting the loss. How do you discover biases and issues with training data without interpretability?

Kiro 2y ago

Is it really that much better? I'm really happy with GPT-4o's coding capabilities and very seldom experience problems with hallucinations or incorrect responses, so I'm intrigued by how much better it can actually be.

p1esk 2y ago

In my experience Sonnet 3.5 is about the same as 4o for coding. Sometimes one provides a better solution, sometimes the other. Both are pretty good.

surfingdino 2y ago

> It seems more and more that the solution to AI's quality problems is... more AI. This reminds me of the passage found in the description of the fuckitpy module: "This module is like violence: if it doesn't work, you just need more of it."