3 ms·

They're also not focused exclusively only on building an LLM, they have video and image generation too. Anthropic has one single focus, and this is why they are

They're also not focused exclusively only on building an LLM, they have video and image generation too. Anthropic has one single focus, and this is why they are usually at the very top in the SWE benchmarks.

epistasis 6mo ago
IMHO the benchmarks aren't useful, and ranking among the frontier models is mostly noise. The extra features around the coding agent have a much bigger impact on productivity than having to provide slightly more specification and guidance to the models; a 90% success rate versus a 92% success rate on the tasks I ask it to do is far more influenced by what I say than what the model is capable of.
phillipcarter 6mo ago
Isn't it the case that OpenAI and Anthropic regularly just swap for whoever is at the top of the latest benchmarks? They're also so close in scores that it's effectively a wash anyways. What OP is referring to is Anthropic aligning with corporate terms and conditions early, positioning themselves to be effectively resold by AWS rather than requiring orgs to procure them directly. This is huge in the enterprise world because the processes to get broad approval are generally far smaller and shorter for "just another AWS service" compared to a whole new vendor.
- Grimblewald 6mo ago
  Isn't it an open secret that benchmarks are largly irrelevant at this point? Why else we do all have a personalized test battery for new models? That said i've stopped testing chatgpt entierly. Its still ok but is beaten by local models and it gets thrashed by non oai frontier providers. I get the history, but holding up oai outputs as equivallent is lile comparing yahoo to google post yahoo's collapse in search domains. Oai language models are largly irrelevant at this point imo.
- djtriptych 6mo ago
  OpenAI did teh same thing with Microsoft/Azure though.
hn_throwaway_99 6mo ago
Open AI is killing Sora though, so it looks like they are looking at Anthropic's playbook of focusing on enterprise use cases and seeing that it's more profitable.
- dannyw 6mo ago
  But then they released gpt-image-2, which is clearly SOTA.
  - hn_throwaway_99 6mo ago
    Image generation would seem like it would have many more enterprise use cases (particularly marketing, but essentially all the business uses of Photoshop) than video.
  - InvidFlower 6mo ago
    Besides what the other person mentioned about being more useful for enterprise, I also heard mentioned on a podcast that gpt-image-2 uses the same general architecture as the LLM models, while Sora was a very different architecture. So they don't need two different sets of everything by shutting down Sora.
DrewADesign 6mo ago
Didn’t they say Sora will only be used to internally create training data? Integrated image generation seems more in the neat feature category than some fundamental advantage, but maybe someone has use cases I haven’t considered.

epistasis 6mo ago

IMHO the benchmarks aren't useful, and ranking among the frontier models is mostly noise. The extra features around the coding agent have a much bigger impact on productivity than having to provide slightly more specification and guidance to the models; a 90% success rate versus a 92% success rate on the tasks I ask it to do is far more influenced by what I say than what the model is capable of.

phillipcarter 6mo ago

Isn't it the case that OpenAI and Anthropic regularly just swap for whoever is at the top of the latest benchmarks? They're also so close in scores that it's effectively a wash anyways. What OP is referring to is Anthropic aligning with corporate terms and conditions early, positioning themselves to be effectively resold by AWS rather than requiring orgs to procure them directly. This is huge in the enterprise world because the processes to get broad approval are generally far smaller and shorter for "just another AWS service" compared to a whole new vendor.

Grimblewald 6mo ago
Isn't it an open secret that benchmarks are largly irrelevant at this point? Why else we do all have a personalized test battery for new models? That said i've stopped testing chatgpt entierly. Its still ok but is beaten by local models and it gets thrashed by non oai frontier providers. I get the history, but holding up oai outputs as equivallent is lile comparing yahoo to google post yahoo's collapse in search domains. Oai language models are largly irrelevant at this point imo.
djtriptych 6mo ago
OpenAI did teh same thing with Microsoft/Azure though.

hn_throwaway_99 6mo ago

Open AI is killing Sora though, so it looks like they are looking at Anthropic's playbook of focusing on enterprise use cases and seeing that it's more profitable.

dannyw 6mo ago
But then they released gpt-image-2, which is clearly SOTA.
- hn_throwaway_99 6mo ago
  Image generation would seem like it would have many more enterprise use cases (particularly marketing, but essentially all the business uses of Photoshop) than video.
- InvidFlower 6mo ago
  Besides what the other person mentioned about being more useful for enterprise, I also heard mentioned on a podcast that gpt-image-2 uses the same general architecture as the LLM models, while Sora was a very different architecture. So they don't need two different sets of everything by shutting down Sora.

DrewADesign 6mo ago

Didn’t they say Sora will only be used to internally create training data? Integrated image generation seems more in the neat feature category than some fundamental advantage, but maybe someone has use cases I haven’t considered.