7 ms·

I' m not sure how they make it back. The guardrails in place are extremely strict. The only people who seem to use it are a subset of developers who are unhap

by ipaddr 2y ago

I' m not sure how they make it back. The guardrails in place are extremely strict. The only people who seem to use it are a subset of developers who are unhappy with OpenAI. With Bard popping up free everywhere taking away much of the general user crowd and OpenAI offering the mini model always free and limited image generation / expensive model. Then you have to do it yourself crowd with llama. What is their target market? Governments? Amazon companies?There free their offers 10 queries and half of them need to be used to get around filters I don't see this positioned well for general customers.

JamesBarney 2y ago
Claude api use is already as high as openai. I believe that market will grow far more over time than chat as AI gets embedded in more of the applications we already use.
reubenmorais 2y ago
With Claude on Bedrock I can use LLMs in production without sending customer data to the US. And if you're already on AWS it's super easy to onboard wrt. auth and billing and compliance.
- maeil 2y ago
  If you're using Bedrock you're still subject to the CLOUD act/FISA meaning the whole angle of "not sending customer data to the US" isn't worth very much.
  - reubenmorais 2y ago
    It's worth enough to customers to make a best effort.
loandbehold 2y ago
Claude is the best model for programming. New generation of code tools like Cursor all use Claude as the main model.
- petesergeant 2y ago
  > Claude is the best model for programming This week.
  - GaggiX 2y ago
    It has been for the last several months now.
  - square_usual 2y ago
    It has held this position since at least June. The Aider LLM leaderboards [1] have the Sonnet 3.5 June version beating 4o handily. Only o1-preview beat it narrowly, but IIRC at much higher costs. Sonnet 3.5 October has taken the lead again by a wide margin. 1: https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/
  - iLemming 2y ago
    Anecdotally, Claude seems to hallucinate more during certain hours. It's amusing to watch, almost like your dog that gets too bored and stops responding to your commands - you say "sit" and he looks at you, tilts his head, looks straight up at you, almost like saying "I know what you're saying..." but then decides to run to another room and bring his toy. And you'd be wondering: "darn, where's that toughest, most obidient and smart Belgian malinois that just a few hour ago was ready to take down a Bin Laden?"
    - petesergeant 2y ago
      Talking of anecdotal, 4o with canvas, which is normally excellent, tends to give up around a certain context length, and you have to copy and paste what you have into a new window to get it to make edits
  - maeil 2y ago
    This week, along with the 20 weeks before that :) Model improvement has slowed down so much that things aren't changing quickly anymore. And Anthropic has only widened the gap with 3.5-v2.
staticman2 2y ago
The Guardrails on Claude Sonnet 3.5 API are not stricter than Openai's guardrails in my experience. More specifically, if you access the models via API or third party services like Poe or Perplexity the guardrails are not stricter than GPT4o. I've never subscribed to Claude.ai so can't comment on that. I have no experience with Claud.ai vs ChatGPT but it's clear the underlying model has no issue with guardrails and this is simply an easily tweaked developer setting if you are correct that they are stricter on Claude.ai. (The old Claude 2.1 was hilariously unwilling to follow reasonable user instructions due to "ethics" but they've come a long way since then.)
- ipaddr 2y ago
  My comment was purely about Claud.ai which is where general customers would go.
  - staticman2 2y ago
    I don't know if Claude.ai or ChatGPT are even profitable at this stage, so they might not particularly want general customers.
- dragonwriter 2y ago
  > The Guardrails on Claude Sonnet 3.5 API are not stricter than Openai’s guardrails in my experience. Both Gemini and Claude (via the API) have substantially tighter guardrails around recitation (producing output matching data from their training set) than OpenAI, which I ran into when testing an image text-extraction-and-document-formatting toolchain against all three. Both Claude and Gemini gave refusals on text extraction from image documents (not available publicly anywhere I can find as text) from a CIA FOIA release Not sure if they are tighter in other areas.
  - staticman2 2y ago
    I just asked GPT4o to recognize a cartoon character (I accessed it via Perplexity) and it told me it isn't able to do that, while Claude Sonnet happily identified the character, so this might vary by use case or even by prompt.
atsaloli 2y ago
I am in Operations. I use it (and pay for it) because the free version seemed to work best for me compared to Perplexity (which had been my go-to) and ChatGPT/OpenAI.
hamburga 2y ago
Government alone could be huge, with this recent nonsense about the military funding a “Manhattan project for AI” and the recently announced Pentagon contracts.
Deegy 2y ago
I mean, they might make back the $4b on the value it brings to programming alone.

by ipaddr 2y ago

JamesBarney 2y ago

Claude api use is already as high as openai. I believe that market will grow far more over time than chat as AI gets embedded in more of the applications we already use.

reubenmorais 2y ago

With Claude on Bedrock I can use LLMs in production without sending customer data to the US. And if you're already on AWS it's super easy to onboard wrt. auth and billing and compliance.

maeil 2y ago
If you're using Bedrock you're still subject to the CLOUD act/FISA meaning the whole angle of "not sending customer data to the US" isn't worth very much.
- reubenmorais 2y ago
  It's worth enough to customers to make a best effort.

loandbehold 2y ago

Claude is the best model for programming. New generation of code tools like Cursor all use Claude as the main model.

petesergeant 2y ago
> Claude is the best model for programming This week.
- GaggiX 2y ago
  It has been for the last several months now.
- square_usual 2y ago
  It has held this position since at least June. The Aider LLM leaderboards [1] have the Sonnet 3.5 June version beating 4o handily. Only o1-preview beat it narrowly, but IIRC at much higher costs. Sonnet 3.5 October has taken the lead again by a wide margin. 1: https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/
- iLemming 2y ago
  Anecdotally, Claude seems to hallucinate more during certain hours. It's amusing to watch, almost like your dog that gets too bored and stops responding to your commands - you say "sit" and he looks at you, tilts his head, looks straight up at you, almost like saying "I know what you're saying..." but then decides to run to another room and bring his toy. And you'd be wondering: "darn, where's that toughest, most obidient and smart Belgian malinois that just a few hour ago was ready to take down a Bin Laden?"
  - petesergeant 2y ago
    Talking of anecdotal, 4o with canvas, which is normally excellent, tends to give up around a certain context length, and you have to copy and paste what you have into a new window to get it to make edits
- maeil 2y ago
  This week, along with the 20 weeks before that :) Model improvement has slowed down so much that things aren't changing quickly anymore. And Anthropic has only widened the gap with 3.5-v2.

staticman2 2y ago

The Guardrails on Claude Sonnet 3.5 API are not stricter than Openai's guardrails in my experience. More specifically, if you access the models via API or third party services like Poe or Perplexity the guardrails are not stricter than GPT4o. I've never subscribed to Claude.ai so can't comment on that. I have no experience with Claud.ai vs ChatGPT but it's clear the underlying model has no issue with guardrails and this is simply an easily tweaked developer setting if you are correct that they are stricter on Claude.ai. (The old Claude 2.1 was hilariously unwilling to follow reasonable user instructions due to "ethics" but they've come a long way since then.)

ipaddr 2y ago
My comment was purely about Claud.ai which is where general customers would go.
- staticman2 2y ago
  I don't know if Claude.ai or ChatGPT are even profitable at this stage, so they might not particularly want general customers.
dragonwriter 2y ago
> The Guardrails on Claude Sonnet 3.5 API are not stricter than Openai’s guardrails in my experience. Both Gemini and Claude (via the API) have substantially tighter guardrails around recitation (producing output matching data from their training set) than OpenAI, which I ran into when testing an image text-extraction-and-document-formatting toolchain against all three. Both Claude and Gemini gave refusals on text extraction from image documents (not available publicly anywhere I can find as text) from a CIA FOIA release Not sure if they are tighter in other areas.
- staticman2 2y ago
  I just asked GPT4o to recognize a cartoon character (I accessed it via Perplexity) and it told me it isn't able to do that, while Claude Sonnet happily identified the character, so this might vary by use case or even by prompt.

atsaloli 2y ago

I am in Operations. I use it (and pay for it) because the free version seemed to work best for me compared to Perplexity (which had been my go-to) and ChatGPT/OpenAI.

hamburga 2y ago

Government alone could be huge, with this recent nonsense about the military funding a “Manhattan project for AI” and the recently announced Pentagon contracts.

Deegy 2y ago

I mean, they might make back the $4b on the value it brings to programming alone.