3 ms·

Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API r

Can somebody clarify for me - if ANTHROPIC_BASE_URL is set to a different provider... then isn't this "marked" system prompt being sent to that provider's API rather than Anthropic's? I understand how this can be useful to Anthropic if the 3rd-party is acting as a proxy (because they end up hitting the Claude API with the marked prompt), but it looks like requests where "hostname contains deepseek" would never be sending data to Anthropic. What am I missing?

andrewmunsell 4mo ago
My guess is for distillation, they need to forward the prompt to Anthropic to get the real Anthropic model's response so they can train their own models on it
dannyw 4mo ago
The theory is probably Deepseek might be collecting those streams, and sending a portion of it to Anthropic to see what the Anthropic/Opus response would be.
pmxi 4mo ago
This catches Claude resellers. Meaning companies who proxy Claude traffic for users in, say, China. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...
- skeptic_ai 4mo ago
  Won’t catch many after has been on hn home page. And now the providers will be even more careful to upgrade the cc code. Might even provide their own agent to prevent this mockery. And isn’t what anthropic did unauthorized use of another pc which is kind of illegal?
  - sandeepkd 4mo ago
    Thats the thing, hoping to control things on client side like this is a lost battle if you are dealing with technical clients. The best they can do is probably based on IP, but again the motivated clients would just create bastion servers in allowed IP ranges. I am surprised why are they even throwing resources in this kind of effort.
    - jgilias 4mo ago
      “Hey Claude, fix the issues with Chinese resellers and distillers. Make no mistake”
- pishpash 4mo ago
  "Catch" as in made a list?
  - eli 4mo ago
    Of the accounts involved, yeah. So they can lock them out.
- throwawayffffas 4mo ago
  Assuming their software directly proxies requests instead of rewriting them.
deleted 4mo ago
[deleted]
andai 4mo ago
Did I understand correctly, that custom base URL triggers this behavior? So if I'm running Claude through a LLM proxy, I'm also affected?
- deleted 4mo ago
  [deleted]
- wett 4mo ago
  Ask Claude to check, lol
nixosbestos 4mo ago
I am also really confused and annoyingly stuck on this. I understand that the model name might appear in prompts for distillation (I guess? "You are RipOffModelv2, learn from these responses from Claude")? I guess the only explanation is that there's a side-telemetry channel that still sends some data to Anthropic, regardless of ANTHROPIC_BASE_URL overrides.
- MallocVoidstar 4mo ago
  There are a lot of companies reselling Claude to Chinese users. You use their base URL but it's still going to Anthropic.
  - breakingcups 4mo ago
    To clarify why Anthropic wants to catch these parties: they save all session logs and sell them to other LLM firms (for distillation) and have been known to use stolen credit-cards to pay for the Anthropic accounts. I'm quite all right with the first, not with the second of course.
- sebastiennight 4mo ago
  > I understand that the model name might appear in prompts for distillation (I guess? "You are RipOffModelv2, learn from these responses from Claude") This does not make sense. You wouldn't send such a prompt to the Claude model. And when you're sending the prompt (anywhere) you don't have the response yet. This is not how distillation works.
  - nixosbestos 4mo ago
    Right, sorry, I'm trying to catch up (in general) here, and am working through assumptions to get my bearings. What you say makes sense, but further adds to my confusion as to why those model names would appear in input sent to Claude at all, then. EDIT: I guess it might be because someone might point Claude at a compatible API, with its model in the URL, which is of interest to them.
eli 4mo ago
Seems like a pretty straightforward approach to collecting session logs from a bunch of different people/devices would be to have them all set their base url to proxy.deepseek.whatever which logs the data and forwards to the real API.

andrewmunsell 4mo ago

My guess is for distillation, they need to forward the prompt to Anthropic to get the real Anthropic model's response so they can train their own models on it

dannyw 4mo ago

The theory is probably Deepseek might be collecting those streams, and sending a portion of it to Anthropic to see what the Anthropic/Opus response would be.

pmxi 4mo ago

This catches Claude resellers. Meaning companies who proxy Claude traffic for users in, say, China. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...

skeptic_ai 4mo ago
Won’t catch many after has been on hn home page. And now the providers will be even more careful to upgrade the cc code. Might even provide their own agent to prevent this mockery. And isn’t what anthropic did unauthorized use of another pc which is kind of illegal?
- sandeepkd 4mo ago
  Thats the thing, hoping to control things on client side like this is a lost battle if you are dealing with technical clients. The best they can do is probably based on IP, but again the motivated clients would just create bastion servers in allowed IP ranges. I am surprised why are they even throwing resources in this kind of effort.
  - jgilias 4mo ago
    “Hey Claude, fix the issues with Chinese resellers and distillers. Make no mistake”
pishpash 4mo ago
"Catch" as in made a list?
- eli 4mo ago
  Of the accounts involved, yeah. So they can lock them out.
throwawayffffas 4mo ago
Assuming their software directly proxies requests instead of rewriting them.

deleted 4mo ago

[deleted]

andai 4mo ago

Did I understand correctly, that custom base URL triggers this behavior? So if I'm running Claude through a LLM proxy, I'm also affected?

deleted 4mo ago
[deleted]
wett 4mo ago
Ask Claude to check, lol

nixosbestos 4mo ago

I am also really confused and annoyingly stuck on this. I understand that the model name might appear in prompts for distillation (I guess? "You are RipOffModelv2, learn from these responses from Claude")? I guess the only explanation is that there's a side-telemetry channel that still sends some data to Anthropic, regardless of ANTHROPIC_BASE_URL overrides.

MallocVoidstar 4mo ago
There are a lot of companies reselling Claude to Chinese users. You use their base URL but it's still going to Anthropic.
- breakingcups 4mo ago
  To clarify why Anthropic wants to catch these parties: they save all session logs and sell them to other LLM firms (for distillation) and have been known to use stolen credit-cards to pay for the Anthropic accounts. I'm quite all right with the first, not with the second of course.
sebastiennight 4mo ago
> I understand that the model name might appear in prompts for distillation (I guess? "You are RipOffModelv2, learn from these responses from Claude") This does not make sense. You wouldn't send such a prompt to the Claude model. And when you're sending the prompt (anywhere) you don't have the response yet. This is not how distillation works.
- nixosbestos 4mo ago
  Right, sorry, I'm trying to catch up (in general) here, and am working through assumptions to get my bearings. What you say makes sense, but further adds to my confusion as to why those model names would appear in input sent to Claude at all, then. EDIT: I guess it might be because someone might point Claude at a compatible API, with its model in the URL, which is of interest to them.

eli 4mo ago

Seems like a pretty straightforward approach to collecting session logs from a bunch of different people/devices would be to have them all set their base url to proxy.deepseek.whatever which logs the data and forwards to the real API.