5 ms·

Here's a probably stupid question - if someone were unbounded by ethics and conceivably had enough power and connections to power to shield themselves from many

Here's a probably stupid question - if someone were unbounded by ethics and conceivably had enough power and connections to power to shield themselves from many consequences of their actions - and that person owned these DCs, could they in theory observe all the streams of tokens coming in and out of these models, and even exfiltrate copies of these models wholesale to have their own teams do what they will with them in the pursuit of building their own competitive models? Or is there something fundamental in the way these models get deployed (encryption or something or than legal contracts?) at this scale that prohibits the owners of the infra from gaining this level of insight / access?

fragmede 5mo ago
There's accusations that the Chinese labs have done essentially that to OpenAI and Anthropic and exfiltrated their models without having DC access, so if you had DC access, yes, you could do that. If you had DC access though you could just copy the model onto an SSD.
- zozbot234 5mo ago
  The Chinese labs have been accused of training on elicited chat logs on a massive scale in violation of ToS. That's possibly a real concern, but it's nowhere close to "exfiltrating" the model or even roughly matching its behavior.
tristanj 5mo ago
1) The situation you described would be covered under the contract between Anthropic and xAI, and that any violation of that would be subject to financial penalties and legal proceedings. The US has a robust corporate legal system, and disputes do get resolved through the court system, although in a slow and costly manner. The contract can stipulate a penalty at a high enough amount to discourage this behavior. 2) Output from models & intra-datacenter communications can be encrypted if customers truly cared. 3) There is no reason do this, because there are far better ways to exfiltrate data from Anthropic models. Chinese companies are already doing this at an industrial scale where they are reselling Claude tokens for 10-20% of the cost while retaining the data to train their own models. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens... If we look at Deepseek V4-pro, created by Deepseek who Anthropic formally accused of harvesting Claude tokens at scale, it performs the same as Claude did 6 months prior.
- seydor 5mo ago
  I guess there's a reason why those Chinese companies are in china
  - overfeed 5mo ago
    Under US law, LLM outputs are not under copyright. Even if those companies were under the American jurisdiction[1], the worst thing they would be guilty of is a breach of ToS. 1. OpenRouter is based in New York, and offers a 1% discount when users enable logging of their inputs/outputs; it doesn't take a genius to figure out why they might incentivize that
- 47282847 5mo ago
  > The US has a robust corporate legal system Thanks for the chuckle. ;)
  - rafram 5mo ago
    It does, though.
    - pyrale 5mo ago
      It does until you're embedded enough with the surveillance system [1]. If a company was able to get immunity in the wat AT&T got it, no contract would protect the other side. [1]: https://en.wikipedia.org/wiki/Hepting_v._AT%26T https://en.wikipedia.org/wiki/Hepting_v._AT%26T
      - tristanj 5mo ago
        That case has nothing to do with contract law.
        whattheheckheck 5mo ago
        And the govt protects its citizens and God is real
deleted 5mo ago
[deleted]
rdgthree 5mo ago
xAI had a lot of negotiating power here because Anthropic had ~0 comparable options and ultimately desperately needed the compute now. So, it wouldn't surprise me if data sharing was an explicit part of the agreement
espeed 5mo ago
What prevents a data center operator from reading your chats? [FEATURE] Provide a way to select your data center #56916 https://github.com/anthropics/claude-code/issues/56916 https://github.com/anthropics/claude-code/issues/56916
giancarlostoro 5mo ago
You could, but Grok is pretty high up there, it might not be "#1" but its definitely up there with the giants, people seem to overlook it. Gemini has a similar problem, it was #1 once, and it seems like Google isn't hell bent on chasing #1 they just want to keep iterating over time, they know they just need it to be "good enough" and they'll keep having repeat customers. If Elon REALLY wanted to do anything like that he would be better off poaching talent from competitors, less legal hell to go through.
- petesergeant 5mo ago
  If I was training a top-tier model, having a competitors’ weights feels like it would be an excellent refining tool. As a user I find cross-model iteration to be a huge power-tool, and presumably the boffins can zero in on areas of relative strength and weakness and work out which area they want massive amounts of synthetic data from.
- jatora 5mo ago
  No Grok is not 'up there'. Not by any stretch of the imagination is it anywhere close to anthropic or openai in any single domain whatsoever. Not even close to deepseek. It really isnt a good model. Their research team's talent is just not good, unfortunately.
  - scubbo 5mo ago
    > in any single domain I hear that it is very much up there in _one_ domain. You know the one.
- mike_d 5mo ago
  You must have missed the part where he did poach all the top talent from Google and OpenAI, and then they all quit because they couldn't stand working for him.
Traster 5mo ago
It's already in the public domain (thanks to the OpenAI trial) that Grok distilled OpenAIs models. Listening to the data going into the models in the data centre would be very similar thing. There's some downsides (you're passively listening, not controlling the queries), and some upsides (way more data). But it only ever gets you to some percentage of the existing production model. It doesn't get you what Musk wants - an AI company capable of designing and deploying leading edge models. It gets you to fast follower status.
Glohrischi 5mo ago
Yes he could do that and I thought about this too. Very weird tbh
HarHarVeryFunny 5mo ago
Bear in mind these data centers were built for X.ai to use themselves, so there would have been no reason to backdoor them. Also, this is all off-the shelf equipment: Dell & Supermicro servers with NVidia compute modules and ethernet switches.
- killerstorm 5mo ago
  The fact that X.ai build it for themselves means that they are in full control. It's not challenging to install a rootkit on hardware under your control. Also if traffic within the datacenter is unencrypted, you can probably get copy of all comms via switches. It's generally far easier to install spying software under a computer under your control than to detect it.
deleted 5mo ago
[deleted]
__MatrixMan__ 5mo ago
Given Musk's previous antics, this strikes me as a relevant line of reasoning. How do you trust space infra? You can't show up and audit some kind of airgap. Whatever encryption keys you have are likely accessible in memory sometime.
ncr100 5mo ago
Could they also, if they owned a 10,300 vehicle fleet of orbiting satellites, add radar or optical surveillance to each, and track every human on planet early in real time?
- fragmede 5mo ago
  Why would they need to do that, when the majority of those humans are already carrying a radio transceiver with them at all times.

fragmede 5mo ago

There's accusations that the Chinese labs have done essentially that to OpenAI and Anthropic and exfiltrated their models without having DC access, so if you had DC access, yes, you could do that. If you had DC access though you could just copy the model onto an SSD.

zozbot234 5mo ago
The Chinese labs have been accused of training on elicited chat logs on a massive scale in violation of ToS. That's possibly a real concern, but it's nowhere close to "exfiltrating" the model or even roughly matching its behavior.

tristanj 5mo ago

1) The situation you described would be covered under the contract between Anthropic and xAI, and that any violation of that would be subject to financial penalties and legal proceedings. The US has a robust corporate legal system, and disputes do get resolved through the court system, although in a slow and costly manner. The contract can stipulate a penalty at a high enough amount to discourage this behavior. 2) Output from models & intra-datacenter communications can be encrypted if customers truly cared. 3) There is no reason do this, because there are far better ways to exfiltrate data from Anthropic models. Chinese companies are already doing this at an industrial scale where they are reselling Claude tokens for 10-20% of the cost while retaining the data to train their own models. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens... If we look at Deepseek V4-pro, created by Deepseek who Anthropic formally accused of harvesting Claude tokens at scale, it performs the same as Claude did 6 months prior.

seydor 5mo ago
I guess there's a reason why those Chinese companies are in china
- overfeed 5mo ago
  Under US law, LLM outputs are not under copyright. Even if those companies were under the American jurisdiction[1], the worst thing they would be guilty of is a breach of ToS. 1. OpenRouter is based in New York, and offers a 1% discount when users enable logging of their inputs/outputs; it doesn't take a genius to figure out why they might incentivize that
47282847 5mo ago
> The US has a robust corporate legal system Thanks for the chuckle. ;)
- rafram 5mo ago
  It does, though.
  - pyrale 5mo ago
    It does until you're embedded enough with the surveillance system [1]. If a company was able to get immunity in the wat AT&T got it, no contract would protect the other side. [1]: https://en.wikipedia.org/wiki/Hepting_v._AT%26T https://en.wikipedia.org/wiki/Hepting_v._AT%26T
    - tristanj 5mo ago
      That case has nothing to do with contract law.
      - whattheheckheck 5mo ago
        And the govt protects its citizens and God is real

deleted 5mo ago

[deleted]

rdgthree 5mo ago

xAI had a lot of negotiating power here because Anthropic had ~0 comparable options and ultimately desperately needed the compute now. So, it wouldn't surprise me if data sharing was an explicit part of the agreement

espeed 5mo ago

What prevents a data center operator from reading your chats? [FEATURE] Provide a way to select your data center #56916 https://github.com/anthropics/claude-code/issues/56916 https://github.com/anthropics/claude-code/issues/56916

giancarlostoro 5mo ago

You could, but Grok is pretty high up there, it might not be "#1" but its definitely up there with the giants, people seem to overlook it. Gemini has a similar problem, it was #1 once, and it seems like Google isn't hell bent on chasing #1 they just want to keep iterating over time, they know they just need it to be "good enough" and they'll keep having repeat customers. If Elon REALLY wanted to do anything like that he would be better off poaching talent from competitors, less legal hell to go through.

petesergeant 5mo ago
If I was training a top-tier model, having a competitors’ weights feels like it would be an excellent refining tool. As a user I find cross-model iteration to be a huge power-tool, and presumably the boffins can zero in on areas of relative strength and weakness and work out which area they want massive amounts of synthetic data from.
jatora 5mo ago
No Grok is not 'up there'. Not by any stretch of the imagination is it anywhere close to anthropic or openai in any single domain whatsoever. Not even close to deepseek. It really isnt a good model. Their research team's talent is just not good, unfortunately.
- scubbo 5mo ago
  > in any single domain I hear that it is very much up there in _one_ domain. You know the one.
mike_d 5mo ago
You must have missed the part where he did poach all the top talent from Google and OpenAI, and then they all quit because they couldn't stand working for him.

Traster 5mo ago

It's already in the public domain (thanks to the OpenAI trial) that Grok distilled OpenAIs models. Listening to the data going into the models in the data centre would be very similar thing. There's some downsides (you're passively listening, not controlling the queries), and some upsides (way more data). But it only ever gets you to some percentage of the existing production model. It doesn't get you what Musk wants - an AI company capable of designing and deploying leading edge models. It gets you to fast follower status.

Glohrischi 5mo ago

Yes he could do that and I thought about this too. Very weird tbh

HarHarVeryFunny 5mo ago

Bear in mind these data centers were built for X.ai to use themselves, so there would have been no reason to backdoor them. Also, this is all off-the shelf equipment: Dell & Supermicro servers with NVidia compute modules and ethernet switches.

killerstorm 5mo ago
The fact that X.ai build it for themselves means that they are in full control. It's not challenging to install a rootkit on hardware under your control. Also if traffic within the datacenter is unencrypted, you can probably get copy of all comms via switches. It's generally far easier to install spying software under a computer under your control than to detect it.

deleted 5mo ago

[deleted]

__MatrixMan__ 5mo ago

Given Musk's previous antics, this strikes me as a relevant line of reasoning. How do you trust space infra? You can't show up and audit some kind of airgap. Whatever encryption keys you have are likely accessible in memory sometime.

ncr100 5mo ago

Could they also, if they owned a 10,300 vehicle fleet of orbiting satellites, add radar or optical surveillance to each, and track every human on planet early in real time?

fragmede 5mo ago
Why would they need to do that, when the majority of those humans are already carrying a radio transceiver with them at all times.