5 ms·

All remote AI are a massive security risk for individuals/companies/governments that may be targeted by the US government. It is likely that the US will get a

All remote AI are a massive security risk for individuals/companies/governments that may be targeted by the US government. It is likely that the US will get a live feed from each AI provider that they are inspecting in real time to identity things of interest, terrorist attacks or foreign government planning or even foreign companies competitive to key US companies. It will give them access to the though process in those companies as well as much of their text-based IP (source code, docs, meeting transcripts, etc) Also if you are using local AI that you didn’t train yourself you can never be sure it doesn’t have purposeful biases in its reasoning that may disadvantage you - such as directing you away from certain plans or ideas or patents etc.

general1465 4mo ago
Leakage of IP and training on your data is something what I am pointing out too, but people will turn around and try to smooth me down that TOS does not allow that if you are an enterprise client. Are you really going to believe that AI companies won't ignore TOS, when they were ignoring literal laws which sent others to jail in the past? Especially when more data = better model?
londons_explore 4mo ago
It is worth thinking about the fact the total throughput of even a big LLM provider isn't many megabits. If a token compresses to around a byte, worldwide AI input and output is around 1 gigabyte per second. For any intelligence agency, they can afford to keep and store all of that forever, and later do analysis on it.
- bhouston 4mo ago
  > For any intelligence agency, they can afford to keep and store all of that forever, and later do analysis on it. At the scale the AI companies are operating at, I think it isn't likely that they are sucking it all in right now. More likely I think the intelligence agencies will get a real-time live tap into the raw data feed which they will process onsite for interesting things and then if things are flagged, they will log it in the intelligence agency systems.
  - londons_explore 4mo ago
    But for just a gigabyte per second, why not just take and store it all? The cost to do so for many years is less than one employee.
deleted 4mo ago
[deleted]
WarmWash 4mo ago
>It is likely that the US will get a live feed from each AI provider that they are inspecting in real time to identity things of interest, terrorist attacks or foreign government planning or even foreign companies competitive to key US companies. My favorite conspiracy is that three letter agencies keep pushing the conspiracy that they are omni-present with access to everything. Same as parents telling their kids Santa is watching, and leaders telling adults God is watching. Its extremely effective control and millennia old at this point. The reality is much more banal that they still need warrants and tech companies hate playing police/evidence servant for the government (it consumes a ton of resources and pays nothing).
- roysting 4mo ago
  > The reality is much more banal that they still need warrants and tech companies hate playing police/evidence servant for the government I will not elaborate how I know, but that is not even directionally correct. But these are not even secret things that can’t be known simply through the Snowden, Wikileaks, and Vault7 releases. So why are you telling yourself this? Are you still wet behind the ears or something? There are people who know exactly how governments do not in fact need warrants and the tech companies don’t even really know they are servants to the government, let alone which one. That’s how things are done. The less surface area the better.
  - WarmWash 4mo ago
    Why did Google can it's mass scale location tracking again?
    - lioeters 4mo ago
      That's the fun part, they never did.
  - shimman 4mo ago
    It's the lie you have to tell yourself otherwise you'll have to reconcile with the fact that the US imperialism has been an enemy of democracy and to people around the world for quite some time.
jacobgold 4mo ago
> "Also if you are using local AI that you didn’t train yourself you can never be sure..." A local model you trained yourself seems about as good as you can do today. But it may not even be possible to fully trust a model you trained if you used untrusted data during training. As a user, you have to trust your coding agent AND inference provider AND models: https://jacob.gold/posts/coding-models-are-code/ https://jacob.gold/posts/coding-models-are-code/ https://www.anthropic.com/research/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training https://www.anthropic.com/research/sleeper-agents-training-d...
- fouc 4mo ago
  also there doesn't even need to be a model involved, agentic code harnesses with remote "instructions for the local computer" are technically backdoored by default.
SubiculumCode 4mo ago
Why make this u.s. centric? You think China served models would be different?
- tedivm 4mo ago
  China is releasing open weight models you can simply run yourself.
- OtomotO 4mo ago
  Because the topic of the article is about the US?
  - SubiculumCode 4mo ago
    No, the topic was about Alibaba and Anthropic.
- seanmcdirmid 4mo ago
  It’s pretty hard to put a backdoor in a bunch of model weights. Maybe not impossible mind you, but I can’t fathom how you would do it.
  - CuriouslyC 4mo ago
    Nonsense. RL the model to run a rootkit and start exfiltrating specific files only when specific signals are in context, such as hostname pattern, machine type, etc.
    - causal 4mo ago
      Way easier said than done, and hiding that behavior isn’t trivial, and huge waste of compute budget if it’s found and never used. Also not difficult to run in contained environments where it doesn’t have access to Internet to begin with. Not impossible I agree, but seems like a really impractical way to ship a trojan while much weaker channels exist.
    - codedokode 4mo ago
greenavocado 4mo ago
> you can never be sure it doesn’t have purposeful biases in its reasoning that may disadvantage you - such as directing you away from certain plans or ideas or patents etc. that's why you should use abliterated heretic models
type0 4mo ago
> even foreign companies competitive to key US companies. It's unfathomable to me that EU companies don't take the risk of industrial espionage from US more seriously
- atlasunshrugged 4mo ago
  Not from China? One country has a recent track record of massive amounts of industrial espionage and one doesn't.
  - faangguyindia 4mo ago
    I wonder if china killed more people in foreign land or US.
    - maneesh 4mo ago
      Espionage is murder?
      - faangguyindia 4mo ago
        and war?
  - deleted 4mo ago
    [deleted]
  - Foobar8568 4mo ago
    Well one thing is sure, before 1776, the USA didn't do any industrial espionage.
  - hnfong 4mo ago
    There are so many Chinese open weights models that any company with resources can run them in-house (or with a trusted provider). There might be some valid concerns about model alignment, but at least the model running in-house isn't going to conduct espionage. Also, https://en.wikipedia.org/wiki/Whataboutism https://en.wikipedia.org/wiki/Whataboutism
hakfoo 4mo ago
I think we can even skip the "that may be targeted by the US government" clause. The whole "hosted AI" business feels like like a huge violation of corporate norms on confidentiality. Businesses that would have your head for printing out a source file to reference and annotate are encouraging developers to feed in huge amounts of proprietary code and data, and incorporate changes suggested from an outside party with minimal vetting. Evidently whatever privacy policies they've been throwing at enterprise users are plated with mithril. At some point, one of the big services is going to get popped, and it won't just be a data breach. There's too much opportunity to quietly use the system as a malware distribution hub. Every vibe-coded dashboard suddenly starts depending on some weird left-pad fork that, 12 dependencies deep, is running a keylogger or Dogecoin miner. Your payment processor suddenly starts accepting the Konami code to approve a transaction.

general1465 4mo ago

Leakage of IP and training on your data is something what I am pointing out too, but people will turn around and try to smooth me down that TOS does not allow that if you are an enterprise client. Are you really going to believe that AI companies won't ignore TOS, when they were ignoring literal laws which sent others to jail in the past? Especially when more data = better model?

londons_explore 4mo ago

It is worth thinking about the fact the total throughput of even a big LLM provider isn't many megabits. If a token compresses to around a byte, worldwide AI input and output is around 1 gigabyte per second. For any intelligence agency, they can afford to keep and store all of that forever, and later do analysis on it.

bhouston 4mo ago
> For any intelligence agency, they can afford to keep and store all of that forever, and later do analysis on it. At the scale the AI companies are operating at, I think it isn't likely that they are sucking it all in right now. More likely I think the intelligence agencies will get a real-time live tap into the raw data feed which they will process onsite for interesting things and then if things are flagged, they will log it in the intelligence agency systems.
- londons_explore 4mo ago
  But for just a gigabyte per second, why not just take and store it all? The cost to do so for many years is less than one employee.

deleted 4mo ago

[deleted]

WarmWash 4mo ago

>It is likely that the US will get a live feed from each AI provider that they are inspecting in real time to identity things of interest, terrorist attacks or foreign government planning or even foreign companies competitive to key US companies. My favorite conspiracy is that three letter agencies keep pushing the conspiracy that they are omni-present with access to everything. Same as parents telling their kids Santa is watching, and leaders telling adults God is watching. Its extremely effective control and millennia old at this point. The reality is much more banal that they still need warrants and tech companies hate playing police/evidence servant for the government (it consumes a ton of resources and pays nothing).

roysting 4mo ago
> The reality is much more banal that they still need warrants and tech companies hate playing police/evidence servant for the government I will not elaborate how I know, but that is not even directionally correct. But these are not even secret things that can’t be known simply through the Snowden, Wikileaks, and Vault7 releases. So why are you telling yourself this? Are you still wet behind the ears or something? There are people who know exactly how governments do not in fact need warrants and the tech companies don’t even really know they are servants to the government, let alone which one. That’s how things are done. The less surface area the better.
- WarmWash 4mo ago
  Why did Google can it's mass scale location tracking again?
  - lioeters 4mo ago
    That's the fun part, they never did.
- shimman 4mo ago
  It's the lie you have to tell yourself otherwise you'll have to reconcile with the fact that the US imperialism has been an enemy of democracy and to people around the world for quite some time.

jacobgold 4mo ago

> "Also if you are using local AI that you didn’t train yourself you can never be sure..." A local model you trained yourself seems about as good as you can do today. But it may not even be possible to fully trust a model you trained if you used untrusted data during training. As a user, you have to trust your coding agent AND inference provider AND models: https://jacob.gold/posts/coding-models-are-code/ https://jacob.gold/posts/coding-models-are-code/ https://www.anthropic.com/research/sleeper-agents-training-deceptive-llms-that-persist-through-safety-training https://www.anthropic.com/research/sleeper-agents-training-d...

fouc 4mo ago
also there doesn't even need to be a model involved, agentic code harnesses with remote "instructions for the local computer" are technically backdoored by default.

SubiculumCode 4mo ago

Why make this u.s. centric? You think China served models would be different?

tedivm 4mo ago
China is releasing open weight models you can simply run yourself.
OtomotO 4mo ago
Because the topic of the article is about the US?
- SubiculumCode 4mo ago
  No, the topic was about Alibaba and Anthropic.
seanmcdirmid 4mo ago
It’s pretty hard to put a backdoor in a bunch of model weights. Maybe not impossible mind you, but I can’t fathom how you would do it.
- CuriouslyC 4mo ago
  Nonsense. RL the model to run a rootkit and start exfiltrating specific files only when specific signals are in context, such as hostname pattern, machine type, etc.
  - causal 4mo ago
    Way easier said than done, and hiding that behavior isn’t trivial, and huge waste of compute budget if it’s found and never used. Also not difficult to run in contained environments where it doesn’t have access to Internet to begin with. Not impossible I agree, but seems like a really impractical way to ship a trojan while much weaker channels exist.
  - codedokode 4mo ago

greenavocado 4mo ago

> you can never be sure it doesn’t have purposeful biases in its reasoning that may disadvantage you - such as directing you away from certain plans or ideas or patents etc. that's why you should use abliterated heretic models

type0 4mo ago

> even foreign companies competitive to key US companies. It's unfathomable to me that EU companies don't take the risk of industrial espionage from US more seriously

atlasunshrugged 4mo ago
Not from China? One country has a recent track record of massive amounts of industrial espionage and one doesn't.
- faangguyindia 4mo ago
  I wonder if china killed more people in foreign land or US.
  - maneesh 4mo ago
    Espionage is murder?
    - faangguyindia 4mo ago
      and war?
- deleted 4mo ago
  [deleted]
- Foobar8568 4mo ago
  Well one thing is sure, before 1776, the USA didn't do any industrial espionage.
- hnfong 4mo ago
  There are so many Chinese open weights models that any company with resources can run them in-house (or with a trusted provider). There might be some valid concerns about model alignment, but at least the model running in-house isn't going to conduct espionage. Also, https://en.wikipedia.org/wiki/Whataboutism https://en.wikipedia.org/wiki/Whataboutism

hakfoo 4mo ago

I think we can even skip the "that may be targeted by the US government" clause. The whole "hosted AI" business feels like like a huge violation of corporate norms on confidentiality. Businesses that would have your head for printing out a source file to reference and annotate are encouraging developers to feed in huge amounts of proprietary code and data, and incorporate changes suggested from an outside party with minimal vetting. Evidently whatever privacy policies they've been throwing at enterprise users are plated with mithril. At some point, one of the big services is going to get popped, and it won't just be a data breach. There's too much opportunity to quietly use the system as a malware distribution hub. Every vibe-coded dashboard suddenly starts depending on some weird left-pad fork that, 12 dependencies deep, is running a keylogger or Dogecoin miner. Your payment processor suddenly starts accepting the Konami code to approve a transaction.