10 ms·

AI isn't like nuclear fission. You can't remotely detect that somebody is training an AI. It's far too late to sequester all the information related to AI lik

AI isn't like nuclear fission. You can't remotely detect that somebody is training an AI. It's far too late to sequester all the information related to AI like what was done with uranium enrichment. The equipment needed to train AI is cheap and ubiquitous. These "safety declarations" are toothless and impossible to enforce. You can't stop AI, you need to adapt. Video and pictures will soon have no evidentiary value. Real life relationships must be valued over online relationships because you know the other person is real. It's unfortunate, but nothing AI is "disrupting" existed 200 years ago and people will learn to adapt like they always have. To quote the fictional comic book villain Toyo Harada, "none of you can stop me. Not any one of you individually nor the whole of you collectively."

sam_lowry_ 2y ago
> You can't remotely detect that somebody is training an AI. Probably not the same way you can detect working centrifuges in Iran... but you definitely can.
- snickerbockers 2y ago
  Like what? All I can think of is tracking GPU purchases but that won't be possible when AMD and NV have viable international competitors.
  - mdhb 2y ago
    There’s a famous saying in cryptography that says “anyone is capable of building encryption algorithm that they can’t break” which I am absolutely positively sure applies here also. In a world full of sensors where everything is logged in some way or another I think that it would actually be not a straightforward activity at all to build a clandestine AI lab at any scale. In the professional intel community they have been talking about this as a general problem for at least a decade now.
    - jsty 2y ago
      > In the professional intel community they have been talking about this as a general problem for at least a decade now. As in they've been discussing detecting clandestine AI labs? Or just how almost no activity is now in principle undetectable?
      - mdhb 2y ago
        I’m referring to the wider issue of what’s referred to by the Americans as “ubiquitous technical surveillance” where they came to the kind of upsetting conclusion for them that they had a long time ago lost the ability to even operate in London without the Brits knowing. I don't think there’s a good public understanding of just how much things have changed in that space in the last decade but a huge percentage of all existing tradecraft had to be completely scrapped because not only does it not work anymore but it will put you on the enemy’s radar very early on and is actively dangerous. It’s also why I think a lot of advice I see targeted towards activist types I think is straight up a bad idea in 2025. It just typically involves a lot of things that aren’t really consistent with any kind of credible innocuous explanation and are very unusual which make you stand out from a crowd.
        snickerbockers 2y ago
        But does that apply to other countries that are operating within their own territory? China is generally the go-to 'boogeyman' when people are talking about the dangers of AI; they are intelligent and extremely industrialized, and have a history of antagonistic relationships with 'the west'. I don't think it's unreasonable to assume that they will eventually have the capability to design and produce their own GPUs capable of competing with the best of NV and AMD; how will the rest of the world know if China is producing a new AI that violates a hypothetical 'AI non-proliferation treaty'? Interesting semi-irrelevant tangent: the Cooley/Tukey 'Fast Fourier Transform' algorithm was initially created because they were negotiating arms control treaties with the Russians, but in order for that to be enforceable they needed a way to detect nuclear weapons testing; the solution was to use seismograms to detect the tremors caused by an underground nuclear detonation, and the FFT was invented in the process because they were using computers to filter for the types of tremors created by a nuclear weapon.
  - mywittyname 2y ago
    Electricity usage, network traffic patterns, etc. If a "data center" is consuming a ton of power but doesn't seem to have an alternate purpose, then it's probably training AI. And maybe it will be like detecting nuclear enrichment. Instead of hacking the firmware in a Siemens device, it's done on server hardware. Israel demonstrated absurd competence at this caliber of spycraft. Sometimes you take low-tech approaches to high tech problems. I.e., get an insider at a shipping facility to swap the labels on two pallets of GPUs, one is authentic originals from the factory and the other are hacked firmware variants of exactly the same models.
    - hn_throwaway_99 2y ago
      None of these techniques are actionable. So what, someone is training AI, it's not like anyone is proposing restricting that. People are trying to make a distinction between "bad AI" and "good AI", like that is a possibility, and that's what the argument basically is, that it's impossible to differentiate or detect the difference between those, and signing declarations pretending you can is worse than useless.
      - thorum 2y ago
        Isn’t that moving the goalposts? The claim was made that it’s impossible to detect AI training runs and investigate what’s going on or take regulatory action. In fact, it is very possible.
        hn_throwaway_99 2y ago
        2 points: 1. I was just granting the GPs point to make the broader point that, for the purposes of this original discussion about these "safety declarations", this is immaterial. These safety declarations are completely unenforceable even if you could detect that someone was training AI. 2. Now, to your point about moving the goalposts, even though I say "if you could detect that someone was training AI", I don't actually even think that is possible. There are far too many normal uses of data centers to determine if one particular use is "training an AI" vs. some other data intensive use. I mean, there have long been supercomputer centers that do stuff like weather analysis and prediction, drug discovery analysis, astronomy tools, etc. that all look pretty indistinguishable from "training an AI" from the outside.
pjc50 2y ago
> Video and pictures will soon have no evidentiary value. I think we may eventually get camera authentication as a result of this, probably legally enforced in the same way and for similar reasons as Japan enforced that digital camera shutters have to make a noise. > but nothing AI is "disrupting" existed 200 years ago 200 years ago there were about 1 billion people on earth; now there are about 8 billion. Anarchoprimitivists and degrowth people make a similar handwave about the advances of the last 200 years, but they're important to holding up the systems which keep a lot of people alive.
- snickerbockers 2y ago
  > I think we may eventually get camera authentication as a result of this, probably legally enforced in the same way and for similar reasons as Japan enforced that digital camera shutters have to make a noise. Maybe, but I'm not bullish on cryptology having a solution to this problem. Every consumer device that's interesting enough to be worth hacking gets hacked within a few years. Even if nobody ever steals the key there will inevitably be side-channel attacks to feed external pictures into the camera that it thinks are coming from its own sensors. And then there's the problem of the US government, which is known to strongarm CAs into signing fraudulent certificates. > 200 years ago there were about 1 billion people on earth; now there are about 8 billion. Anarchoprimitivists and degrowth people make a similar handwave about the advances of the last 200 years, but they're important to holding up the systems which keep a lot of people alive. I think that's a good argument against the kazinksy-ites, but I was primarily speaking towards concerns such as 'misinformation' and machines pushing humans out of jobs. We're still going to have food, medicine, and shelter. AI can't take that away; the only concern is adapting our society so that we can either feed significant populations of unproductive people, or move those people into whatever jobs machines can't do yet. We might be teetering on the edge of a dystopian techno-feudalism where a significant portion of the population languishes in slums because industry has no use for them, but that's why I said we need to adapt. There has always been something that has the potential to destroy civilization in the near future, but if you're reading this post then your ancestors weren't the ones that failed to adapt.
  - ben_w 2y ago
deleted 2y ago
[deleted]
ben_w 2y ago
> It's far too late to sequester all the information related to AI like what was done with uranium enrichment. I think this presumes that Sam Altman is correct to claim that they can scale their way to, in the practical sense of the word, AGI. If he is right about that, you are right that it's too late to hide it; if he's wrong, I think the AI architecture and/or training methods we have yet to invent are in the set of things we could usefully sequester. > The equipment needed to train AI is cheap and ubiquitous. Again, possibly: If we were already close even before DeepSeek's models, yes, the hardware is too cheap and too ubiquitous. If we're still not close even despite DeepSeek's cost reductions, then the hardware isn't cheap enough — and Yudkowsky's call for a global treaty on maximum size of data centre to be enforced by cruise missiles when governments can't or won't use police action, still makes sense.
- dragonwriter 2y ago
  > If he is right about that, you are right that it's too late to hide it; if he's wrong, I think the AI architecture and/or training methods we have yet to invent are in the set of things we could usefully sequester. If it takes software technology that we have already developed outside of secret government labs, it is probably too late to sequester it. If it takes software technology that has been developed in secret government labs, its probably too late to sequester the already public precursors with which independent development of the same technology is impossible, getting us back to the preceding. It takes software technology that hasn't been developed, we don't know what we would need to sequester, and won't until we are in one of the two preceding states. If it takes a breakthrough in hardware technology, then if we make that breakthrough in a way which doesn't become widely public and used very quickly after being made and the hardware technology is naturally amenable to control (i.e., requires distinct infrastructure of similar order to enrichment of material for nuclear weapons), maybe, with intense effort of large nations, we can sequester it to a limited club of AGI powers. I think control at all is most likely a pipe dream, but one which serves as a justification for the exercise of power in ways which will please both authoritarians and favored industry actors, and even if it is possible it is simply a recipe for a durable global hegemony of actors that cannot be relied on to be benevolent.
  - 2y ago
JoshTriplett 2y ago
> These "safety declarations" are toothless and impossible to enforce. You can't stop AI, you need to adapt. Deepfakes are a distraction from more important things here. The point of AI safety is "it doesn't matter who builds unaligned AGI, if someone builds it we all die". If you agree that unaligned AGI is a death sentence for humanity, then it's worth trying to stop it. If you think AGI is unlikely to come about at all, then it should be a no-op to say "don't build it, take steps to avoid building it". If you think AGI is going to come about and magically be aligned and not be a death sentence for humanity, pay close attention to the very large number of AI experts saying otherwise. https://en.wikipedia.org/wiki/P(doom) https://en.wikipedia.org/wiki/P(doom) If your argument is "but some experts don't believe that", ask yourself whether it's reasonable to say "well, experts disagree about whether this will kill us all, so we shouldn't do anything".
- janalsncm 2y ago
  Alignment is a completely incoherent concept. Humans do not agree on what values are correct. Why is it possible even in principle for an AI to crystallize any set of principles we all agree on?
  - hollerith 2y ago
    Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it is impossible even in principle for the parents to take the preferences of all 5 children into account in approximately equal measure as to what activities or non-activities to pursue? Also: are you pursuing a complete tangent or do you see your point as bearing on whether frontier AI research should be banned? (If so, I cannot tell whether you consider your point to support a ban or oppose a ban.)
    - janalsncm 2y ago
      The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.
htrp 2y ago
> Real life relationships must be valued over online relationships because you know the other person is real. Until we get replicants
- deadbabe 2y ago
  Of which you yourself may be one without really knowing it.
  - easton 2y ago
    Maybe you won’t! I know why I left that turtle on his back.
  - fennecfoxy 2y ago
    I never understood why this would be a problem so long as a replicant has all the rights/lifespan etc of a human being. Those are usually the plot devices I suppose, especially in the case or BR where you're hunted down/don't live as long.
moffkalast 2y ago
> because you know the other person is real Technically both are real people, one is just not human. At least by the person/people definition that would include sentient aliens and such.
hollerith 2y ago
>You can't remotely detect that somebody is training an AI. There are training runs in progress that will use billions of dollars of electricity and GPUs. Quite detectable -- and stoppable by any government that wants to stop such things from happening on territory it controls. And certainly we can reduce the economic incentive for investing money on such a run by banning AI-based services like ChatGPT.
- milesrout 2y ago
  And none of them want to do that. Why would they! AI is perfectly safe. The idea it will take over the world is ludicrous and all "AI safety" in practice seems to mean is censoring it so it won't make jokes about women or ethnic minorities.
  - deleted 2y ago
    [deleted]
  - hollerith 2y ago
    Yes, as applied to the current generation of AIs, "safety" and "alignment" refer to things like preventing the product from making jokes about women or ethnic minorities, but that is because the current generation is not powerful enough to threaten human safety and human survival. The OP in contrast is about what will happen if the labs succeed in their stated goal of creating AIs that are much more powerful.
    - Sophira 2y ago
      I think we all know this is going to happen, though. We have AIs that are capable of self-correcting the code that they write, and people have built automatic interfaces for them to receive errors that they get from compilation. We also have interfaces that can allow an AI to use a Linux terminal. It's not a stretch to imagine that somebody out there is at this very moment using these in a way that would allow an AI to be fully autonomous with creating, running, and testing its own software using unit tests it wrote itself. And while the current status of AI means that such a program is likely to just not work, you have to admit that we are very, very close to the threshold point where it will work. This on its own is not enough to threaten human safety, but toss in some bad human decisions...
timewizard 2y ago
There isn't a single AI on the face of the earth. So that's easy. Nothing to actually worry about. Other than Sam Altman and Elon Musks' pending ego fight.
parliament32 2y ago
>Video and pictures will soon have no evidentiary value. This is one bit that has a technological solution. Canon's had some version of this since the early 2000s: https://www.bhphotovideo.com/c/product/319787-REG/Canon_9314A001_DVK_E2_Data_Verification_Kit.html https://www.bhphotovideo.com/c/product/319787-REG/Canon_9314... A more recent initiative: https://c2pa.org/ https://c2pa.org/
- mzajc 2y ago
  This is purely security by obscurity. I don't see why someone with motivation and capability to forge evidence wouldn't be able to forge these signatures, considering the private keys presumably come with the camera you buy.
  - parliament32 2y ago
    Shipping secure secrets is also a somewhat solved problem: TPMs ship with EKs that, AFAIK, nobody has managed to extract (yet?): https://docs.trustauthority.intel.com/main/articles/tpm-ak-provision.html#endorsement-key-ek https://docs.trustauthority.intel.com/main/articles/tpm-ak-p...
  - rocqua 2y ago
    If you make it expensive enough to extract, and tie the private key to a real identity, then you can make it hard to abuse on scale. Here I mean that at point of sale you register yourself as owner for the camera. And you make extracting a key cost about a million. Then bulk forgeries won't happen.
    - ironmagma 2y ago
      But the whole reason video evidence exists is because cameras are cheap and everyone has one as a result.
JumpCrisscross 2y ago
> Video and pictures will soon have no evidentiary value We still accept eyewitness testimony in courts. Video and pictures will be fine, their context is what will matter. Where we'll have a generation of chaos is in the public sphere, as everyone born before somewhere between 1975 and now fails to think critically when presented with an image they'd like to believe is true.
- wand3r 2y ago
  I think we'll have a decade of chaos but not because of this. A lot of stories during the election cycle in news media and on the internet were simply Democratic or Republican "fan fiction". I don't want to make this political, I only illustrate this example to say, that I was burned in believing some of these things and you develop the muscle pretty quickly. Tweets, anecdotes, images and even stories reported by "reputable" media companies already require a degree of critical thinking. I haven't really believed in aliens existing on earth for most of my adult life. However, I have sort of come around to at least entertaining the idea in recent years but would need solid photographic or video evidence. I am now convinced that aliens could basically land in broad daylight in 3 years while being heavily photographed and it would easily be able to be explained away as AI. Especially if governments want to do propaganda or counter propaganda.
  - deleted 2y ago
    [deleted]
- Sophira 2y ago
  What happens when you eventually get read/write brain interfaces? Because I'm pretty sure that's going to happen at some point. It sounds like complete science fiction, but so did where we are with generative AI only a few decades ago.
abdullahkhalids 2y ago
You can't really tell if someone is developing chemical weapons. You can tell when such weapons are used. This is very similar to AI. Yet, the international agreements on non-use of chemical weapons have held up remarkably well.
- czhu12 2y ago
  I actually agree with you, but just wanted to bring up this interesting article challenging that: https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/ https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch... Basically claims that chemical weapons have been phased out because they aren't effective, not because we've become more moral, or international standards have been set. "During WWII, everyone seems to have expected the use of chemical weapons, but never actually found a situation where doing so was advantageous... I struggle to imagine that, with the Nazis at the very gates of Moscow, Stalin was moved either by escalation concerns or the moral compass he so clearly lacked at every other moment of his life."
- Der_Einzige 2y ago
  Really? What happened to Bashir Al Assad after he gassed his own people? Oh yeah, nothing.
  - frank_nitti 2y ago
    I’m not certain that premise is valid: https://thegrayzone.com/2021/04/18/at-un-aaron-mate-debunks-opcws-syria-lies-and-confronts-us-uk-on-cover-up/ https://thegrayzone.com/2021/04/18/at-un-aaron-mate-debunks-...
manquer 2y ago
> You can't remotely detect that somebody is training an AI. Energy use is energy use, training is still incredibly energy intensive and the GPU heat signatures are different from non GPU ones, it fairly trivial to detect large scale GPU usage. Enforcement is a different problem, and is not specific to AI, if you cannot enforce an agreement it doesn't matter if its AI or nuclear or sarin gas.
- whiplash451 2y ago
  It is a lot easier to distinguish civil/military usage of uranium than it is to distinguish "good" vs "bad" usage of a model being trained.
  - manquer 2y ago
    Not if you are making a dirty bomb . Any radioactive material even at the levels found in power reactors can be dangerous. The point is not the usage is harmful or not, almost any tech can be used for bad purposes if you wish to do so. You can put controls is the point , controls here could be agent Dameons monitoring the gpus and tallying usage to heat signals, or firmware etc . The controls on what is being trained would be at a higher level than just agents process on a gpu .
  - hollerith 2y ago
    That's why we should ban all large training runs without trying to distinguish good ones from bad ones. People will have to be satisfied with the foundation models we have now.
johnflan 2y ago
>Video and pictures will soon have no evidentiary value. Thats a very interesting point
tonymet 2y ago
The US worked with all printer manufacturers to add watermarking. In theory they could work with fabs or service providers to embed instruction detection, , similar to how hosting providers do mining instruction detection.
- bee_rider 2y ago
  A lot of AI tools are just basic linear algebra functions used cleverly. If I need a license to do a matvec I will go become a pig farmer instead.
  - tonymet 2y ago
    Same could be said about mining and the detection works
    - bee_rider 2y ago
      I mean, mining involves matvecs of course, https://sparse.tamu.edu/GHS_indef/tuma2 https://sparse.tamu.edu/GHS_indef/tuma2 but I think there are a bunch of additional steps as well, they need to pick up rocks and carry them to the surface or something.
- talldayo 2y ago
  And now counterfeiters import their printers from Temu. If China wanted a domestic training cluster they could make one, maybe not as well as Nvidia but they could certainly make one.
  - tonymet 2y ago
    All countermeasures have loss
549tj35p4tjk 2y ago
You really can. The government often knows when an individual makes a bomb in their garage. The know the recipes and they monitor the ingredients. When someone buys tens of thousands of GPUs, people notice. When someone builds a new foundry, people notice. These are enormous changes.
- fennecfoxy 2y ago
  The U.S. government can't enough stop China from Acquiring the latest Nvidia GPUs; they go through back channels. It's easy to make claims that the government knows everything and yet somehow terrorists still successfully shoot or blow up places from time to time. The number of times in Europe I've seen a bad guy be "known to police" or "on a watch-list" like what does that even mean, and why wasn't anything done about it? Because what it really means is that they've created a list due to some profiling which due to obvious flaws, can't be leveraged due to the number of false positives. It's a weird situation because I want the government to both be up in all of our business to protect us and yet not such that we still have privacy - impossible, really. If anything AI will hopefully increase the Gov's capabilities to detect nefarious shit without intruding upon privacy. Technology usually does its job perfectly, it's just misuse by us dumb humans that always causes problems.
L-four 2y ago
Videos and Pictures are not evidence. The declarations of the videos and photos to be accurate depiction of events is the evidence. The law was one step ahead the whole time.
- dragonwriter 2y ago
  > Videos and Pictures are not evidence. Legally, videos and pictures are physical evidence. > The declarations of the videos and photos to be accurate depiction of events is the evidence. No, those declarations are conclusions that are generally reserved to the trier of fact.(the jury, in a jury trial, or the judge in a bench trial.) Declarations of personal knowledge as to events in how the videos or films were created or found, etc., which can support or refute such conclusions are, OTOH, testimonial evidence, and at least some of that kind of evidence is generally necessary to support each piece of physical evidence. (And, on the other side, such evidence can be submitted/elicited by the other side to impeach the physical evidence.)
  - L-four 2y ago
    Thanks for your insight, I internally confused Fact with Evidence.

sam_lowry_ 2y ago

> You can't remotely detect that somebody is training an AI. Probably not the same way you can detect working centrifuges in Iran... but you definitely can.

snickerbockers 2y ago
Like what? All I can think of is tracking GPU purchases but that won't be possible when AMD and NV have viable international competitors.
- mdhb 2y ago
  There’s a famous saying in cryptography that says “anyone is capable of building encryption algorithm that they can’t break” which I am absolutely positively sure applies here also. In a world full of sensors where everything is logged in some way or another I think that it would actually be not a straightforward activity at all to build a clandestine AI lab at any scale. In the professional intel community they have been talking about this as a general problem for at least a decade now.
  - jsty 2y ago
    > In the professional intel community they have been talking about this as a general problem for at least a decade now. As in they've been discussing detecting clandestine AI labs? Or just how almost no activity is now in principle undetectable?
    - mdhb 2y ago
      I’m referring to the wider issue of what’s referred to by the Americans as “ubiquitous technical surveillance” where they came to the kind of upsetting conclusion for them that they had a long time ago lost the ability to even operate in London without the Brits knowing. I don't think there’s a good public understanding of just how much things have changed in that space in the last decade but a huge percentage of all existing tradecraft had to be completely scrapped because not only does it not work anymore but it will put you on the enemy’s radar very early on and is actively dangerous. It’s also why I think a lot of advice I see targeted towards activist types I think is straight up a bad idea in 2025. It just typically involves a lot of things that aren’t really consistent with any kind of credible innocuous explanation and are very unusual which make you stand out from a crowd.
      - snickerbockers 2y ago
        But does that apply to other countries that are operating within their own territory? China is generally the go-to 'boogeyman' when people are talking about the dangers of AI; they are intelligent and extremely industrialized, and have a history of antagonistic relationships with 'the west'. I don't think it's unreasonable to assume that they will eventually have the capability to design and produce their own GPUs capable of competing with the best of NV and AMD; how will the rest of the world know if China is producing a new AI that violates a hypothetical 'AI non-proliferation treaty'? Interesting semi-irrelevant tangent: the Cooley/Tukey 'Fast Fourier Transform' algorithm was initially created because they were negotiating arms control treaties with the Russians, but in order for that to be enforceable they needed a way to detect nuclear weapons testing; the solution was to use seismograms to detect the tremors caused by an underground nuclear detonation, and the FFT was invented in the process because they were using computers to filter for the types of tremors created by a nuclear weapon.
- mywittyname 2y ago
  Electricity usage, network traffic patterns, etc. If a "data center" is consuming a ton of power but doesn't seem to have an alternate purpose, then it's probably training AI. And maybe it will be like detecting nuclear enrichment. Instead of hacking the firmware in a Siemens device, it's done on server hardware. Israel demonstrated absurd competence at this caliber of spycraft. Sometimes you take low-tech approaches to high tech problems. I.e., get an insider at a shipping facility to swap the labels on two pallets of GPUs, one is authentic originals from the factory and the other are hacked firmware variants of exactly the same models.
  - hn_throwaway_99 2y ago
    None of these techniques are actionable. So what, someone is training AI, it's not like anyone is proposing restricting that. People are trying to make a distinction between "bad AI" and "good AI", like that is a possibility, and that's what the argument basically is, that it's impossible to differentiate or detect the difference between those, and signing declarations pretending you can is worse than useless.
    - thorum 2y ago
      Isn’t that moving the goalposts? The claim was made that it’s impossible to detect AI training runs and investigate what’s going on or take regulatory action. In fact, it is very possible.
      - hn_throwaway_99 2y ago
        2 points: 1. I was just granting the GPs point to make the broader point that, for the purposes of this original discussion about these "safety declarations", this is immaterial. These safety declarations are completely unenforceable even if you could detect that someone was training AI. 2. Now, to your point about moving the goalposts, even though I say "if you could detect that someone was training AI", I don't actually even think that is possible. There are far too many normal uses of data centers to determine if one particular use is "training an AI" vs. some other data intensive use. I mean, there have long been supercomputer centers that do stuff like weather analysis and prediction, drug discovery analysis, astronomy tools, etc. that all look pretty indistinguishable from "training an AI" from the outside.

pjc50 2y ago

> Video and pictures will soon have no evidentiary value. I think we may eventually get camera authentication as a result of this, probably legally enforced in the same way and for similar reasons as Japan enforced that digital camera shutters have to make a noise. > but nothing AI is "disrupting" existed 200 years ago 200 years ago there were about 1 billion people on earth; now there are about 8 billion. Anarchoprimitivists and degrowth people make a similar handwave about the advances of the last 200 years, but they're important to holding up the systems which keep a lot of people alive.

snickerbockers 2y ago
> I think we may eventually get camera authentication as a result of this, probably legally enforced in the same way and for similar reasons as Japan enforced that digital camera shutters have to make a noise. Maybe, but I'm not bullish on cryptology having a solution to this problem. Every consumer device that's interesting enough to be worth hacking gets hacked within a few years. Even if nobody ever steals the key there will inevitably be side-channel attacks to feed external pictures into the camera that it thinks are coming from its own sensors. And then there's the problem of the US government, which is known to strongarm CAs into signing fraudulent certificates. > 200 years ago there were about 1 billion people on earth; now there are about 8 billion. Anarchoprimitivists and degrowth people make a similar handwave about the advances of the last 200 years, but they're important to holding up the systems which keep a lot of people alive. I think that's a good argument against the kazinksy-ites, but I was primarily speaking towards concerns such as 'misinformation' and machines pushing humans out of jobs. We're still going to have food, medicine, and shelter. AI can't take that away; the only concern is adapting our society so that we can either feed significant populations of unproductive people, or move those people into whatever jobs machines can't do yet. We might be teetering on the edge of a dystopian techno-feudalism where a significant portion of the population languishes in slums because industry has no use for them, but that's why I said we need to adapt. There has always been something that has the potential to destroy civilization in the near future, but if you're reading this post then your ancestors weren't the ones that failed to adapt.
- ben_w 2y ago

deleted 2y ago

[deleted]

ben_w 2y ago

> It's far too late to sequester all the information related to AI like what was done with uranium enrichment. I think this presumes that Sam Altman is correct to claim that they can scale their way to, in the practical sense of the word, AGI. If he is right about that, you are right that it's too late to hide it; if he's wrong, I think the AI architecture and/or training methods we have yet to invent are in the set of things we could usefully sequester. > The equipment needed to train AI is cheap and ubiquitous. Again, possibly: If we were already close even before DeepSeek's models, yes, the hardware is too cheap and too ubiquitous. If we're still not close even despite DeepSeek's cost reductions, then the hardware isn't cheap enough — and Yudkowsky's call for a global treaty on maximum size of data centre to be enforced by cruise missiles when governments can't or won't use police action, still makes sense.

dragonwriter 2y ago
> If he is right about that, you are right that it's too late to hide it; if he's wrong, I think the AI architecture and/or training methods we have yet to invent are in the set of things we could usefully sequester. If it takes software technology that we have already developed outside of secret government labs, it is probably too late to sequester it. If it takes software technology that has been developed in secret government labs, its probably too late to sequester the already public precursors with which independent development of the same technology is impossible, getting us back to the preceding. It takes software technology that hasn't been developed, we don't know what we would need to sequester, and won't until we are in one of the two preceding states. If it takes a breakthrough in hardware technology, then if we make that breakthrough in a way which doesn't become widely public and used very quickly after being made and the hardware technology is naturally amenable to control (i.e., requires distinct infrastructure of similar order to enrichment of material for nuclear weapons), maybe, with intense effort of large nations, we can sequester it to a limited club of AGI powers. I think control at all is most likely a pipe dream, but one which serves as a justification for the exercise of power in ways which will please both authoritarians and favored industry actors, and even if it is possible it is simply a recipe for a durable global hegemony of actors that cannot be relied on to be benevolent.
- 2y ago

JoshTriplett 2y ago

> These "safety declarations" are toothless and impossible to enforce. You can't stop AI, you need to adapt. Deepfakes are a distraction from more important things here. The point of AI safety is "it doesn't matter who builds unaligned AGI, if someone builds it we all die". If you agree that unaligned AGI is a death sentence for humanity, then it's worth trying to stop it. If you think AGI is unlikely to come about at all, then it should be a no-op to say "don't build it, take steps to avoid building it". If you think AGI is going to come about and magically be aligned and not be a death sentence for humanity, pay close attention to the very large number of AI experts saying otherwise. https://en.wikipedia.org/wiki/P(doom) https://en.wikipedia.org/wiki/P(doom) If your argument is "but some experts don't believe that", ask yourself whether it's reasonable to say "well, experts disagree about whether this will kill us all, so we shouldn't do anything".

janalsncm 2y ago
Alignment is a completely incoherent concept. Humans do not agree on what values are correct. Why is it possible even in principle for an AI to crystallize any set of principles we all agree on?
- hollerith 2y ago
  Humans do not agree on what values are correct, but values can be averaged. So for example if a family with 5 children is on vacation, do you maintain that it is impossible even in principle for the parents to take the preferences of all 5 children into account in approximately equal measure as to what activities or non-activities to pursue? Also: are you pursuing a complete tangent or do you see your point as bearing on whether frontier AI research should be banned? (If so, I cannot tell whether you consider your point to support a ban or oppose a ban.)
  - janalsncm 2y ago
    The vast majority of harms from “AI” are actually harms from the corporations and governments that control them, who have mutually incompatible goals, getting what they want. This is why alignment folks at OpenAI are quickly learning that the first problem they need to solve is what happens when their values don’t align with the company’s (spoiler: they get fired). Therefore the actual solution is not coming up with more and more clever “guardrails” but aligning corporations and governments to human needs. In other words, politics. There are other problems like enabling new types of scams which will require political solutions. At a technical level the best these companies can do is mitigation.

htrp 2y ago

> Real life relationships must be valued over online relationships because you know the other person is real. Until we get replicants

deadbabe 2y ago
Of which you yourself may be one without really knowing it.
- easton 2y ago
  Maybe you won’t! I know why I left that turtle on his back.
- fennecfoxy 2y ago
  I never understood why this would be a problem so long as a replicant has all the rights/lifespan etc of a human being. Those are usually the plot devices I suppose, especially in the case or BR where you're hunted down/don't live as long.

moffkalast 2y ago

> because you know the other person is real Technically both are real people, one is just not human. At least by the person/people definition that would include sentient aliens and such.

hollerith 2y ago

>You can't remotely detect that somebody is training an AI. There are training runs in progress that will use billions of dollars of electricity and GPUs. Quite detectable -- and stoppable by any government that wants to stop such things from happening on territory it controls. And certainly we can reduce the economic incentive for investing money on such a run by banning AI-based services like ChatGPT.

milesrout 2y ago
And none of them want to do that. Why would they! AI is perfectly safe. The idea it will take over the world is ludicrous and all "AI safety" in practice seems to mean is censoring it so it won't make jokes about women or ethnic minorities.
- deleted 2y ago
  [deleted]
- hollerith 2y ago
  Yes, as applied to the current generation of AIs, "safety" and "alignment" refer to things like preventing the product from making jokes about women or ethnic minorities, but that is because the current generation is not powerful enough to threaten human safety and human survival. The OP in contrast is about what will happen if the labs succeed in their stated goal of creating AIs that are much more powerful.
  - Sophira 2y ago
    I think we all know this is going to happen, though. We have AIs that are capable of self-correcting the code that they write, and people have built automatic interfaces for them to receive errors that they get from compilation. We also have interfaces that can allow an AI to use a Linux terminal. It's not a stretch to imagine that somebody out there is at this very moment using these in a way that would allow an AI to be fully autonomous with creating, running, and testing its own software using unit tests it wrote itself. And while the current status of AI means that such a program is likely to just not work, you have to admit that we are very, very close to the threshold point where it will work. This on its own is not enough to threaten human safety, but toss in some bad human decisions...

timewizard 2y ago

There isn't a single AI on the face of the earth. So that's easy. Nothing to actually worry about. Other than Sam Altman and Elon Musks' pending ego fight.

parliament32 2y ago

>Video and pictures will soon have no evidentiary value. This is one bit that has a technological solution. Canon's had some version of this since the early 2000s: https://www.bhphotovideo.com/c/product/319787-REG/Canon_9314A001_DVK_E2_Data_Verification_Kit.html https://www.bhphotovideo.com/c/product/319787-REG/Canon_9314... A more recent initiative: https://c2pa.org/ https://c2pa.org/

mzajc 2y ago
This is purely security by obscurity. I don't see why someone with motivation and capability to forge evidence wouldn't be able to forge these signatures, considering the private keys presumably come with the camera you buy.
- parliament32 2y ago
  Shipping secure secrets is also a somewhat solved problem: TPMs ship with EKs that, AFAIK, nobody has managed to extract (yet?): https://docs.trustauthority.intel.com/main/articles/tpm-ak-provision.html#endorsement-key-ek https://docs.trustauthority.intel.com/main/articles/tpm-ak-p...
- rocqua 2y ago
  If you make it expensive enough to extract, and tie the private key to a real identity, then you can make it hard to abuse on scale. Here I mean that at point of sale you register yourself as owner for the camera. And you make extracting a key cost about a million. Then bulk forgeries won't happen.
  - ironmagma 2y ago
    But the whole reason video evidence exists is because cameras are cheap and everyone has one as a result.

JumpCrisscross 2y ago

> Video and pictures will soon have no evidentiary value We still accept eyewitness testimony in courts. Video and pictures will be fine, their context is what will matter. Where we'll have a generation of chaos is in the public sphere, as everyone born before somewhere between 1975 and now fails to think critically when presented with an image they'd like to believe is true.

wand3r 2y ago
I think we'll have a decade of chaos but not because of this. A lot of stories during the election cycle in news media and on the internet were simply Democratic or Republican "fan fiction". I don't want to make this political, I only illustrate this example to say, that I was burned in believing some of these things and you develop the muscle pretty quickly. Tweets, anecdotes, images and even stories reported by "reputable" media companies already require a degree of critical thinking. I haven't really believed in aliens existing on earth for most of my adult life. However, I have sort of come around to at least entertaining the idea in recent years but would need solid photographic or video evidence. I am now convinced that aliens could basically land in broad daylight in 3 years while being heavily photographed and it would easily be able to be explained away as AI. Especially if governments want to do propaganda or counter propaganda.
- deleted 2y ago
  [deleted]
Sophira 2y ago
What happens when you eventually get read/write brain interfaces? Because I'm pretty sure that's going to happen at some point. It sounds like complete science fiction, but so did where we are with generative AI only a few decades ago.

abdullahkhalids 2y ago

You can't really tell if someone is developing chemical weapons. You can tell when such weapons are used. This is very similar to AI. Yet, the international agreements on non-use of chemical weapons have held up remarkably well.

czhu12 2y ago
I actually agree with you, but just wanted to bring up this interesting article challenging that: https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/ https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch... Basically claims that chemical weapons have been phased out because they aren't effective, not because we've become more moral, or international standards have been set. "During WWII, everyone seems to have expected the use of chemical weapons, but never actually found a situation where doing so was advantageous... I struggle to imagine that, with the Nazis at the very gates of Moscow, Stalin was moved either by escalation concerns or the moral compass he so clearly lacked at every other moment of his life."
Der_Einzige 2y ago
Really? What happened to Bashir Al Assad after he gassed his own people? Oh yeah, nothing.
- frank_nitti 2y ago
  I’m not certain that premise is valid: https://thegrayzone.com/2021/04/18/at-un-aaron-mate-debunks-opcws-syria-lies-and-confronts-us-uk-on-cover-up/ https://thegrayzone.com/2021/04/18/at-un-aaron-mate-debunks-...

manquer 2y ago

> You can't remotely detect that somebody is training an AI. Energy use is energy use, training is still incredibly energy intensive and the GPU heat signatures are different from non GPU ones, it fairly trivial to detect large scale GPU usage. Enforcement is a different problem, and is not specific to AI, if you cannot enforce an agreement it doesn't matter if its AI or nuclear or sarin gas.

whiplash451 2y ago
It is a lot easier to distinguish civil/military usage of uranium than it is to distinguish "good" vs "bad" usage of a model being trained.
- manquer 2y ago
  Not if you are making a dirty bomb . Any radioactive material even at the levels found in power reactors can be dangerous. The point is not the usage is harmful or not, almost any tech can be used for bad purposes if you wish to do so. You can put controls is the point , controls here could be agent Dameons monitoring the gpus and tallying usage to heat signals, or firmware etc . The controls on what is being trained would be at a higher level than just agents process on a gpu .
- hollerith 2y ago
  That's why we should ban all large training runs without trying to distinguish good ones from bad ones. People will have to be satisfied with the foundation models we have now.

johnflan 2y ago

>Video and pictures will soon have no evidentiary value. Thats a very interesting point

tonymet 2y ago

The US worked with all printer manufacturers to add watermarking. In theory they could work with fabs or service providers to embed instruction detection, , similar to how hosting providers do mining instruction detection.

bee_rider 2y ago
A lot of AI tools are just basic linear algebra functions used cleverly. If I need a license to do a matvec I will go become a pig farmer instead.
- tonymet 2y ago
  Same could be said about mining and the detection works
  - bee_rider 2y ago
    I mean, mining involves matvecs of course, https://sparse.tamu.edu/GHS_indef/tuma2 https://sparse.tamu.edu/GHS_indef/tuma2 but I think there are a bunch of additional steps as well, they need to pick up rocks and carry them to the surface or something.
talldayo 2y ago
And now counterfeiters import their printers from Temu. If China wanted a domestic training cluster they could make one, maybe not as well as Nvidia but they could certainly make one.
- tonymet 2y ago
  All countermeasures have loss

549tj35p4tjk 2y ago

You really can. The government often knows when an individual makes a bomb in their garage. The know the recipes and they monitor the ingredients. When someone buys tens of thousands of GPUs, people notice. When someone builds a new foundry, people notice. These are enormous changes.

fennecfoxy 2y ago
The U.S. government can't enough stop China from Acquiring the latest Nvidia GPUs; they go through back channels. It's easy to make claims that the government knows everything and yet somehow terrorists still successfully shoot or blow up places from time to time. The number of times in Europe I've seen a bad guy be "known to police" or "on a watch-list" like what does that even mean, and why wasn't anything done about it? Because what it really means is that they've created a list due to some profiling which due to obvious flaws, can't be leveraged due to the number of false positives. It's a weird situation because I want the government to both be up in all of our business to protect us and yet not such that we still have privacy - impossible, really. If anything AI will hopefully increase the Gov's capabilities to detect nefarious shit without intruding upon privacy. Technology usually does its job perfectly, it's just misuse by us dumb humans that always causes problems.

L-four 2y ago

Videos and Pictures are not evidence. The declarations of the videos and photos to be accurate depiction of events is the evidence. The law was one step ahead the whole time.

dragonwriter 2y ago
> Videos and Pictures are not evidence. Legally, videos and pictures are physical evidence. > The declarations of the videos and photos to be accurate depiction of events is the evidence. No, those declarations are conclusions that are generally reserved to the trier of fact.(the jury, in a jury trial, or the judge in a bench trial.) Declarations of personal knowledge as to events in how the videos or films were created or found, etc., which can support or refute such conclusions are, OTOH, testimonial evidence, and at least some of that kind of evidence is generally necessary to support each piece of physical evidence. (And, on the other side, such evidence can be submitted/elicited by the other side to impeach the physical evidence.)
- L-four 2y ago
  Thanks for your insight, I internally confused Fact with Evidence.