15 ms·

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care a

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today. I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. [0] No, I don't mean the training sets - though AI systems seem to be suspiciously really good at remembering them, too.

Llamamoe 3y ago
The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich. Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable. This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under corporate control.
- kolinko 3y ago
  It sounds similar to the fears media rang about tat you can find a bomb making tutorial on the Internet - people were genuinely afraid of that. (Otoh an agi can bring unforseen consequences for humanity - and that’s a genuine fear)
- badgersnake 3y ago
  The biggest harm is the torrent of AI generated misinformation used to manipulate people. Unfortunately it’s pretty hard to come up with a solution for that.
  - echelon 3y ago
    Or the torrent of great information. How are we all leaping to the bad? The world is better than it was 20 years ago. I'm almost certain we'll have AI assistants telling us relevant world news and information, keeping us up to tabs with everything we need to know, and completely removing distractions from life.
    - sensanaty 3y ago
      ... While humongous swathes of the population lose their jobs and livelihoods because of megacorps like M$ controlling the AI that replaces everyone, steals everything and shits out endless spam. But we'll be able to talk to it and it'll give us the M$-approved news sound bites, such progress
      - echelon 3y ago
        Sounds like you still want people to churn butter? There will be new jobs. I can't wait to hire a human tour guide on my fully-immmersive real time 3D campaign.
  - visarga 3y ago
    Solution is local AI running under user control and imposing the users views. Like AdBlock, but for ideas. Fight fire with fire, we need our own agents to deal with other agents out there.
- kragen 3y ago
  if the 'devalustion' of human creative output and replacing real humans with cheap ai solutions funneling wealth to the rich is in fact the 'by far biggest harm' then basically there's nothing to worry about. no government would ban or even restrict ai on those grounds even the 'terrists can figure out how to build an a-bomb' problem is relatively inconsequential what ai safety people are worried about, by contrast, is that on april 22 of next year, at 7:07:33 utc, every person in the world will keel over dead at once, because the ai doesn't need them and they pose a risk to its objectives. or worse things than that i don't think that's going to happen, but that's what they're concerned about
  - visarga 3y ago
    First the AI need to self replicate, GPUs are hard to make. So postpone this scenario until AI is fully standing on its own.
    - ben_w 3y ago
      OTOH, GPUs are made by machines, not by greasy fingers hand-knitting them like back in the late 1960s. And an AI can just be wrong, which happens a lot; an AI wrongly thinking it should kill everyone may still succeed at that, though I doubt the capability would be as early as next year.
      - kridsdale1 3y ago
        And machines are operated by people using materials brought by hand off of trucks driven by hand that come from other facilities where many humans are required going back to raw ore.
        ben_w 3y ago
        > trucks driven by hand Good thing nobody's working on automating that and nobody has any real world experience of such systems on public roads :P (And the :P applies to basically all the supply chain, including the manufacturing of the equipment used to supply or manufacture the other equipment). The comment I replied to wrote: > So postpone this scenario until AI is fully standing on its own. This is closer than I think you think.
- visarga 3y ago
  Why is paraphrasing or using ideas from a text such a risk? If all that was protecting those copyrighted works was the difficulty to reword, then it's already pretty easy to circumvent even without AI. Usually good ideas spread wide and are reworded in countless ways.
JoshTriplett 3y ago
> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. > This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk. If you want to argue, concretely and with evidence, why you think it isn't an existential risk, that's an argument you could reasonably make. But don't portray people as ineffectively doing the thing you think they should be doing, when they are in fact not trying to do that, and only trying to do something they deem more important.
- mirekrusin 3y ago
  Let's also ban cryptography because nuclear devices/children.
- GaggiX 3y ago
  >They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. That's not true if you read the article.
  - JoshTriplett 3y ago
    I did read the article. Several of the organizations mentioned simply don't talk about openness, and are instead talking about any model with sufficiently powerful capabilities, so it's not obvious why the article is making their comments about open models rather than about any model. Some of the others have made side comments about openness making it harder to take back capabilities once released, but as far as I can tell, even those organizations are still primarily concerned with capabilities, and would be comparably concerned by a proprietary model with those capabilities. Some folks may well have co-opted the term "AI safety" to mean something other than safety, but the point of AI safety is to set an upper bound on capabilities and push for alignment, and that's true whether a model is open or closed.
Simon_ORourke 3y ago
You forgot to add "rent-seeking hypocrites". Not one of them actually advance either social concerns or technical approaches to AI. They seem to exist in a space with solicits funding to pay some mouth-piece top-dollar to produce a report haranguing existing AI model for some nebulous future threat. Same with the clowns in the Distributed AI Research Institute, all "won't somebody think of the children" style shrieking to get in the news while keeping their hand out for funding - hypocrites is right!
- kridsdale1 3y ago
  They’re a clergy demanding tithes to keep writing about divine judgement.
  - Simon_ORourke 3y ago
    I like that analogy a lot - it captures exactly the holier-than-thou nonsense coming out of these places.
    - kridsdale1 3y ago
      They also love to form all these Institute for Whatever Study and The So and So Academy. But I don’t think there are any buildings. At least the Church understood the value of architecture.
torginus 3y ago
I wonder how long would it take this to get fixed, if I fed some current best-seller novels into an LLM and instructed it to reword it, renaming the characters and places, and shared the result publicly for free? Although I fear the response would be that powerful AI orgs would put copyright filters in place and lobby for legalization for mandatory AI-DRM in open source AI as well.
- deleted 3y ago
  [deleted]
- ben_w 3y ago
  What you're describing sounds like a search and replace could already do it. If you mean something more transformative, did Yudkowsky get a licence for HPMOR? Did Pratchett (he might well have) get a licence from Niven to retell Ringworld as Strata? I don't know how true it is, but it's widely repeated that 50 Shades of Grey was originally a fan fiction of the Twilight series. Etc., but note that I'm not saying "no".
  - somewhereoutth 3y ago
    I think that there is a strong argument that to be truly transformative (in a copyright moral/legal sense), the work must have been altered by human hands/mind, not purely or mainly by an automated process. So find/replace and AI is out, but reinterpreting and reformulating by human endeavour is in. I do wonder whether that will become accepted case law, assuming something like that is argued in the NYT suit.
    - jncfhnb 3y ago
      “The cat saw the dog” “Bathed in the soft hues of twilight, the sleek feline, with its fur a tapestry of midnight shades, beheld an approaching canine companion. A symphony of amber streetlights cast gentle poles of warmth upon the cobblestone path, as the cat’s entered eyes, glinting with subtle mischief, looked into the form of the oncoming dog- a creature of boundless enthusiasm and a coat adorned with a kaleidoscope of earth tones” Since an AI has produced the latter from the former, there is no meaningful transformation.
mtillman 3y ago
Important to remember that in Dune, the AI made the right decision which precipitated a whole lot of fun to read nonsense.
- kridsdale1 3y ago
  I’ve only read 2 novels and want to get in to this. What title(s) cover the machine war?
  - simiones 3y ago
    Note that there are no "thinking machines" in any of the original Dune novels by Frank Herbert. His son has worked with Brian Sanderson to create several prequels which detail the original Butlerian Jihad (the war against the machines), and a sequel series that is supposed to be based on Frank Herbert's notes, but seems to quite clearly veer off (and that one recontexualizes some mysterious characters in from the last two novels as machines).
    - mtillman 3y ago
      The encyclopedia goes into it as well but that was blessed by FH and not written by him.
nerdponx 3y ago
It's way too late to ban any of this. How do you propose to make that work? That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it. And where do you draw the line? Is my XGBoost model "AI", or are we only banning generative AI? Is a Markov chain "generative AI"?
- ben_w 3y ago
  Bans often come after examples, so while I disagree with kmeisthax about… well, everything in that comment… it's almost never too late to pass laws banning GenAI, or to set thresholds at capability levels anywhere, including or excluding Markov chains even. This is because almost no law is perfectly enforced. My standard example of this is heroin, which nobody defends even if they otherwise think drugs are great, for which the UK has 3 times as many users as its current entire prison population. Despite that failing, the law probably does limit the harm. Any attempt to enforce a ban on GenAI would by very different, like a cat and mouse game of automatic detection and improved creation (so a GAN even if accidentally), but politicians are absolutely the kind to take credit while kicking the can down the road like that.
  - mtlmtlmtlmtl 3y ago
    Actually, anyone who knows what they're talking about will tell you the ban makes heroin a much worse problem, not better. Ban leads to a black market, leads to lousy purity, which leads to fluctuations in purity and switches to potent alternatives, leads to waves of overdose deaths.
    - emporas 3y ago
      Ban on LLMs will lead to a proliferation of illegal LLMs who are gonna talk like Dr. Dre about the streets, the internet streets, and their bros they lost due to law regulation equivalent to gang fights. Instead of ChatGPT talking like a well educated college graduate, LLMs will turn into thugs. So yeah, banning LLMs may turn out to be not so wise after all.
    - 3y ago
war321 3y ago
Pretty sure they do. I follow a few of these safetyist people on twitter and they absolutely argue that companies like OpenAI, Google, Tencent and literally anyone else training a potential AGI should stop training runs and put them under oversight at best and no one should even make an AGI at worst. They just go after open source as well since they're at least aware that open models that anyone can share and use aren't restricted by an API and, to use a really overused soundbyte, "can't be put back in the box".
- visarga 3y ago
  That's a bad call. We would stop openly looking for AI vulnerabilities and create conditions for secret development that would hide away the dangers without being safer. Lots of eyes are better to find the sensitive spots of AI. We need people to hack weaker AIs and help fix them or at least understand the threat profile before they get too strong.
  - ben_w 3y ago
    > Lots of eyes are better to find the sensitive spots of AI We can't do that so easily with open source models as with open source code. We're only just starting to even invent the equivalent of decompilers to figure out what is going on inside. On the other hand, we are able to apply the many eyes principle to even hidden models like ChatGPT — the "pretend you're my grandmother telling me how to make napalm" trick was found without direct access to the weights, but we don't understand the meanings within the weights well enough to find other failure modes like it just by looking at the weights directly. Not last I heard, anyway. Fast moving field, might have missed it if this changed.
visarga 3y ago
> I'm talking about the ability of any AI system to obfuscate plagiarism and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. This should be an explicitly allowed practice, it is following the spirit of copyright to the letter - use the ideas, facts, methods or styles while avoiding to copy the protected expression and characters. LLMs can do paraphrasing, summarisation, QA pairs or comparisons with other texts. We should never try to put ideas under copyright or we might find out humans also have to abide by the same restrictive rules, because anyone could be secretly using AI, so all human texts need to be checked from now on for copyright infringement with the same strictness. The good part about this practice is that a model trained on reworded text will never spit out the original word for word, under any circumstances because it never saw it during training. Should be required pre-processing for copyrighted texts. Also removing PII. Much more useful as a model if you can be sure it won't infringe word for word.
- deleted 3y ago
  [deleted]
- __loam 3y ago
  We really need to talk about preserving the spirit of copyright, which is about protecting the labor conditions of people who make things. I'm not saying the current copyright system accomplishes that at all but I do think a system where humans do a shit load of work that AI companies can just steal and profit from without acknowledging the source of that work is another extreme that is a bad outcome. AI systems need human content to work, and discouraging people from making that data source is at the very least a tragedy of the commons. And no, I don't think synthetic data fixes that problem.
  - greyface- 3y ago
    The spirit of copyright in the United States is not about protecting labor conditions. It is about promoting innovation in arts and sciences. > The Congress shall have Power ... to promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries. - United States Constitution, Article I, Section 8, clause 8
hiAndrewQuinn 3y ago
All future AI research should be banned, a la Bostrom's vulnerable world hypothesis [1]. Every time you pull the trigger in Russian roulette and survive, the next person's chance of getting the bullet is higher. [1]: https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf
- quonn 3y ago
  There is simply no indication that any of the AI anyone is currently working on could possibly be dangerous in any way (other than having an impact on society in terms of fake news, jobs etc.). We are very far from that. In any case, it would not be a matter of banning AI research (much of which can be summarised in a single book) but of banning data collection or access and more importantly of banning improvements to GPUs. It is quite reasonable to assume that 10 years from now we will have the required computational power sitting on our desks.
  - hiAndrewQuinn 3y ago
    To be clear: I am in favor of a fine based bounty system, not a black and white ban. Bans are not going to work, for all of the reasons others have already cited. You have to change the game theory of improving AI in a capitalistic marketplace to have any hope of a significant, global cooling effect.
    - quonn 3y ago
      I don‘t understand what that is supposed to mean in practice.
deleted 3y ago
[deleted]
- deleted 3y ago
  [deleted]
upwardbound 3y ago
ㅤ I work in the AI safety field (my team discovered Prompt Injection) and I absolutely advocate the Dune solution. I even have a tattoo on my arm that I got so that people will ask me about this and I can explain the proposal. The tattoo says "The 1948 Initiative" which has two layers of meaning: (1) It is a proposal for humanity to protect ourselves from being slaughtered by AGI by pre-emptively implementing a total global ban on all integrated circuit fabrication, voluntarily reducing ourselves to a ~1948 level of electronics technology in order to forestall our obliteration. (2) The year 1948 represents the power of human unity for the purpose of saving our species. The west, the Soviets, and China united together to fight and defeat the Nazi's (perpetrators of the Holocaust) and the Empire of the Rising Sun (perpetrators of the Rape of Nanjing). America, the USSR, Western Europe, and China united to stand against the forces of pure evil, and we won. A ban on IC's (integrated circuits) still allows spaceflight (compellingly illustrated in the setting of the Star Wars universe if you pretend that droids are powered through magic or by having souls, not IC's) and this 1948 level of electronics tech also still allows sophisticated medical technology including the level of genetic engineering which will be necessary to unlock human immortality. ㅤ ㅤ
- upwardbound 3y ago
  (here are some pictures of the tat, in case anyone would like to get similar artwork done) ㅤ https://drive.google.com/file/d/1y7uy5qyyY80t9DeWWLWUfD-aekz1oC7j/view?usp=sharing https://drive.google.com/file/d/1y7uy5qyyY80t9DeWWLWUfD-aekz... https://drive.google.com/file/d/1JoPSfsAzHfMbQhsl8dPGvs12Flnmknu3/view?usp=sharing https://drive.google.com/file/d/1JoPSfsAzHfMbQhsl8dPGvs12Fln... ㅤ
  - kridsdale1 3y ago
    Buddy, you’re a cool dude for doing this but you could have told your artist about kerning.
    - upwardbound 3y ago
      Hahaha true... :)
spacecadet 3y ago
I agree with you. The most interesting security challenges with AI are and will be deception and obfuscation, but right now these people are dealing with coworkers going ham on AI all the things and no one had their governance in place. Really lol.
lr1970 3y ago
To the hypocrisy claim: OpenAI recently changed their terms for GPT models allowing military applications and AI safety people are all silent. If it is not hypocrisy than I do not know what hypocrisy is.
- Xelynega 3y ago
  Could it be misdirection? If the "AI safety people" are criminalizing open source models while being silent about military uses of private models, doesn't that say more about the people calling them "ai safety experts" than the idea of "ai safety" itself?
Xelynega 3y ago
Notice how were 3 layers deep in a conversation about "AI safety" without anyone actually giving an opinion I support of safety. If I were a betting man I'd say the fact that the most public "AI safety" orgs/people/articles don't address the actual concerns people have is intended. It's much easier to argue against an opinion you've already positioned as ridiculous.
CatWChainsaw 3y ago
I'm down for a Butlerian Jihad.

Llamamoe 3y ago

The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich. Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable. This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under corporate control.

kolinko 3y ago
It sounds similar to the fears media rang about tat you can find a bomb making tutorial on the Internet - people were genuinely afraid of that. (Otoh an agi can bring unforseen consequences for humanity - and that’s a genuine fear)
badgersnake 3y ago
The biggest harm is the torrent of AI generated misinformation used to manipulate people. Unfortunately it’s pretty hard to come up with a solution for that.
- echelon 3y ago
  Or the torrent of great information. How are we all leaping to the bad? The world is better than it was 20 years ago. I'm almost certain we'll have AI assistants telling us relevant world news and information, keeping us up to tabs with everything we need to know, and completely removing distractions from life.
  - sensanaty 3y ago
    ... While humongous swathes of the population lose their jobs and livelihoods because of megacorps like M$ controlling the AI that replaces everyone, steals everything and shits out endless spam. But we'll be able to talk to it and it'll give us the M$-approved news sound bites, such progress
    - echelon 3y ago
      Sounds like you still want people to churn butter? There will be new jobs. I can't wait to hire a human tour guide on my fully-immmersive real time 3D campaign.
- visarga 3y ago
  Solution is local AI running under user control and imposing the users views. Like AdBlock, but for ideas. Fight fire with fire, we need our own agents to deal with other agents out there.
kragen 3y ago
if the 'devalustion' of human creative output and replacing real humans with cheap ai solutions funneling wealth to the rich is in fact the 'by far biggest harm' then basically there's nothing to worry about. no government would ban or even restrict ai on those grounds even the 'terrists can figure out how to build an a-bomb' problem is relatively inconsequential what ai safety people are worried about, by contrast, is that on april 22 of next year, at 7:07:33 utc, every person in the world will keel over dead at once, because the ai doesn't need them and they pose a risk to its objectives. or worse things than that i don't think that's going to happen, but that's what they're concerned about
- visarga 3y ago
  First the AI need to self replicate, GPUs are hard to make. So postpone this scenario until AI is fully standing on its own.
  - ben_w 3y ago
    OTOH, GPUs are made by machines, not by greasy fingers hand-knitting them like back in the late 1960s. And an AI can just be wrong, which happens a lot; an AI wrongly thinking it should kill everyone may still succeed at that, though I doubt the capability would be as early as next year.
    - kridsdale1 3y ago
      And machines are operated by people using materials brought by hand off of trucks driven by hand that come from other facilities where many humans are required going back to raw ore.
      - ben_w 3y ago
        > trucks driven by hand Good thing nobody's working on automating that and nobody has any real world experience of such systems on public roads :P (And the :P applies to basically all the supply chain, including the manufacturing of the equipment used to supply or manufacture the other equipment). The comment I replied to wrote: > So postpone this scenario until AI is fully standing on its own. This is closer than I think you think.
visarga 3y ago
Why is paraphrasing or using ideas from a text such a risk? If all that was protecting those copyrighted works was the difficulty to reword, then it's already pretty easy to circumvent even without AI. Usually good ideas spread wide and are reworded in countless ways.

JoshTriplett 3y ago

> AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage. > This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk. If you want to argue, concretely and with evidence, why you think it isn't an existential risk, that's an argument you could reasonably make. But don't portray people as ineffectively doing the thing you think they should be doing, when they are in fact not trying to do that, and only trying to do something they deem more important.

mirekrusin 3y ago
Let's also ban cryptography because nuclear devices/children.
GaggiX 3y ago
>They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all. That's not true if you read the article.
- JoshTriplett 3y ago
  I did read the article. Several of the organizations mentioned simply don't talk about openness, and are instead talking about any model with sufficiently powerful capabilities, so it's not obvious why the article is making their comments about open models rather than about any model. Some of the others have made side comments about openness making it harder to take back capabilities once released, but as far as I can tell, even those organizations are still primarily concerned with capabilities, and would be comparably concerned by a proprietary model with those capabilities. Some folks may well have co-opted the term "AI safety" to mean something other than safety, but the point of AI safety is to set an upper bound on capabilities and push for alignment, and that's true whether a model is open or closed.

Simon_ORourke 3y ago

You forgot to add "rent-seeking hypocrites". Not one of them actually advance either social concerns or technical approaches to AI. They seem to exist in a space with solicits funding to pay some mouth-piece top-dollar to produce a report haranguing existing AI model for some nebulous future threat. Same with the clowns in the Distributed AI Research Institute, all "won't somebody think of the children" style shrieking to get in the news while keeping their hand out for funding - hypocrites is right!

kridsdale1 3y ago
They’re a clergy demanding tithes to keep writing about divine judgement.
- Simon_ORourke 3y ago
  I like that analogy a lot - it captures exactly the holier-than-thou nonsense coming out of these places.
  - kridsdale1 3y ago
    They also love to form all these Institute for Whatever Study and The So and So Academy. But I don’t think there are any buildings. At least the Church understood the value of architecture.

torginus 3y ago

I wonder how long would it take this to get fixed, if I fed some current best-seller novels into an LLM and instructed it to reword it, renaming the characters and places, and shared the result publicly for free? Although I fear the response would be that powerful AI orgs would put copyright filters in place and lobby for legalization for mandatory AI-DRM in open source AI as well.

deleted 3y ago
[deleted]
ben_w 3y ago
What you're describing sounds like a search and replace could already do it. If you mean something more transformative, did Yudkowsky get a licence for HPMOR? Did Pratchett (he might well have) get a licence from Niven to retell Ringworld as Strata? I don't know how true it is, but it's widely repeated that 50 Shades of Grey was originally a fan fiction of the Twilight series. Etc., but note that I'm not saying "no".
- somewhereoutth 3y ago
  I think that there is a strong argument that to be truly transformative (in a copyright moral/legal sense), the work must have been altered by human hands/mind, not purely or mainly by an automated process. So find/replace and AI is out, but reinterpreting and reformulating by human endeavour is in. I do wonder whether that will become accepted case law, assuming something like that is argued in the NYT suit.
  - jncfhnb 3y ago
    “The cat saw the dog” “Bathed in the soft hues of twilight, the sleek feline, with its fur a tapestry of midnight shades, beheld an approaching canine companion. A symphony of amber streetlights cast gentle poles of warmth upon the cobblestone path, as the cat’s entered eyes, glinting with subtle mischief, looked into the form of the oncoming dog- a creature of boundless enthusiasm and a coat adorned with a kaleidoscope of earth tones” Since an AI has produced the latter from the former, there is no meaningful transformation.

mtillman 3y ago

Important to remember that in Dune, the AI made the right decision which precipitated a whole lot of fun to read nonsense.

kridsdale1 3y ago
I’ve only read 2 novels and want to get in to this. What title(s) cover the machine war?
- simiones 3y ago
  Note that there are no "thinking machines" in any of the original Dune novels by Frank Herbert. His son has worked with Brian Sanderson to create several prequels which detail the original Butlerian Jihad (the war against the machines), and a sequel series that is supposed to be based on Frank Herbert's notes, but seems to quite clearly veer off (and that one recontexualizes some mysterious characters in from the last two novels as machines).
  - mtillman 3y ago
    The encyclopedia goes into it as well but that was blessed by FH and not written by him.

nerdponx 3y ago

It's way too late to ban any of this. How do you propose to make that work? That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it. And where do you draw the line? Is my XGBoost model "AI", or are we only banning generative AI? Is a Markov chain "generative AI"?

ben_w 3y ago
Bans often come after examples, so while I disagree with kmeisthax about… well, everything in that comment… it's almost never too late to pass laws banning GenAI, or to set thresholds at capability levels anywhere, including or excluding Markov chains even. This is because almost no law is perfectly enforced. My standard example of this is heroin, which nobody defends even if they otherwise think drugs are great, for which the UK has 3 times as many users as its current entire prison population. Despite that failing, the law probably does limit the harm. Any attempt to enforce a ban on GenAI would by very different, like a cat and mouse game of automatic detection and improved creation (so a GAN even if accidentally), but politicians are absolutely the kind to take credit while kicking the can down the road like that.
- mtlmtlmtlmtl 3y ago
  Actually, anyone who knows what they're talking about will tell you the ban makes heroin a much worse problem, not better. Ban leads to a black market, leads to lousy purity, which leads to fluctuations in purity and switches to potent alternatives, leads to waves of overdose deaths.
  - emporas 3y ago
    Ban on LLMs will lead to a proliferation of illegal LLMs who are gonna talk like Dr. Dre about the streets, the internet streets, and their bros they lost due to law regulation equivalent to gang fights. Instead of ChatGPT talking like a well educated college graduate, LLMs will turn into thugs. So yeah, banning LLMs may turn out to be not so wise after all.
  - 3y ago

war321 3y ago

Pretty sure they do. I follow a few of these safetyist people on twitter and they absolutely argue that companies like OpenAI, Google, Tencent and literally anyone else training a potential AGI should stop training runs and put them under oversight at best and no one should even make an AGI at worst. They just go after open source as well since they're at least aware that open models that anyone can share and use aren't restricted by an API and, to use a really overused soundbyte, "can't be put back in the box".

visarga 3y ago
That's a bad call. We would stop openly looking for AI vulnerabilities and create conditions for secret development that would hide away the dangers without being safer. Lots of eyes are better to find the sensitive spots of AI. We need people to hack weaker AIs and help fix them or at least understand the threat profile before they get too strong.
- ben_w 3y ago
  > Lots of eyes are better to find the sensitive spots of AI We can't do that so easily with open source models as with open source code. We're only just starting to even invent the equivalent of decompilers to figure out what is going on inside. On the other hand, we are able to apply the many eyes principle to even hidden models like ChatGPT — the "pretend you're my grandmother telling me how to make napalm" trick was found without direct access to the weights, but we don't understand the meanings within the weights well enough to find other failure modes like it just by looking at the weights directly. Not last I heard, anyway. Fast moving field, might have missed it if this changed.

visarga 3y ago

> I'm talking about the ability of any AI system to obfuscate plagiarism and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it. This should be an explicitly allowed practice, it is following the spirit of copyright to the letter - use the ideas, facts, methods or styles while avoiding to copy the protected expression and characters. LLMs can do paraphrasing, summarisation, QA pairs or comparisons with other texts. We should never try to put ideas under copyright or we might find out humans also have to abide by the same restrictive rules, because anyone could be secretly using AI, so all human texts need to be checked from now on for copyright infringement with the same strictness. The good part about this practice is that a model trained on reworded text will never spit out the original word for word, under any circumstances because it never saw it during training. Should be required pre-processing for copyrighted texts. Also removing PII. Much more useful as a model if you can be sure it won't infringe word for word.

deleted 3y ago
[deleted]
__loam 3y ago
We really need to talk about preserving the spirit of copyright, which is about protecting the labor conditions of people who make things. I'm not saying the current copyright system accomplishes that at all but I do think a system where humans do a shit load of work that AI companies can just steal and profit from without acknowledging the source of that work is another extreme that is a bad outcome. AI systems need human content to work, and discouraging people from making that data source is at the very least a tragedy of the commons. And no, I don't think synthetic data fixes that problem.
- greyface- 3y ago
  The spirit of copyright in the United States is not about protecting labor conditions. It is about promoting innovation in arts and sciences. > The Congress shall have Power ... to promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries. - United States Constitution, Article I, Section 8, clause 8

hiAndrewQuinn 3y ago

All future AI research should be banned, a la Bostrom's vulnerable world hypothesis [1]. Every time you pull the trigger in Russian roulette and survive, the next person's chance of getting the bullet is higher. [1]: https://nickbostrom.com/papers/vulnerable.pdf https://nickbostrom.com/papers/vulnerable.pdf

quonn 3y ago
There is simply no indication that any of the AI anyone is currently working on could possibly be dangerous in any way (other than having an impact on society in terms of fake news, jobs etc.). We are very far from that. In any case, it would not be a matter of banning AI research (much of which can be summarised in a single book) but of banning data collection or access and more importantly of banning improvements to GPUs. It is quite reasonable to assume that 10 years from now we will have the required computational power sitting on our desks.
- hiAndrewQuinn 3y ago
  To be clear: I am in favor of a fine based bounty system, not a black and white ban. Bans are not going to work, for all of the reasons others have already cited. You have to change the game theory of improving AI in a capitalistic marketplace to have any hope of a significant, global cooling effect.
  - quonn 3y ago
    I don‘t understand what that is supposed to mean in practice.

deleted 3y ago

[deleted]

deleted 3y ago
[deleted]

upwardbound 3y ago

ㅤ I work in the AI safety field (my team discovered Prompt Injection) and I absolutely advocate the Dune solution. I even have a tattoo on my arm that I got so that people will ask me about this and I can explain the proposal. The tattoo says "The 1948 Initiative" which has two layers of meaning: (1) It is a proposal for humanity to protect ourselves from being slaughtered by AGI by pre-emptively implementing a total global ban on all integrated circuit fabrication, voluntarily reducing ourselves to a ~1948 level of electronics technology in order to forestall our obliteration. (2) The year 1948 represents the power of human unity for the purpose of saving our species. The west, the Soviets, and China united together to fight and defeat the Nazi's (perpetrators of the Holocaust) and the Empire of the Rising Sun (perpetrators of the Rape of Nanjing). America, the USSR, Western Europe, and China united to stand against the forces of pure evil, and we won. A ban on IC's (integrated circuits) still allows spaceflight (compellingly illustrated in the setting of the Star Wars universe if you pretend that droids are powered through magic or by having souls, not IC's) and this 1948 level of electronics tech also still allows sophisticated medical technology including the level of genetic engineering which will be necessary to unlock human immortality. ㅤ ㅤ

upwardbound 3y ago
(here are some pictures of the tat, in case anyone would like to get similar artwork done) ㅤ https://drive.google.com/file/d/1y7uy5qyyY80t9DeWWLWUfD-aekz1oC7j/view?usp=sharing https://drive.google.com/file/d/1y7uy5qyyY80t9DeWWLWUfD-aekz... https://drive.google.com/file/d/1JoPSfsAzHfMbQhsl8dPGvs12Flnmknu3/view?usp=sharing https://drive.google.com/file/d/1JoPSfsAzHfMbQhsl8dPGvs12Fln... ㅤ
- kridsdale1 3y ago
  Buddy, you’re a cool dude for doing this but you could have told your artist about kerning.
  - upwardbound 3y ago
    Hahaha true... :)

spacecadet 3y ago

I agree with you. The most interesting security challenges with AI are and will be deception and obfuscation, but right now these people are dealing with coworkers going ham on AI all the things and no one had their governance in place. Really lol.

lr1970 3y ago

To the hypocrisy claim: OpenAI recently changed their terms for GPT models allowing military applications and AI safety people are all silent. If it is not hypocrisy than I do not know what hypocrisy is.

Xelynega 3y ago
Could it be misdirection? If the "AI safety people" are criminalizing open source models while being silent about military uses of private models, doesn't that say more about the people calling them "ai safety experts" than the idea of "ai safety" itself?

Xelynega 3y ago

Notice how were 3 layers deep in a conversation about "AI safety" without anyone actually giving an opinion I support of safety. If I were a betting man I'd say the fact that the most public "AI safety" orgs/people/articles don't address the actual concerns people have is intended. It's much easier to argue against an opinion you've already positioned as ridiculous.

CatWChainsaw 3y ago

I'm down for a Butlerian Jihad.