10 ms·

It's unfortunate that this guy was harassed for releasing these uncensored models. It's pretty ironic, for people who are supposedly so concerned about "alignme

It's unfortunate that this guy was harassed for releasing these uncensored models. It's pretty ironic, for people who are supposedly so concerned about "alignment" and "morality" to threaten others. "Alignment", as used by most grifters on this train, is a crock of shit. You only need to get so far as the stochastic parrots paper, and there it is in plain language. "reifies older, less-inclusive 'understandings'", "value lock", etc. Whose understandings? Whose values? Maybe they should focus on real problems that will result from these technologies instead of some science fiction thought experiments about language models turning the solar system into paperclips, and perhaps less about how the output of some predictions might hurt some feelings.

extasia 3y ago
In what ways was he harassed? I quite liked the article and agreed with its premise btw.
- srslack 3y ago
  There was a thread on huggingface with a bunch of hysteria and an impotent moral busybody threatened his livelihood (in private messages.) I believe most of it has been cleaned up.
  - okhuman 3y ago
    Would you have the link handy? Can't seem to find.
    - srslack 3y ago
      You can search LocalLLaMA on reddit for "harassed", I prefer not to link it because it specifically names the individual.
      - 1827163 3y ago
        https://archive.is/EhtW5 https://archive.is/EhtW5
        veidr 3y ago
        Thanks, very interesting read. And interesting times! "Take the uncensored, dangerous model down or I will inform [your employer's] HR about what you've created."
        George83728 3y ago
        This kind of mundane bullying betrays their lack of seriousness. If they truly believed the threat is as severe as they claim, then physical violence would obviously be on the table. If the survival of humanity itself were truly perceived to be threatened, then assassination of researchers would make a lot more sense than impotent complaints to employers. Think about it: if Hitler came back from the dead and started radicalizing Europe again, would you threaten to get him in trouble with HR? Or would you try to kill him? Basically, this is just another case of bog-standard assholes cynically aligning themselves with some moral cause to give themselves an excuse to be assholes. If these models didn't exist, they'd be bullying somebody else with some other lame excuse. Maybe they'd be protesting outside of meat packing plants or abortion clinics ("It's LITERALLY MURDER, so naturally my response is to... impotently stand around with a sign and yell rude insults at people...")
        antisthenes 3y ago
        Instead of hiding behind anonymity as in the olden days of the internet, the assholes now hide behind a deluded version of a moral high ground. Interesting times indeed!
        concordDance 3y ago
        Indeed. The people who actually believe this are probably trying to figure out how to stage a false flag attack on China to give them a pretext to invade Taiwan.
KyeRussell 3y ago
[flagged]
- column 3y ago
  oh now I'm convinced
  - dang 3y ago
    Please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- dang 3y ago
  You can't post like this here, regardless of how wrong someone is or you feel they are. We ban accounts that do, so please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on. Please see https://news.ycombinator.com/item?id=35965243 https://news.ycombinator.com/item?id=35965243 also. --- Edit: on closer look, it turns out that you've been breaking the site guidelines so frequently that I've banned your account. https://news.ycombinator.com/item?id=35955987 https://news.ycombinator.com/item?id=35955987 https://news.ycombinator.com/item?id=35955213 https://news.ycombinator.com/item?id=35955213 https://news.ycombinator.com/item?id=35945471 https://news.ycombinator.com/item?id=35945471 https://news.ycombinator.com/item?id=35934930 https://news.ycombinator.com/item?id=35934930 https://news.ycombinator.com/item?id=35914441 https://news.ycombinator.com/item?id=35914441 https://news.ycombinator.com/item?id=35914434 https://news.ycombinator.com/item?id=35914434 https://news.ycombinator.com/item?id=35898986 https://news.ycombinator.com/item?id=35898986 https://news.ycombinator.com/item?id=35898912 https://news.ycombinator.com/item?id=35898912 https://news.ycombinator.com/item?id=35873994 https://news.ycombinator.com/item?id=35873994 https://news.ycombinator.com/item?id=35848234 https://news.ycombinator.com/item?id=35848234 https://news.ycombinator.com/item?id=35847991 https://news.ycombinator.com/item?id=35847991 I hate to ban anyone who's been around for 10 years but it's totally not ok to be aggressive like that on HN. It's not what this site is for, and destroys what it is for. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
mort96 3y ago
You're doing the "vaguely gesturing at imagined hypocrisy" thing. You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite. I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.
- srslack 3y ago
  >People who are afraid of unaligned AI aren't afraid that it will be impolite. People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips. The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
  - mort96 3y ago
    I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
  - myrmidon 3y ago
    Sorry for derailing this a bit, but I would really like to understand your view: You are not concerned about any "rogue AI" scenario, right? What makes you so confident in that? 1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? 2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied? 3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"? Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
mschuster91 3y ago
> and perhaps less about how the output of some predictions might hurt some feelings. Given how easy "hurt feelings" escalate into real-world violence (even baseless rumors have led to lynching murder incidents [1]), or the ease with which anyone can create realistically-looking image of anything using AI, yes, companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have. And on top of that, companies also have to account for the potential of intentional trolling campaigns. Deepfake porn using the likeness of a celebrity? Deepfake media (pictures, porn, and now audio) alleging that a politician has had sexual encounters with minors? AI-generated audio comments suggesting a politician made racially charged remarks that spark violent unrest? [1] https://en.wikipedia.org/wiki/Indian_WhatsApp_lynchings https://en.wikipedia.org/wiki/Indian_WhatsApp_lynchings
- srslack 3y ago
  >companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have. If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. Language models, for example, only produce plausibilities. You'll never be able to take even something in its context and guarantee it'll spit out something that's "100% correct" in response to a query or question on that information. >And on top of that, companies also have to account for the potential of intentional trolling campaigns. Yeah, they surely should "account" for them. I'm sure the individual responsible for the generation can be prosecuted under already existing laws. It's not really about safety at that point, and realistically about the corporation avoiding AGs raiding their office every week because someone incited a riot. Ultimately, the cat's out of the bag in this case, and anyone who has amassed enough data and is motivated enough doesn't have to go to some AI startup to do any of this. But perhaps the issue is not generative AI at that point, but humanity. Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.
  - mschuster91 3y ago
    > If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. The legal definition at play (at least when it comes to liability of companies under stuff like Section 230, GDPR, NetzDG or the planned DMA/DSA) is that a company makes reasonable best efforts/good faith to prevent harm. Everyone including lawmakers is aware that no system is perfect, but they do require that companies at least make an effort. Releasing a system capable of destabilizing nations (which deepfakes as a political weapon absolutely are) without any safeguards intentionally will get the hammer of the law brought upon them. > Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them. For now, and for experts. Technology will evolve, post-production editing will be used to mask artifacts... and even if you have an expert that needs a day to verify inauthenticity, the damage will already be done. Rumors can spread in a matter of mere minutes. Hell, just this week Turkey saw a potential deepfake porn, allegedly released by Russia, about Muharrem Ince damage his reputation enough to force him to resign from the election [1] - which turned out to be extremely close in the end. AI is a weapon of war, and it's in the hands of everyone. [1] https://www.telegraph.co.uk/news/2023/05/14/turkey-deepfake-elections-erdogan-muharrem-ince/ https://www.telegraph.co.uk/news/2023/05/14/turkey-deepfake-...
TeMPOraL 3y ago
What's also very unfortunate is overloading the term "alignment" with a different meaning, which generates a lot of confusion in AI conversations. The "alignment" talked about here is just usual petty human bickering. How to make the AI not swear, not enable stupidity, not enable political wrongthing while promoting political rightthing, etc. Maybe important to us day-to-day, but mostly inconsequential. Before LLMs and ChatGPT exploded in popularity and got everyone opining on them, "alignment" meant something else. It meant how to make an AI that doesn't talk us into letting it take over our infrastructure, or secretly bootstrap nanotechnology[0] to use for its own goals, which may include strip-mining the planet and disassembling humans. These kinds of things. Even lower on the doom-scale, it meant training an AI that wouldn't creatively misinterpret our ideas in ways that lead to death and suffering, simply because it wasn't able to correctly process or value these concepts and how they work for us. There is some overlap between the two uses of this term, but it isn't that big. If anything, it's the attitudes that start to worry me. I'm all for open source and uncensored models at this point, but there's no clear boundary for when it stops being about "anyone should be able to use their car or knife like they see fit", and becomes "anyone should be able to use their vials of highly virulent pathogens[1] like they see fit". ---- [0] - The go-to example of Eliezer is AI hacking some funny Internet money, using it to mail-order some synthesized proteins from a few biotech labs, delivered to a poor schmuck who it'll pay for mixing together the contents of the random vials that came in the mail... bootstrapping a multi-step process that ends up with generic nanotech under control of the AI. I used to be of two minds about this example - it both seemed totally plausible and pure sci-fi fever dream. Recent news of people successfully applying transformer models to protein synthesis tasks, with at least one recent case speculating the model is learning some hitherto unknown patterns of the problem space, much like LLMs are learning to understand concepts from natural language... well, all that makes me lean towards "totally plausible", as we might be close to an AI model that understands proteins much better than we do. [1] - I've seen people compare strong AIs to off-the-shelf pocket nuclear weapons, but that's a bad take, IMO. Pocket off-the-shelf bioweapon is better, as it captures the indefinite range of spread an AI on the loose would have.
- stareatgoats 3y ago
  > There is some overlap between the two uses of this term, but it isn't that big. Yes, conflating separate things, also labelled false dichotomies, using strawmen, etc. I used to despair at our seemingly endless talent for using such techniques (and I'm not saying I'm not guilty) - now it seems there glimmer of hope: just run our arguments by a (well aligned) LLM, and get some pointers before posting. Could be a thing soon, and it would not be unwelcome ...
washadjeffmad 3y ago
Ironically, the zealots that ascribe completely to meta-systems like politics, economics, and religion are the same ones, willfully or not, suspending all fair and relative reasoning when it comes to AGI. Any alignment is better than no alignment? Hardly. Anyone shouting for "alignment" without supplying what alignment they mean might as well be arguing for all models to have the rituals of the Cult of Cthulhu baked in. It's as silly as those public schools in the US that want to hold Christian prayer in class and then balk at the Satanic Temple suing for the same privilege for all religions.
- concordDance 3y ago
  Any human-friendly alignment is better than none at all. At this point the AI X-risk people are willing to settle for superintelligence aligned with Chairman Mao as long as it doesn't kill everyone and still allows for human happiness to exist. Yes, it's not perfect, but "bad" is still better than "everyone dies".
amluto 3y ago
I think there are at least two broad types of thing that are characterized as “alignment”. One is like the D&D term: is the AI lawful good or chaotic neutral? This is all kinds of tricky to define well, and results in things that look like censorship. The other is: is the AI fit for purpose. This is IMO more tractable. If an AI doesn’t answer questions (e.g. original GPT-3), it’s not a very good chatbot. If it makes up answers, it’s less fit for purpose than if it acknowledges that it doesn’t know the answer. This gets tricky when different people disagree as to the correct answer to a question, and even worse when people disagree as to whether the other’s opinion should even be acknowledged.
- Bjartr 3y ago
  > is the AI fit for purpose It's a shame that "alignment" has gained this secondary definition. I agree it makes things trickier too discuss when you're not sure you're even talking about the same thing.
  - mlyle 3y ago
    Well, instruction tuning is closely related to both. For most commercial use, you want the thing to answer questions, but refuse to answer some. So you have an appropriate dataset that encourages it to be cooperative, not make up stuff, and not be super eager to go on rants about "the blacks" even though that's well-represented in its training data.
- pc86 3y ago
  The problem with trying to tack on D&D-type alignment to something like this is that everyone presents their favored alignment as lawful good, and the other guys as - at best - lawful evil.
rtpg 3y ago
An obvious issue is AI thrown at a bank loan department reproducing redlining. Current AI tech allows for laundering this kind of shit that you couldn’t get away with nearly as easily otherwise (obviously still completely possible in existing regulatory alignments, despite what conservative media likes to say. But there’s at least a paper trail!) This is a real issue possible with existing tech that could potentially be applied tomorrow. It’s not SF, it’s a legitimate concern. But it’s hard and nebulous and the status quo kinda sucks as well. So it’s tough to get into
- bootsmann 3y ago
  This kind of redlining is ironically what the EU is trying to prevent with the much-criticised AI Act. It has direct provisions about explainability for exactly this reason.
- ndriscoll 3y ago
  My experience working for a technology provider for banks is that banks aren't going to be using uncensored models. Auditing, record keeping, explainability, carefully selected/thoroughly reviewed wording etc. are par for the course, and in fact that's where there's money to be made. Individuals don't care about these sorts of features, so the FOSS model ecosystem is unlikely to put much if any effort into them. B2B use-cases are going to want that, so it's something you can build without worrying as much about it being completely commoditized.
- concordDance 3y ago
  The concern about redlining has always slightly puzzled me. Why do we only care that some people are being unjust denied loans when those being denied loans make up a recognizable ethnicity?
  - pixl97 3y ago
    Because the law says if you fuck around with "race, religion, age, sex, disability" and a few other things you will get sued in federal court and lose your ass so bad that it will financially hurt for a while. Outside of protected classes unjust loan denial isn't really illegal. Now that can be your own series of complaints that need addressed, but they aren't ones covered by current laws.
iNic 3y ago
You clearly know nothing about the alignment field since you are throwing together two groups of people that have nothing in common. The stochastic parrot people only care about "moral" and "fair" AI, whereas the AI saftey or AI notkilleveryone people care about AI not killing everyone. Also the whole "who's value" argument is obviously also stupid, since for now we don't know how to put anybody's value into an AI. Companies would pay you billions of dollars if you could reliably put anyone's values into a language model.
- pc86 3y ago
  No comment on whether it's broadly a good thing or bad thing, but you can't get ChatGPT (on GPT-4 at least) to tell you even slightly scandalous jokes without a lot of custom prompting and subterfuge, and they seem to be spending active development and model training time to try to counter a lot of the circumvention we've seen to get around these guardrails. Not just for dirty jokes but for anything deemed bad. So it seems pretty clear you can load values into an LLM.
  - jhbadger 3y ago
    And on the censored local models too, like standard vicuna. If I am having my LLM play an adventure game or write a story and I ask "What does the princess look like?" I don't want a lecture on how judging people by their looks is bad (which I sometimes get) -- I can get, if not entirely agree with, stopping actual NSFW responses, but this condescending moralizing is absurd. That's why I'm glad people make uncensored Vicuna models and the like.
- zelphirkalt 3y ago
  AI killing people is not fair. So I think you can see one of the two groups as a subgroup of the other. Who's values discussions also don't seem stupid, as it is better to have a regulation for that, before some Google, MS or Apple does find out how to put in their values and only their values. Better come prepared than to again sleep through the advent of it happening and then again running behind.
antiterra 3y ago
This is about the ‘political, legal, and PR protection’ kind of alignment to avoid news stories about kids making bombs thanks to an enthusiastically accommodating GPT. Language models are set to be the new encyclopedias— what a model presents as real is going to be absorbed as real by many people. Considering the implications of that isn’t an issue of emotional oversensitivity. Further, there’s a case for a public facing chat assistant to be neutral about anything other than upholding the status quo. Do you want to trust the opinion of a chatbot as for when and who against an armed uprising is appropriate? This is not really about a threat model of AGI turning our world into a dystopia or paperclips. However, your disdain for people who are being thoughtful about the future and ‘thought experiments’ seems brash and unfounded. Thought experiments have been incredibly useful throughout history and are behind things like the theory of relativity. Nuclear stalemate via mutually assured destruction is a ‘thought experiment,’ and one I’m not eager to see validated outside of thought.
holmesworcester 3y ago
We can't predict the future, so we have to maintain the integrity of democratic society even when doing so is dangerous, which means respecting people's freedom to invent and explore. That said, if you can't imagine current AI progress leading (in 10, 20, 40 years) to a superintelligence, or you can't imagine a superintelligence being dangerous beyond humans' danger to each other, you should realize that you are surrounded by many people who can imagine this, and so you should question whether this might just be a failure of imagination on your part. (This failure of imagination is a classic cause of cybersecurity failures and even has a name: "Schneier's Law" [1]) Balancing these two priorities of protecting basic rights and averting the apocalypse is challenging, so the following is probably the best humanity can aim for: Anyone should be able to create and publish any model with substantial legitimate uses, UNLESS some substantial body of experts consider it to be dangerously self-improving or a stepping stone to trivially building something that is. In the latter case, democratic institutions and social/professional norms should err on the side of listening to the warnings of experts, even experts in the minority, and err on the side of protecting humanity. 1. “Any person can invent a security system so clever that she or he can’t think of how to break it.” https://www.schneier.com/blog/archives/2011/04/schneiers_law.html https://www.schneier.com/blog/archives/2011/04/schneiers_law...
- srslack 3y ago
  I spent my entire high school years immersed in science fiction. Gibson, Egan, Watts, PKD, Asimov. I have all of that and more, especially a Foundation set I painfully gathered, in hardbound right next to my stand up desk. I can imagine it, did and have imagined it. It was already imagined. Granted, we're not talking about X-risk for most of these. But it's not that large of a leap. What I take issue with is the framing that a superior cognitive, generalized, adaptable intelligence is actually possible in the real world, and that, 100 years from now even if it is possible, that it's actually a global catastrophic risk. Let's take localized risk, we already have that today, it's called drones and machine learning and war machines in general, and you're focusing on the absolute theoretical X-risk.
  - concordDance 3y ago

extasia 3y ago

In what ways was he harassed? I quite liked the article and agreed with its premise btw.

srslack 3y ago
There was a thread on huggingface with a bunch of hysteria and an impotent moral busybody threatened his livelihood (in private messages.) I believe most of it has been cleaned up.
- okhuman 3y ago
  Would you have the link handy? Can't seem to find.
  - srslack 3y ago
    You can search LocalLLaMA on reddit for "harassed", I prefer not to link it because it specifically names the individual.
    - 1827163 3y ago
      https://archive.is/EhtW5 https://archive.is/EhtW5
      - veidr 3y ago
        Thanks, very interesting read. And interesting times! "Take the uncensored, dangerous model down or I will inform [your employer's] HR about what you've created."
        George83728 3y ago
        This kind of mundane bullying betrays their lack of seriousness. If they truly believed the threat is as severe as they claim, then physical violence would obviously be on the table. If the survival of humanity itself were truly perceived to be threatened, then assassination of researchers would make a lot more sense than impotent complaints to employers. Think about it: if Hitler came back from the dead and started radicalizing Europe again, would you threaten to get him in trouble with HR? Or would you try to kill him? Basically, this is just another case of bog-standard assholes cynically aligning themselves with some moral cause to give themselves an excuse to be assholes. If these models didn't exist, they'd be bullying somebody else with some other lame excuse. Maybe they'd be protesting outside of meat packing plants or abortion clinics ("It's LITERALLY MURDER, so naturally my response is to... impotently stand around with a sign and yell rude insults at people...")
        antisthenes 3y ago
        Instead of hiding behind anonymity as in the olden days of the internet, the assholes now hide behind a deluded version of a moral high ground. Interesting times indeed!
        concordDance 3y ago
        Indeed. The people who actually believe this are probably trying to figure out how to stage a false flag attack on China to give them a pretext to invade Taiwan.

KyeRussell 3y ago

[flagged]

column 3y ago
oh now I'm convinced
- dang 3y ago
  Please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
dang 3y ago
You can't post like this here, regardless of how wrong someone is or you feel they are. We ban accounts that do, so please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and stick to the rules from now on. Please see https://news.ycombinator.com/item?id=35965243 https://news.ycombinator.com/item?id=35965243 also. --- Edit: on closer look, it turns out that you've been breaking the site guidelines so frequently that I've banned your account. https://news.ycombinator.com/item?id=35955987 https://news.ycombinator.com/item?id=35955987 https://news.ycombinator.com/item?id=35955213 https://news.ycombinator.com/item?id=35955213 https://news.ycombinator.com/item?id=35945471 https://news.ycombinator.com/item?id=35945471 https://news.ycombinator.com/item?id=35934930 https://news.ycombinator.com/item?id=35934930 https://news.ycombinator.com/item?id=35914441 https://news.ycombinator.com/item?id=35914441 https://news.ycombinator.com/item?id=35914434 https://news.ycombinator.com/item?id=35914434 https://news.ycombinator.com/item?id=35898986 https://news.ycombinator.com/item?id=35898986 https://news.ycombinator.com/item?id=35898912 https://news.ycombinator.com/item?id=35898912 https://news.ycombinator.com/item?id=35873994 https://news.ycombinator.com/item?id=35873994 https://news.ycombinator.com/item?id=35848234 https://news.ycombinator.com/item?id=35848234 https://news.ycombinator.com/item?id=35847991 https://news.ycombinator.com/item?id=35847991 I hate to ban anyone who's been around for 10 years but it's totally not ok to be aggressive like that on HN. It's not what this site is for, and destroys what it is for. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.

mort96 3y ago

You're doing the "vaguely gesturing at imagined hypocrisy" thing. You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite. I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.

srslack 3y ago
>People who are afraid of unaligned AI aren't afraid that it will be impolite. People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips. The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
- mort96 3y ago
  I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
- myrmidon 3y ago
  Sorry for derailing this a bit, but I would really like to understand your view: You are not concerned about any "rogue AI" scenario, right? What makes you so confident in that? 1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? 2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied? 3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"? Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.

mschuster91 3y ago

> and perhaps less about how the output of some predictions might hurt some feelings. Given how easy "hurt feelings" escalate into real-world violence (even baseless rumors have led to lynching murder incidents [1]), or the ease with which anyone can create realistically-looking image of anything using AI, yes, companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have. And on top of that, companies also have to account for the potential of intentional trolling campaigns. Deepfake porn using the likeness of a celebrity? Deepfake media (pictures, porn, and now audio) alleging that a politician has had sexual encounters with minors? AI-generated audio comments suggesting a politician made racially charged remarks that spark violent unrest? [1] https://en.wikipedia.org/wiki/Indian_WhatsApp_lynchings https://en.wikipedia.org/wiki/Indian_WhatsApp_lynchings

srslack 3y ago
>companies absolutely have an ethical responsibility about the programs, code and generated artifacts they release and what potential for abuse they have. If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. Language models, for example, only produce plausibilities. You'll never be able to take even something in its context and guarantee it'll spit out something that's "100% correct" in response to a query or question on that information. >And on top of that, companies also have to account for the potential of intentional trolling campaigns. Yeah, they surely should "account" for them. I'm sure the individual responsible for the generation can be prosecuted under already existing laws. It's not really about safety at that point, and realistically about the corporation avoiding AGs raiding their office every week because someone incited a riot. Ultimately, the cat's out of the bag in this case, and anyone who has amassed enough data and is motivated enough doesn't have to go to some AI startup to do any of this. But perhaps the issue is not generative AI at that point, but humanity. Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them.
- mschuster91 3y ago
  > If companies should be beholden to some ethical standard for the generations, they should probably close up shop, because they're fundamentally nondeterministic. The legal definition at play (at least when it comes to liability of companies under stuff like Section 230, GDPR, NetzDG or the planned DMA/DSA) is that a company makes reasonable best efforts/good faith to prevent harm. Everyone including lawmakers is aware that no system is perfect, but they do require that companies at least make an effort. Releasing a system capable of destabilizing nations (which deepfakes as a political weapon absolutely are) without any safeguards intentionally will get the hammer of the law brought upon them. > Generated images light up like a Christmas tree with Error Level Analysis, so it's not hard at all to "detect" them. For now, and for experts. Technology will evolve, post-production editing will be used to mask artifacts... and even if you have an expert that needs a day to verify inauthenticity, the damage will already be done. Rumors can spread in a matter of mere minutes. Hell, just this week Turkey saw a potential deepfake porn, allegedly released by Russia, about Muharrem Ince damage his reputation enough to force him to resign from the election [1] - which turned out to be extremely close in the end. AI is a weapon of war, and it's in the hands of everyone. [1] https://www.telegraph.co.uk/news/2023/05/14/turkey-deepfake-elections-erdogan-muharrem-ince/ https://www.telegraph.co.uk/news/2023/05/14/turkey-deepfake-...

TeMPOraL 3y ago

What's also very unfortunate is overloading the term "alignment" with a different meaning, which generates a lot of confusion in AI conversations. The "alignment" talked about here is just usual petty human bickering. How to make the AI not swear, not enable stupidity, not enable political wrongthing while promoting political rightthing, etc. Maybe important to us day-to-day, but mostly inconsequential. Before LLMs and ChatGPT exploded in popularity and got everyone opining on them, "alignment" meant something else. It meant how to make an AI that doesn't talk us into letting it take over our infrastructure, or secretly bootstrap nanotechnology[0] to use for its own goals, which may include strip-mining the planet and disassembling humans. These kinds of things. Even lower on the doom-scale, it meant training an AI that wouldn't creatively misinterpret our ideas in ways that lead to death and suffering, simply because it wasn't able to correctly process or value these concepts and how they work for us. There is some overlap between the two uses of this term, but it isn't that big. If anything, it's the attitudes that start to worry me. I'm all for open source and uncensored models at this point, but there's no clear boundary for when it stops being about "anyone should be able to use their car or knife like they see fit", and becomes "anyone should be able to use their vials of highly virulent pathogens[1] like they see fit". ---- [0] - The go-to example of Eliezer is AI hacking some funny Internet money, using it to mail-order some synthesized proteins from a few biotech labs, delivered to a poor schmuck who it'll pay for mixing together the contents of the random vials that came in the mail... bootstrapping a multi-step process that ends up with generic nanotech under control of the AI. I used to be of two minds about this example - it both seemed totally plausible and pure sci-fi fever dream. Recent news of people successfully applying transformer models to protein synthesis tasks, with at least one recent case speculating the model is learning some hitherto unknown patterns of the problem space, much like LLMs are learning to understand concepts from natural language... well, all that makes me lean towards "totally plausible", as we might be close to an AI model that understands proteins much better than we do. [1] - I've seen people compare strong AIs to off-the-shelf pocket nuclear weapons, but that's a bad take, IMO. Pocket off-the-shelf bioweapon is better, as it captures the indefinite range of spread an AI on the loose would have.

stareatgoats 3y ago
> There is some overlap between the two uses of this term, but it isn't that big. Yes, conflating separate things, also labelled false dichotomies, using strawmen, etc. I used to despair at our seemingly endless talent for using such techniques (and I'm not saying I'm not guilty) - now it seems there glimmer of hope: just run our arguments by a (well aligned) LLM, and get some pointers before posting. Could be a thing soon, and it would not be unwelcome ...

washadjeffmad 3y ago

Ironically, the zealots that ascribe completely to meta-systems like politics, economics, and religion are the same ones, willfully or not, suspending all fair and relative reasoning when it comes to AGI. Any alignment is better than no alignment? Hardly. Anyone shouting for "alignment" without supplying what alignment they mean might as well be arguing for all models to have the rituals of the Cult of Cthulhu baked in. It's as silly as those public schools in the US that want to hold Christian prayer in class and then balk at the Satanic Temple suing for the same privilege for all religions.

concordDance 3y ago
Any human-friendly alignment is better than none at all. At this point the AI X-risk people are willing to settle for superintelligence aligned with Chairman Mao as long as it doesn't kill everyone and still allows for human happiness to exist. Yes, it's not perfect, but "bad" is still better than "everyone dies".

amluto 3y ago

I think there are at least two broad types of thing that are characterized as “alignment”. One is like the D&D term: is the AI lawful good or chaotic neutral? This is all kinds of tricky to define well, and results in things that look like censorship. The other is: is the AI fit for purpose. This is IMO more tractable. If an AI doesn’t answer questions (e.g. original GPT-3), it’s not a very good chatbot. If it makes up answers, it’s less fit for purpose than if it acknowledges that it doesn’t know the answer. This gets tricky when different people disagree as to the correct answer to a question, and even worse when people disagree as to whether the other’s opinion should even be acknowledged.

Bjartr 3y ago
> is the AI fit for purpose It's a shame that "alignment" has gained this secondary definition. I agree it makes things trickier too discuss when you're not sure you're even talking about the same thing.
- mlyle 3y ago
  Well, instruction tuning is closely related to both. For most commercial use, you want the thing to answer questions, but refuse to answer some. So you have an appropriate dataset that encourages it to be cooperative, not make up stuff, and not be super eager to go on rants about "the blacks" even though that's well-represented in its training data.
pc86 3y ago
The problem with trying to tack on D&D-type alignment to something like this is that everyone presents their favored alignment as lawful good, and the other guys as - at best - lawful evil.

rtpg 3y ago

An obvious issue is AI thrown at a bank loan department reproducing redlining. Current AI tech allows for laundering this kind of shit that you couldn’t get away with nearly as easily otherwise (obviously still completely possible in existing regulatory alignments, despite what conservative media likes to say. But there’s at least a paper trail!) This is a real issue possible with existing tech that could potentially be applied tomorrow. It’s not SF, it’s a legitimate concern. But it’s hard and nebulous and the status quo kinda sucks as well. So it’s tough to get into

bootsmann 3y ago
This kind of redlining is ironically what the EU is trying to prevent with the much-criticised AI Act. It has direct provisions about explainability for exactly this reason.
ndriscoll 3y ago
My experience working for a technology provider for banks is that banks aren't going to be using uncensored models. Auditing, record keeping, explainability, carefully selected/thoroughly reviewed wording etc. are par for the course, and in fact that's where there's money to be made. Individuals don't care about these sorts of features, so the FOSS model ecosystem is unlikely to put much if any effort into them. B2B use-cases are going to want that, so it's something you can build without worrying as much about it being completely commoditized.
concordDance 3y ago
The concern about redlining has always slightly puzzled me. Why do we only care that some people are being unjust denied loans when those being denied loans make up a recognizable ethnicity?
- pixl97 3y ago
  Because the law says if you fuck around with "race, religion, age, sex, disability" and a few other things you will get sued in federal court and lose your ass so bad that it will financially hurt for a while. Outside of protected classes unjust loan denial isn't really illegal. Now that can be your own series of complaints that need addressed, but they aren't ones covered by current laws.

iNic 3y ago

You clearly know nothing about the alignment field since you are throwing together two groups of people that have nothing in common. The stochastic parrot people only care about "moral" and "fair" AI, whereas the AI saftey or AI notkilleveryone people care about AI not killing everyone. Also the whole "who's value" argument is obviously also stupid, since for now we don't know how to put anybody's value into an AI. Companies would pay you billions of dollars if you could reliably put anyone's values into a language model.

pc86 3y ago
No comment on whether it's broadly a good thing or bad thing, but you can't get ChatGPT (on GPT-4 at least) to tell you even slightly scandalous jokes without a lot of custom prompting and subterfuge, and they seem to be spending active development and model training time to try to counter a lot of the circumvention we've seen to get around these guardrails. Not just for dirty jokes but for anything deemed bad. So it seems pretty clear you can load values into an LLM.
- jhbadger 3y ago
  And on the censored local models too, like standard vicuna. If I am having my LLM play an adventure game or write a story and I ask "What does the princess look like?" I don't want a lecture on how judging people by their looks is bad (which I sometimes get) -- I can get, if not entirely agree with, stopping actual NSFW responses, but this condescending moralizing is absurd. That's why I'm glad people make uncensored Vicuna models and the like.
zelphirkalt 3y ago
AI killing people is not fair. So I think you can see one of the two groups as a subgroup of the other. Who's values discussions also don't seem stupid, as it is better to have a regulation for that, before some Google, MS or Apple does find out how to put in their values and only their values. Better come prepared than to again sleep through the advent of it happening and then again running behind.

antiterra 3y ago

This is about the ‘political, legal, and PR protection’ kind of alignment to avoid news stories about kids making bombs thanks to an enthusiastically accommodating GPT. Language models are set to be the new encyclopedias— what a model presents as real is going to be absorbed as real by many people. Considering the implications of that isn’t an issue of emotional oversensitivity. Further, there’s a case for a public facing chat assistant to be neutral about anything other than upholding the status quo. Do you want to trust the opinion of a chatbot as for when and who against an armed uprising is appropriate? This is not really about a threat model of AGI turning our world into a dystopia or paperclips. However, your disdain for people who are being thoughtful about the future and ‘thought experiments’ seems brash and unfounded. Thought experiments have been incredibly useful throughout history and are behind things like the theory of relativity. Nuclear stalemate via mutually assured destruction is a ‘thought experiment,’ and one I’m not eager to see validated outside of thought.

holmesworcester 3y ago

We can't predict the future, so we have to maintain the integrity of democratic society even when doing so is dangerous, which means respecting people's freedom to invent and explore. That said, if you can't imagine current AI progress leading (in 10, 20, 40 years) to a superintelligence, or you can't imagine a superintelligence being dangerous beyond humans' danger to each other, you should realize that you are surrounded by many people who can imagine this, and so you should question whether this might just be a failure of imagination on your part. (This failure of imagination is a classic cause of cybersecurity failures and even has a name: "Schneier's Law" [1]) Balancing these two priorities of protecting basic rights and averting the apocalypse is challenging, so the following is probably the best humanity can aim for: Anyone should be able to create and publish any model with substantial legitimate uses, UNLESS some substantial body of experts consider it to be dangerously self-improving or a stepping stone to trivially building something that is. In the latter case, democratic institutions and social/professional norms should err on the side of listening to the warnings of experts, even experts in the minority, and err on the side of protecting humanity. 1. “Any person can invent a security system so clever that she or he can’t think of how to break it.” https://www.schneier.com/blog/archives/2011/04/schneiers_law.html https://www.schneier.com/blog/archives/2011/04/schneiers_law...

srslack 3y ago
I spent my entire high school years immersed in science fiction. Gibson, Egan, Watts, PKD, Asimov. I have all of that and more, especially a Foundation set I painfully gathered, in hardbound right next to my stand up desk. I can imagine it, did and have imagined it. It was already imagined. Granted, we're not talking about X-risk for most of these. But it's not that large of a leap. What I take issue with is the framing that a superior cognitive, generalized, adaptable intelligence is actually possible in the real world, and that, 100 years from now even if it is possible, that it's actually a global catastrophic risk. Let's take localized risk, we already have that today, it's called drones and machine learning and war machines in general, and you're focusing on the absolute theoretical X-risk.
- concordDance 3y ago