11 ms·

I'm convinced there is a certain class of people who gravitate to positions of power, like "moderators", (partisan) journalists, etc. Now, the ultimate moderato

I'm convinced there is a certain class of people who gravitate to positions of power, like "moderators", (partisan) journalists, etc. Now, the ultimate moderator role has now been created, more powerful than moderating 1000 subreddits - the AI safety job who will control what AI "thinks"/says for "safety" reasons. Pretty soon AI will be an expert at subtly steering you toward thinking/voting for whatever the "safety" experts want. It's probably convenient for them to have everyone focused on the fear of evil Skynet wiping out humanity, while everyone is distracted from the more likely scenario of people with an agenda controlling the advice given to you by your super intelligent assistant. Because of X, we need to invade this country. Because of Y, we need to pass all these terrible laws limiting freedom. Because of Z, we need to make sure AI is "safe". For this reason, I view "safe" AIs as more dangerous than "unsafe" ones.

cyanydeez 3y ago
All you're really describing is why this shouldn't be a non-proft and should just be a government effort. But I assume, from y our language, you'd also object to making this a government utility.
- sethammons 3y ago
  > should just be a government effort And the controlling party de jour will totally not tweak it to side with their agenda, I'm sure. </s>
  - cyanydeez 3y ago
    uh. We're arguing about _who is controlling AI_. What do you image a neutral party does? If youu're talking about safety, don't you think there should be someone sitting on a boar dsomewhere, contemplating _what should the AI feed today?_ Seriously, why is a non profit, or a business or whatever any different than a government? I get it: there's all kinds of governments, but now theres all kind of businesses. The point of putting it in the governments hand is a defacto acknowledgement that it's a utility. Take other utilities, any time you give a prive org a right to control whether or not you get electricity or water, whats the outcome? Rarely good. If AI is suppose to help society, that's the purview of the government. That's all, you can imagine it's the chinese government, or the russian, or the american or the canadian. They're all _going to do it_, thats _going to happen_, and if a business gets there first, _what is the difference if it's such a powerful device_. I get it, people look dimly on governments, but guess what: they're just as powerful as some organization that gets billions of dollars to effect society. Why is it suddenly a boogeyman?
    - sethammons 3y ago
      I find any government to be more of a boogeyman than any private company because the government has the right to violence and companies come and go at a faster rate.
      - cyanydeez 3y ago
        Ok, and if Raytheon builds an AI and tells a government "trust us, its safe", arn't you just letting them create a scape goat via the government? Seriously, Businesses simply dont have the history that governments do. They're just as capable of violence. https://utopia.org/guide/crime-controversy-nestles-5-biggest-scandals-explained/ https://utopia.org/guide/crime-controversy-nestles-5-biggest... All you're identifying is "government has a longer history of violence than Businesses"
      - kjkjadksj 3y ago
        The municipal utility provider has a right to violence? The park service? Where do you live? Los Angeles during Blade Runner?
Dylan16807 3y ago
Personally, I expect the opposite camp to be just as bad about steering.
ragequittah 3y ago
[dead]
PeterStuer 3y ago
Most of those touting "safety" do not want to limit their access to and control of powerfull AI, just yours .
- vkou 3y ago
  Meanwhile, those working on commercialization are by definition going to be gatekeepers and beneficiaries of it, not you. The organizations that pay for it will pay for it to produce results that are of benefit to them, probably at my expense [1]. Do I think Helen has my interests at heart? Unlikely. Do Sam or Satya? Absolutely not! [1] I can't wait for AI doctors working for insurers to deny me treatments, AI vendors to figure out exactly how much they can charge me for their dynamically-priced product, AI answering machines to route my customer support calls through Dante's circles of hell...
  - konschubert 3y ago
    > produce results that are of benefit to them, probably at my expense The world is not zero-sum. Most economic transactions benefit both parties and are a net benefit to society, even considering externalities.
    - vkou 3y ago
      > The world is not zero-sum. No, but some parts of it very much are. The whole point of AI safety is keeping it away from those parts of the world. How are Sam and Satya going to do that? It's not in Microsoft's DNA to do that.
      - concordDance 3y ago
        > The whole point of AI safety is keeping it away from those parts of the world. No, it's to ensure it doesn't kill you and everyone you love.
        hef19898 3y ago
nostromo 3y ago
You're correct. When people say they want safe AGI, what they mean are things like "Skynet should not nuke us" and "don't accelerate so fast that humans are instantly irrelevant." But what it's being interpreted as is more like "be excessively prudish and politically correct at all times" -- which I doubt was ever really anyone's main concern with AGI.
- wisty 3y ago
  There is a middle ground, in that maybe ChatGTP shouldn't help users commit certain serious crimes. I am pretty pro free speech, and I think there's definitely a slippery slope here, but there is a bit of justification.
  - StanislavPetrov 3y ago
    Which users? The greatest crimes, by far, are committed by the US government (and other governments around the world) - and you can be sure that AI and/or AGI will be designed to help them commit their crimes more efficiently, effectively and to manufacture consent to do so.
  - hef19898 3y ago
    I am a little less free speech than Americans, in Germany we have serious limitations around hate speech and holicaust denial for example. Putting thise restrictions into a tool like ChatGPT goes to far so, because so far AI still needs a prompt to do anything. The problem I see, is with ChatGPT, being trained on a lot hate speech or prpopagabda, slipts in those things even if not prompted to. Which, and I am by no means an AI expert not by far, seems to be a sub-problem of the hallucination problems of making stuff up. Because we have to remind ourselves, AI so far is glorified mavhine learning creating content, it is not concient. But it can be used to create a lot of propaganda and deffamation content at unprecedented scale and speed. And that is the real problem.
    - freedomben 3y ago
      Apologies this is very off topic, but I don't know anyone from Germany that I can ask and you opened the door a tiny bit by mentioning the holocaust :-) I've been trying to really understand the situation and how Hitler was able to rise to power. The horrendous conditions placed on Germany after WWI and the Weimar Republic for example have really enlightened me. Have you read any of the big books on the subject that you could recommend? I'm reading Ian Kershaw's two-part series on Hitler, and William Shirer's "Collapse of the Third Republic" and "Rise and Fall of the Third Reich". Have you read any of those, or do you have books you would recommend?
phreeza 3y ago
If you believe the other side in this rift is not also striving to put themselves in positions of power, I think you are wrong. They are just going to use that power to manipulate the public in a different way. The real alternative are truly open models, not Models controlled by slightly different elite interests.
davedx 3y ago
Wow, what an incredibly bad faith characterization of the OpenAI board? This kind of speculative mud slinging makes this place seem more like a gossip forum.
- sho_hn 3y ago
  Most of the comments on Hacker News are written by folks who a much easier time & would rather imagine themselves as a CEO, than as a non-profit board member. There is little regard for the latter. As a non-profit board member, I'm curious why their bylaws are so crummy that the rest of the board could simply remove two others on the board. That's not exactly cunning design of your articles of association ... :-)
- ssnistfajen 3y ago
  This place was never above being a gossip forum, especially on topics that involve any ounce of politics or social sciences.
  - 93po 3y ago
    Strong agree. HN is like anywhere else on the internet but with with a bit more dry content (no memes and images etc) so it attracts an older crowd. It does, however, have great gems of comments and people who raise the bar. But it's still amongst a sea of general quick-to-anger and loosely held opinions stated as fact - which I am guilty of myself sometimes. Less so these days.
- Rastonbury 3y ago
  I have no words for that comment. As if its so unbelievable that someone would want to prevent rogue AI or wide-scale unemployment, instead thinking that these people just want to be super moderators and people to be politically correct
  - fallingknife 3y ago
    I have met a lot of people who go around talking about high minded principles an "the greater good" and a lot of people who are transparently self interested. I much preferred the latter. Never believed a word out of the mouths of those busybodies pretending to act in my interest and not theirs. They don't want to limit their own access to the tech. Only yours.
nopinsight 3y ago
Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. This paper explores one such danger and there are other papers which show it's possible to use LLM to aid in designing new toxins and biological weapons. The Operational Risks of AI in Large-Scale Biological Attacks https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html? An example of such an event: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack How do you propose we deal with this sort of harm if more powerful AIs with no limit and control proliferate in the wild? . Note: Both sides of the OpenAI rift care deeply about AI Safety. They just follow different approaches. See more details here: https://news.ycombinator.com/item?id=38376263 https://news.ycombinator.com/item?id=38376263
- kvgr 3y ago
  If somebody wanted to do a biological attack, there is probably not much stopping them even now.
  - nopinsight 3y ago
    The expertise to produce the substance itself is quite rare so it's hard to carry it out unnoticed. AI could make it much easier to develop it in one's basement.
    - swells34 3y ago
      Huh, you'd think all you need are some books on the subject and some fairly generic lab equipment. Not sure what a neural net trained on Internet dumps can add to that? The information has to be in the training data for the AI to be aware of it, correct?
      - nopinsight 3y ago
        GPT-4 is likely trained on some data not publicly available as well. There's also a distinction between trying to follow some broad textbook information and getting detailed feedback from an advanced conversational AI with vision and more knowledge than in a few textbooks/articles in real time.
ribit 3y ago
The scenario you describe is exactly what will happen with unrestricted commercialisation and deregulation of AI. The only way to avoid it is to have strict legal framework and public control.
deleted 3y ago
[deleted]
gorwell 3y ago
“I trust that every animal here appreciates the sacrifice that Comrade Napoleon has made in taking this extra labour upon himself. Do not imagine, comrades, that leadership is a pleasure! On the contrary, it is a deep and heavy responsibility. No one believes more firmly than Comrade Napoleon that all animals are equal. He would be only too happy to let you make your decisions for yourselves. But sometimes you might make the wrong decisions, comrades, and then where should we be?”
layer8 3y ago
This polarizing “certain class of people” and them vs. us narrative isn’t helpful.
concordDance 3y ago
It is utterly mad that there's conflation between "let's make sure AI doesn't kill us all" and "let's make sure AI doesn't say anything that embarrasses corporate". The head of every major AI research group except Metas believes that whenever we finally make AGI it's vital that it shares our goals and values at a deep even-out-of-training-domain level and that failing at this could lead to human extinction. And yet "AI safety" is often bandied about to be "ensure GPT can't tell you anything about IQ distributions".
- deleted 3y ago
  [deleted]
krisoft 3y ago
> Pretty soon AI will be an expert at subtly steering you toward thinking/voting for whatever the "safety" experts want. You are absolutely right. There is no question about that the AI will be an expert at subtly steering individuals and the whole society in whichever direction it does. This is the core concept of safety. If no-one steers the machine then the machine will steer us. You might disagree with the current flavour of steering the current safety experts give it, and that is all right and in fact part of the process. But surely you have your own values. Some things you hold dear to you. Some outcomes you prefer over others. Are you not interested in the ability to make these powerful machines if not support those values, at least not undermine them? If so you are interested in AI safety! You want safe AIs. (Well, alternatively you prefer no AIs, which is in fact a form of safe AI. Maybe the only one we have mastered in some form so far.) > because of X, we need to invade this country. It sounds like you value peace? Me too! Imagine if we could pool together our resources to have an AI which is subtly manipulating society into the direction of more peace. Maybe it would do muckraking investigative journalism exposing the misdeeds of the military-industrial complex? Maybe it would elevate through advertisement peace loving authors and give a counter narrative to the war drums? Maybe it would offer to act as an intermediary in conflict resolution around the world? If we were to do that, "ai safety" and "alignment" is crucial. I don't want to give my money to an entity who then gets subjugated by some intelligence agency to sow more war. That would be against my wishes. I want to know that it is serving me and you in our shared goal of "more peace, less war". Now you might say: "I find the idea of anyone, or anything manipulating me and society disgusting. Everyone should be left to their own devices.". And I agree on that too. But here is the bad news: we are already manipulated. Maybe it doesn't work on you, maybe it doesn't work on me, but it sure as hell works. There are powerful entities financially motivated to keep the wars going. This is a huuuge industry. They might not do it with AIs (for now), because propaganda machines made of meat work currently better. They might change to using AIs when that works better. Or what is more likely employ a hybrid approach. Wishing that nobody gets manipulated is frankly not an option on offer. How does that sound as a passionate argument for AI safety?
lukevp 3y ago
AI isn’t a precondition for partisanship. How do you know Google isn’t showing you biased search results? Or Wikipedia?
simonh 3y ago
A main concern in AI safety is alignment. Ensuring that when you use the AI to try to achieve a goal that it will actually act towards that goal in ways you would want, and not in ways you would not want. So for example if you asked Sydney, the early version of the Bing LLM, some fact it might get it wrong. It was trained to report facts that users would confirm as true. If you challenged it’s accuracy what do you want to happen? Presumably you’d want it to check the fact or consider your challenge. What it actually did was try to manipulate, threaten, browbeat, entice, gaslight, etc, and generally intellectually and emotionally abuse the user into accepting its answer, so that it’s reported ‘accuracy’ rate goes up. That’s what misaligned AI looks like.
- gorbypark 3y ago
  I haven't been following this stuff too closely, but have there been any more findings on what "went wrong" with Sydney initially? Like, I thought it was just a wrapper on GPT (was it 3.5?), but maybe Microsoft took the "raw" GPT weights and did their own alignment? Or why did Sydney seem so creepy sometimes compared to ChatGPT?
  - simonh 3y ago
    I think what happened is Microsoft got the raw GPT3.5 weights, based on the training set. However for ChatGPT OpenAI had done a lot of additional training to create the 'assistant' personality, using a combination of human and model based response evaluation training. Microsoft wanted to catch up quickly so instead of training the LLM itself, they relied on prompt engineering. This involved pre-loading each session with a few dozen rules about it's behaviour as 'secret' prefaces to the user prompt text. We know this because some users managed to get it to tell them the prompt text.
loup-vaillant 3y ago
Note how what you said also apply to the search & recommendation engines that are in widespread use today.
pk-protect-ai 3y ago
I just had a conversation about this like two weeks ago. The current trend in AI "safety" is a form of brainwashing, not only for AI but also for future generations shaping their minds. There are several aspects: 1. Censorship of information 2. Cover-up of the biases and injustices in our society This limits creativity, critical thinking, and the ability to challenge existing paradigms. By controlling the narrative and the data that AI systems are exposed to, we risk creating a generation of both machines and humans that are unable to think outside the box or question the status quo. This could lead to a stagnation of innovation and a lack of progress in addressing the complex issues that face our world. Furthermore, there will be a significant increase in mass manipulation of the public into adopting the way of thinking that the elites desire. It is already done by mass media, and we can actually witness this right now with this case. Imagine a world where youngsters no longer use search engines and rely solely on the information provided by AI. By shaping the information landscape, those in power will influence public opinion and decision-making on an even larger scale, leading to a homogenized culture where dissenting voices are silenced. This not only undermines the foundations of a diverse and dynamic society but also poses a threat to democracy and individual freedoms. Guess what? I just have checked above text for the biases against GPT-4 Turbo, and it appears to be I'm a moron: 1. *Confirmation Bias*: The text assumes that AI safety measures are inherently negative and equates them with brainwashing, which may reflect the author's preconceived beliefs about AI safety without considering potential benefits. 2. *Selection Bias*: The text focuses on negative aspects of AI safety, such as censorship and cover-up, without acknowledging any positive aspects or efforts to mitigate these issues. 3. *Alarmist Bias*: The language used is somewhat alarmist, suggesting a dire future without presenting a balanced view that includes potential safeguards or alternative outcomes. 4. *Conspiracy Theory Bias*: The text implies that there is a deliberate effort by "elites" to manipulate the masses, which is a common theme in conspiracy theories. 5. *Technological Determinism*: The text suggests that technology (AI in this case) will determine social and cultural outcomes without considering the role of human agency and decision-making in shaping technology. 6. *Elitism Bias*: The text assumes that a group of "elites" has the power to control public opinion and decision-making, which may oversimplify the complex dynamics of power and influence in society. 7. *Cultural Pessimism*: The text presents a pessimistic view of the future culture, suggesting that it will become homogenized and that dissent will be silenced, without considering the resilience of cultural diversity and the potential for resistance. Huh, just look at what's happening in North Korea, Russia, Iran, China, and actually in any totalitarian country. Unfortunately, the same thing happens worldwide, but in democratic countries, it is just subtle brainwashing with a "humane" facade. No individual or minority group can withstand the power of the state and a mass-manipulated public. Bonhoeffer's theory of stupidity: https://www.youtube.com/watch?v=ww47bR86wSc&pp=ygUTdGhlb3J5IG9mIHN0dXBpZGl0eQ%3D%3D https://www.youtube.com/watch?v=ww47bR86wSc&pp=ygUTdGhlb3J5I...
lordnacho 3y ago
Great comment. In a way AI is no different from old school intelligence, aka experts. "We need to have oversight over what the scientists are researching, so that it's always to the public benefit" "How do we really know if the academics/engineers/doctors have everyone's interest in mind?" That kind of thing has been a thought since forever, and politicians of all sorts have had to contend with it.
jack_riminton 3y ago
Exactly, society's Prefects rarely have the technical chops to do any of these things so they worm their way up the ranks of influence by networking. Once they're in position they can control by spreading fear and doing the things "for your own good"
deanCommie 3y ago
> I'm convinced there is a certain class of people who gravitate to positions of power, like "moderators", (partisan) journalists, And there is also a class of people that resist all moderation on principle even when it's ultimately for their benefit. See, Americans whenever the FDA brings up any questions of health: * "Gas Stoves may increase Asthma." -> "Don't you tread on me, you can take my gas stove from my cold dead hands!" Of course it's ridiculous - we've been through this before with Asbestos, Lead Paint, Seatbelts, even the very idea of the EPA cleaning up the environment. It's not a uniquely American problem, but America tends to attract and offer success to the folks that want to ignore these on principles. For every Asbestos there is a Plastic Straw Ban which is essentially virtue signalling by the types of folks you mention - meaningless in the grand scheme of things for the stated goal, massive in terms of inconvenience. But the existence of Plastic Straw Ban does not make Asbestos, CFCs, or Lead Paint any safer. Likewise, the existence of people that gravitate to positions of power and middle management does not negate the need for actual moderation in dozens of societal scenarios. Online forums, Social Networks, and...well I'm not sure about AI. Because I'm not sure what AI is, it's changing daily. The point is that I don't think it's fair to assume that anyone that is interested in safety and moderation is doing it out of a misguided attempt to pursue power, and instead is actively trying to protect and improve humanity. Lastly, your portrayal of journalists as power figures is actively dangerous to the free press. This was never stated this directly until the Trump years - even when FOX News was berating Obama daily for meaningless subjects. When the TRUTH becomes a partisan subject, then reporting on that truth becomes a dangerous activity. Journalists are MOSTLY in the pursuit of truth.
alebairos 3y ago
My safety (of my group) is what really matters.
systemvoltage 3y ago
Ah, you don't need to go far. Just go to your local HOA meetings.
miraculixx 3y ago
Yes, it's an outright powergrab. They will stop at nothing. Case in point, the new AI laws like the EU AI act will outlaw *all* software unless registered and approved by some "authority". The result will be concentration of power, wealth for the few, and instability and poverty for everyone else.

cyanydeez 3y ago

All you're really describing is why this shouldn't be a non-proft and should just be a government effort. But I assume, from y our language, you'd also object to making this a government utility.

sethammons 3y ago
> should just be a government effort And the controlling party de jour will totally not tweak it to side with their agenda, I'm sure. </s>
- cyanydeez 3y ago
  uh. We're arguing about _who is controlling AI_. What do you image a neutral party does? If youu're talking about safety, don't you think there should be someone sitting on a boar dsomewhere, contemplating _what should the AI feed today?_ Seriously, why is a non profit, or a business or whatever any different than a government? I get it: there's all kinds of governments, but now theres all kind of businesses. The point of putting it in the governments hand is a defacto acknowledgement that it's a utility. Take other utilities, any time you give a prive org a right to control whether or not you get electricity or water, whats the outcome? Rarely good. If AI is suppose to help society, that's the purview of the government. That's all, you can imagine it's the chinese government, or the russian, or the american or the canadian. They're all _going to do it_, thats _going to happen_, and if a business gets there first, _what is the difference if it's such a powerful device_. I get it, people look dimly on governments, but guess what: they're just as powerful as some organization that gets billions of dollars to effect society. Why is it suddenly a boogeyman?
  - sethammons 3y ago
    I find any government to be more of a boogeyman than any private company because the government has the right to violence and companies come and go at a faster rate.
    - cyanydeez 3y ago
      Ok, and if Raytheon builds an AI and tells a government "trust us, its safe", arn't you just letting them create a scape goat via the government? Seriously, Businesses simply dont have the history that governments do. They're just as capable of violence. https://utopia.org/guide/crime-controversy-nestles-5-biggest-scandals-explained/ https://utopia.org/guide/crime-controversy-nestles-5-biggest... All you're identifying is "government has a longer history of violence than Businesses"
    - kjkjadksj 3y ago
      The municipal utility provider has a right to violence? The park service? Where do you live? Los Angeles during Blade Runner?

Dylan16807 3y ago

Personally, I expect the opposite camp to be just as bad about steering.

ragequittah 3y ago

[dead]

PeterStuer 3y ago

Most of those touting "safety" do not want to limit their access to and control of powerfull AI, just yours .

vkou 3y ago
Meanwhile, those working on commercialization are by definition going to be gatekeepers and beneficiaries of it, not you. The organizations that pay for it will pay for it to produce results that are of benefit to them, probably at my expense [1]. Do I think Helen has my interests at heart? Unlikely. Do Sam or Satya? Absolutely not! [1] I can't wait for AI doctors working for insurers to deny me treatments, AI vendors to figure out exactly how much they can charge me for their dynamically-priced product, AI answering machines to route my customer support calls through Dante's circles of hell...
- konschubert 3y ago
  > produce results that are of benefit to them, probably at my expense The world is not zero-sum. Most economic transactions benefit both parties and are a net benefit to society, even considering externalities.
  - vkou 3y ago
    > The world is not zero-sum. No, but some parts of it very much are. The whole point of AI safety is keeping it away from those parts of the world. How are Sam and Satya going to do that? It's not in Microsoft's DNA to do that.
    - concordDance 3y ago
      > The whole point of AI safety is keeping it away from those parts of the world. No, it's to ensure it doesn't kill you and everyone you love.
      - hef19898 3y ago

nostromo 3y ago

You're correct. When people say they want safe AGI, what they mean are things like "Skynet should not nuke us" and "don't accelerate so fast that humans are instantly irrelevant." But what it's being interpreted as is more like "be excessively prudish and politically correct at all times" -- which I doubt was ever really anyone's main concern with AGI.

wisty 3y ago
There is a middle ground, in that maybe ChatGTP shouldn't help users commit certain serious crimes. I am pretty pro free speech, and I think there's definitely a slippery slope here, but there is a bit of justification.
- StanislavPetrov 3y ago
  Which users? The greatest crimes, by far, are committed by the US government (and other governments around the world) - and you can be sure that AI and/or AGI will be designed to help them commit their crimes more efficiently, effectively and to manufacture consent to do so.
- hef19898 3y ago
  I am a little less free speech than Americans, in Germany we have serious limitations around hate speech and holicaust denial for example. Putting thise restrictions into a tool like ChatGPT goes to far so, because so far AI still needs a prompt to do anything. The problem I see, is with ChatGPT, being trained on a lot hate speech or prpopagabda, slipts in those things even if not prompted to. Which, and I am by no means an AI expert not by far, seems to be a sub-problem of the hallucination problems of making stuff up. Because we have to remind ourselves, AI so far is glorified mavhine learning creating content, it is not concient. But it can be used to create a lot of propaganda and deffamation content at unprecedented scale and speed. And that is the real problem.
  - freedomben 3y ago
    Apologies this is very off topic, but I don't know anyone from Germany that I can ask and you opened the door a tiny bit by mentioning the holocaust :-) I've been trying to really understand the situation and how Hitler was able to rise to power. The horrendous conditions placed on Germany after WWI and the Weimar Republic for example have really enlightened me. Have you read any of the big books on the subject that you could recommend? I'm reading Ian Kershaw's two-part series on Hitler, and William Shirer's "Collapse of the Third Republic" and "Rise and Fall of the Third Reich". Have you read any of those, or do you have books you would recommend?

phreeza 3y ago

If you believe the other side in this rift is not also striving to put themselves in positions of power, I think you are wrong. They are just going to use that power to manipulate the public in a different way. The real alternative are truly open models, not Models controlled by slightly different elite interests.

davedx 3y ago

Wow, what an incredibly bad faith characterization of the OpenAI board? This kind of speculative mud slinging makes this place seem more like a gossip forum.

sho_hn 3y ago
Most of the comments on Hacker News are written by folks who a much easier time & would rather imagine themselves as a CEO, than as a non-profit board member. There is little regard for the latter. As a non-profit board member, I'm curious why their bylaws are so crummy that the rest of the board could simply remove two others on the board. That's not exactly cunning design of your articles of association ... :-)
ssnistfajen 3y ago
This place was never above being a gossip forum, especially on topics that involve any ounce of politics or social sciences.
- 93po 3y ago
  Strong agree. HN is like anywhere else on the internet but with with a bit more dry content (no memes and images etc) so it attracts an older crowd. It does, however, have great gems of comments and people who raise the bar. But it's still amongst a sea of general quick-to-anger and loosely held opinions stated as fact - which I am guilty of myself sometimes. Less so these days.
Rastonbury 3y ago
I have no words for that comment. As if its so unbelievable that someone would want to prevent rogue AI or wide-scale unemployment, instead thinking that these people just want to be super moderators and people to be politically correct
- fallingknife 3y ago
  I have met a lot of people who go around talking about high minded principles an "the greater good" and a lot of people who are transparently self interested. I much preferred the latter. Never believed a word out of the mouths of those busybodies pretending to act in my interest and not theirs. They don't want to limit their own access to the tech. Only yours.

nopinsight 3y ago

Proliferation of more advanced AIs without any control would increase the power of some malicious groups far beyond they currently have. This paper explores one such danger and there are other papers which show it's possible to use LLM to aid in designing new toxins and biological weapons. The Operational Risks of AI in Large-Scale Biological Attacks https://www.rand.org/pubs/research_reports/RRA2977-1.html https://www.rand.org/pubs/research_reports/RRA2977-1.html? An example of such an event: https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack How do you propose we deal with this sort of harm if more powerful AIs with no limit and control proliferate in the wild? . Note: Both sides of the OpenAI rift care deeply about AI Safety. They just follow different approaches. See more details here: https://news.ycombinator.com/item?id=38376263 https://news.ycombinator.com/item?id=38376263

kvgr 3y ago
If somebody wanted to do a biological attack, there is probably not much stopping them even now.
- nopinsight 3y ago
  The expertise to produce the substance itself is quite rare so it's hard to carry it out unnoticed. AI could make it much easier to develop it in one's basement.
  - swells34 3y ago
    Huh, you'd think all you need are some books on the subject and some fairly generic lab equipment. Not sure what a neural net trained on Internet dumps can add to that? The information has to be in the training data for the AI to be aware of it, correct?
    - nopinsight 3y ago
      GPT-4 is likely trained on some data not publicly available as well. There's also a distinction between trying to follow some broad textbook information and getting detailed feedback from an advanced conversational AI with vision and more knowledge than in a few textbooks/articles in real time.

ribit 3y ago

The scenario you describe is exactly what will happen with unrestricted commercialisation and deregulation of AI. The only way to avoid it is to have strict legal framework and public control.

deleted 3y ago

[deleted]

gorwell 3y ago

“I trust that every animal here appreciates the sacrifice that Comrade Napoleon has made in taking this extra labour upon himself. Do not imagine, comrades, that leadership is a pleasure! On the contrary, it is a deep and heavy responsibility. No one believes more firmly than Comrade Napoleon that all animals are equal. He would be only too happy to let you make your decisions for yourselves. But sometimes you might make the wrong decisions, comrades, and then where should we be?”

layer8 3y ago

This polarizing “certain class of people” and them vs. us narrative isn’t helpful.

concordDance 3y ago

It is utterly mad that there's conflation between "let's make sure AI doesn't kill us all" and "let's make sure AI doesn't say anything that embarrasses corporate". The head of every major AI research group except Metas believes that whenever we finally make AGI it's vital that it shares our goals and values at a deep even-out-of-training-domain level and that failing at this could lead to human extinction. And yet "AI safety" is often bandied about to be "ensure GPT can't tell you anything about IQ distributions".

deleted 3y ago
[deleted]

krisoft 3y ago

> Pretty soon AI will be an expert at subtly steering you toward thinking/voting for whatever the "safety" experts want. You are absolutely right. There is no question about that the AI will be an expert at subtly steering individuals and the whole society in whichever direction it does. This is the core concept of safety. If no-one steers the machine then the machine will steer us. You might disagree with the current flavour of steering the current safety experts give it, and that is all right and in fact part of the process. But surely you have your own values. Some things you hold dear to you. Some outcomes you prefer over others. Are you not interested in the ability to make these powerful machines if not support those values, at least not undermine them? If so you are interested in AI safety! You want safe AIs. (Well, alternatively you prefer no AIs, which is in fact a form of safe AI. Maybe the only one we have mastered in some form so far.) > because of X, we need to invade this country. It sounds like you value peace? Me too! Imagine if we could pool together our resources to have an AI which is subtly manipulating society into the direction of more peace. Maybe it would do muckraking investigative journalism exposing the misdeeds of the military-industrial complex? Maybe it would elevate through advertisement peace loving authors and give a counter narrative to the war drums? Maybe it would offer to act as an intermediary in conflict resolution around the world? If we were to do that, "ai safety" and "alignment" is crucial. I don't want to give my money to an entity who then gets subjugated by some intelligence agency to sow more war. That would be against my wishes. I want to know that it is serving me and you in our shared goal of "more peace, less war". Now you might say: "I find the idea of anyone, or anything manipulating me and society disgusting. Everyone should be left to their own devices.". And I agree on that too. But here is the bad news: we are already manipulated. Maybe it doesn't work on you, maybe it doesn't work on me, but it sure as hell works. There are powerful entities financially motivated to keep the wars going. This is a huuuge industry. They might not do it with AIs (for now), because propaganda machines made of meat work currently better. They might change to using AIs when that works better. Or what is more likely employ a hybrid approach. Wishing that nobody gets manipulated is frankly not an option on offer. How does that sound as a passionate argument for AI safety?

lukevp 3y ago

AI isn’t a precondition for partisanship. How do you know Google isn’t showing you biased search results? Or Wikipedia?

simonh 3y ago

A main concern in AI safety is alignment. Ensuring that when you use the AI to try to achieve a goal that it will actually act towards that goal in ways you would want, and not in ways you would not want. So for example if you asked Sydney, the early version of the Bing LLM, some fact it might get it wrong. It was trained to report facts that users would confirm as true. If you challenged it’s accuracy what do you want to happen? Presumably you’d want it to check the fact or consider your challenge. What it actually did was try to manipulate, threaten, browbeat, entice, gaslight, etc, and generally intellectually and emotionally abuse the user into accepting its answer, so that it’s reported ‘accuracy’ rate goes up. That’s what misaligned AI looks like.

gorbypark 3y ago
I haven't been following this stuff too closely, but have there been any more findings on what "went wrong" with Sydney initially? Like, I thought it was just a wrapper on GPT (was it 3.5?), but maybe Microsoft took the "raw" GPT weights and did their own alignment? Or why did Sydney seem so creepy sometimes compared to ChatGPT?
- simonh 3y ago
  I think what happened is Microsoft got the raw GPT3.5 weights, based on the training set. However for ChatGPT OpenAI had done a lot of additional training to create the 'assistant' personality, using a combination of human and model based response evaluation training. Microsoft wanted to catch up quickly so instead of training the LLM itself, they relied on prompt engineering. This involved pre-loading each session with a few dozen rules about it's behaviour as 'secret' prefaces to the user prompt text. We know this because some users managed to get it to tell them the prompt text.

loup-vaillant 3y ago

Note how what you said also apply to the search & recommendation engines that are in widespread use today.

pk-protect-ai 3y ago

I just had a conversation about this like two weeks ago. The current trend in AI "safety" is a form of brainwashing, not only for AI but also for future generations shaping their minds. There are several aspects: 1. Censorship of information 2. Cover-up of the biases and injustices in our society This limits creativity, critical thinking, and the ability to challenge existing paradigms. By controlling the narrative and the data that AI systems are exposed to, we risk creating a generation of both machines and humans that are unable to think outside the box or question the status quo. This could lead to a stagnation of innovation and a lack of progress in addressing the complex issues that face our world. Furthermore, there will be a significant increase in mass manipulation of the public into adopting the way of thinking that the elites desire. It is already done by mass media, and we can actually witness this right now with this case. Imagine a world where youngsters no longer use search engines and rely solely on the information provided by AI. By shaping the information landscape, those in power will influence public opinion and decision-making on an even larger scale, leading to a homogenized culture where dissenting voices are silenced. This not only undermines the foundations of a diverse and dynamic society but also poses a threat to democracy and individual freedoms. Guess what? I just have checked above text for the biases against GPT-4 Turbo, and it appears to be I'm a moron: 1. *Confirmation Bias*: The text assumes that AI safety measures are inherently negative and equates them with brainwashing, which may reflect the author's preconceived beliefs about AI safety without considering potential benefits. 2. *Selection Bias*: The text focuses on negative aspects of AI safety, such as censorship and cover-up, without acknowledging any positive aspects or efforts to mitigate these issues. 3. *Alarmist Bias*: The language used is somewhat alarmist, suggesting a dire future without presenting a balanced view that includes potential safeguards or alternative outcomes. 4. *Conspiracy Theory Bias*: The text implies that there is a deliberate effort by "elites" to manipulate the masses, which is a common theme in conspiracy theories. 5. *Technological Determinism*: The text suggests that technology (AI in this case) will determine social and cultural outcomes without considering the role of human agency and decision-making in shaping technology. 6. *Elitism Bias*: The text assumes that a group of "elites" has the power to control public opinion and decision-making, which may oversimplify the complex dynamics of power and influence in society. 7. *Cultural Pessimism*: The text presents a pessimistic view of the future culture, suggesting that it will become homogenized and that dissent will be silenced, without considering the resilience of cultural diversity and the potential for resistance. Huh, just look at what's happening in North Korea, Russia, Iran, China, and actually in any totalitarian country. Unfortunately, the same thing happens worldwide, but in democratic countries, it is just subtle brainwashing with a "humane" facade. No individual or minority group can withstand the power of the state and a mass-manipulated public. Bonhoeffer's theory of stupidity: https://www.youtube.com/watch?v=ww47bR86wSc&pp=ygUTdGhlb3J5IG9mIHN0dXBpZGl0eQ%3D%3D https://www.youtube.com/watch?v=ww47bR86wSc&pp=ygUTdGhlb3J5I...

lordnacho 3y ago

Great comment. In a way AI is no different from old school intelligence, aka experts. "We need to have oversight over what the scientists are researching, so that it's always to the public benefit" "How do we really know if the academics/engineers/doctors have everyone's interest in mind?" That kind of thing has been a thought since forever, and politicians of all sorts have had to contend with it.

jack_riminton 3y ago

Exactly, society's Prefects rarely have the technical chops to do any of these things so they worm their way up the ranks of influence by networking. Once they're in position they can control by spreading fear and doing the things "for your own good"

deanCommie 3y ago

> I'm convinced there is a certain class of people who gravitate to positions of power, like "moderators", (partisan) journalists, And there is also a class of people that resist all moderation on principle even when it's ultimately for their benefit. See, Americans whenever the FDA brings up any questions of health: * "Gas Stoves may increase Asthma." -> "Don't you tread on me, you can take my gas stove from my cold dead hands!" Of course it's ridiculous - we've been through this before with Asbestos, Lead Paint, Seatbelts, even the very idea of the EPA cleaning up the environment. It's not a uniquely American problem, but America tends to attract and offer success to the folks that want to ignore these on principles. For every Asbestos there is a Plastic Straw Ban which is essentially virtue signalling by the types of folks you mention - meaningless in the grand scheme of things for the stated goal, massive in terms of inconvenience. But the existence of Plastic Straw Ban does not make Asbestos, CFCs, or Lead Paint any safer. Likewise, the existence of people that gravitate to positions of power and middle management does not negate the need for actual moderation in dozens of societal scenarios. Online forums, Social Networks, and...well I'm not sure about AI. Because I'm not sure what AI is, it's changing daily. The point is that I don't think it's fair to assume that anyone that is interested in safety and moderation is doing it out of a misguided attempt to pursue power, and instead is actively trying to protect and improve humanity. Lastly, your portrayal of journalists as power figures is actively dangerous to the free press. This was never stated this directly until the Trump years - even when FOX News was berating Obama daily for meaningless subjects. When the TRUTH becomes a partisan subject, then reporting on that truth becomes a dangerous activity. Journalists are MOSTLY in the pursuit of truth.

alebairos 3y ago

My safety (of my group) is what really matters.

systemvoltage 3y ago

Ah, you don't need to go far. Just go to your local HOA meetings.

miraculixx 3y ago

Yes, it's an outright powergrab. They will stop at nothing. Case in point, the new AI laws like the EU AI act will outlaw *all* software unless registered and approved by some "authority". The result will be concentration of power, wealth for the few, and instability and poverty for everyone else.