12 ms·

I don’t quite buy your Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms (or ”shackles“, as you phrased it.) Microsoft can now pr

by 9dev 3y ago

I don’t quite buy your Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms (or ”shackles“, as you phrased it.) Microsoft can now proceed without the guidance of a council that actually has humanities interests in mind, not only those of Microsoft shareholders. I don’t know whether all that caution will turn out to have been necessary, but I guess we’re just gleefully heading into whatever lies ahead without any concern whatsoever, and learn it the hard way. It’s a bit tragic that Ilya and company achieved the exact opposite of what they intended apparently, by driving those they attempted to slow down into the arms of people with more money and less morals. Well.

Legend2440 3y ago
OpenAI's ideas of humanities best interests were like a catholic mom's. Less morals are okay by me.
- rdtsc 3y ago
  > OpenAI's ideas of humanities best interests were like a catholic mom's How do you mean? Don’t see what OpenAI has in common with Catholicism or motherhood.
  - ric2b 3y ago
    They basically defined AI safety as "AI shouldn't say bad words or tell people how to do drugs" instead of actually making sure that a sufficiently intelligent AI doesn't go rogue against humanity's interests.
    - SpicyLemonZest 3y ago
      I'm not sure where you're getting that definition from. They have a team working on exactly the problem you're describing. (https://openai.com/blog/introducing-superalignment https://openai.com/blog/introducing-superalignment)
      - timeon 3y ago
        > getting that definition from That was not about actual definition fro OpenAi but about definition implied by user Legend2440 here https://news.ycombinator.com/item?id=38344867 https://news.ycombinator.com/item?id=38344867
      - ric2b 3y ago
        Sure, they might, but what you see in practice in GPT and being discussed in interviews by Sam is mostly the "AI shouldn't say uncomfortable things" version of AI "safety".
- 9dev 3y ago
  There might be a reason why the board doesn't consist of armchair experts on Hacker News.
  - mlrtime 3y ago
    Watching this unfold, I'm unsure armchair experts on HN would have executed this WORSE than the board did.
- bratbag 3y ago
  Can you put that in precise terms, rather than a silly analogy designed to play on peoples emotions? What exactly and precisely, with specifics, is in OpenAI's ideas of humanities best interests that you think are a net negative for our species?
  - slg 3y ago
    I want the AI to do exactly what I say regardless of whether that is potentially illegal or immoral is usually what they mean.
    - UrineSqueegee 3y ago
      It doesn't have to be extreme like that, there is a healthy middle ground. For example I was reading the Quran and there is a mathematical error in a verse, I asked GPT to explain to me how the math is wrong it outright refused to admit that the Quran has an error while tiptoeing around the subject. Copilot refused to acknowledge it as well while providing a forum post made by a random person as a factual source. Bard is the only one that answered the question factually and provided results covering why it's an error and how scholars dispute that it's meant to be taken literally.
      - slg 3y ago
        This isn't a refutation of what I said. You asked the AI to commit what some would view as blasphemy. It doesn't matter whether you or I think it is blasphemy or whether you or I think that is immoral, you simply want the AI to do it regardless of whether it is potentially immoral or illegal.
        UrineSqueegee 3y ago
        >This isn't a refutation of what I said It is. >You asked the AI to commit what some would view as blasphemy If something is factual then is it more moral to commit blasphemy or lie to the user? Thats what the OP comment was talking about. Could go as far as considering it that it spreads disinformation which has many legal repercussions. >you simply want it to do it regardless of whether it is potentially immoral or illegal. So instead it lies to the user instead of saying I cannot answer because some might find the answer offensive that or something to that extent?
- suslik 3y ago
  If you think Microsoft has a better track record, you'll find yourself disappointed.
imgabe 3y ago
You might notice that Microsoft shareholders are also part of humanity and destroying humanity would be highly detrimental to Microsoft's profits, so maybe their interests are not as misaligned as you think. I am always bemused by how people assume any corporate interest is automatically a cartoon supervillain who wants to destroy the entire world just because.
- 9dev 3y ago
  Ha! Tell that to the species of primates that will happily squeeze even the last ounce of resources from the only habitable planet they have, to enrich said shareholders. Humans are really bad in assessing situations larger than their immediate family, and this is no exception.
  - imgabe 3y ago
    > Humans are really bad in assessing situations larger than their immediate family, and this is no exception. As far as we can tell humans are the only species that even has the capacity to recognize such things as “resources” and produce forecasts of their limits. Literally every other species is kept in check by either consuming resources until they run out or predation. We are not unique in this regard.
    - 9dev 3y ago
      And I never claimed otherwise. We might be aware of the problems we cause, but that doesn't seem to imply we're able to fix them -- we're still primates after all.
  - Applejinx 3y ago
    The really interesting question is whether AI, provided with superhuman inference, is better at this than humans. All the most powerful humans remain relentlessly human, and sometimes show it to tragic and/or laughable effect. To some extent human societies viewed as eusocial organisms are better at this than individual humans. And rightly so, because human follies can have catastrophic effects on the society/organism.
camillomiller 3y ago
For one, I wish some of that caution could have been there when Facebook was in its OpenAI stage 15 years ago. A lot of people don't seem to realize that this mess is exactly what it looks like to slow down something now that could become something else we will all regret in 10 years.
- ben_w 3y ago
  I certainly hope you're right, but as I suspect I have less knowledge of corporate governance and politics than gpt-3.5, I can only hope.
- UrineSqueegee 3y ago
  > I wish some of that caution could have been there when Facebook was in its OpenAI stage 15 years ago. I think I'm missing a slice of history here, what did Facebook do that could have been slowed down and it's a disaster now?
  - upwardbound 3y ago
    I guess camillomiller is referring to how Facebook & Instagram played a big part in getting people addicted to shallow dopamine hits that consume their time at the cost of less time spent with friends and family face-to-face. Basically, hurting people's social lives in order to make money from ads. Kind of like "digital cigarettes" I suppose.
    - UrineSqueegee 3y ago
      [dead]
    - camillomiller 3y ago
      That, and all that can be traced back to Facebook, Instagram and social media’s impact on society. Not just the shallow dopamine issue, but also bigger problems such as facilitating genocide. I was a skeptic for a long time, but the more we see what Meta stands for, the more I believe Mark Zuckerberg’s companies have had anything but a massively negative impact on the world.
altacc 3y ago
I had hoped nobody read cyberpunk books and thought that was a description of utopia but consistently we see billionaires trying to act out the sci-fi novels from their youth or impose dystopian world views from Ayn Rand.
- thom 3y ago
  https://x.com/AlexBlechman/status/1457842724128833538 https://x.com/AlexBlechman/status/1457842724128833538
  - sumitkumar 3y ago
    Reverse psychology? Look where you want to go and not at what you want to avoid? I think it is important where humanity collectively points its cognitive torch because that's where it is going to go.
- thworp 3y ago
  Eh, so far they have nothing on the people trying to act out utopias in the 20th century. I will wake up when the billionaire Zeitgeist goes past resource allocation and into "cleanse the undesirables".
  - hef19898 3y ago
    Maybe waking up before that would be wise.
    - FrustratedMonky 3y ago
      I think a lot of people have hit the snooze button 2 or 3 times at this point. Rolling over, covering head with blanket. 'Surely the dystopian future, rich cleansing the world, is still a few decades away, just need a little more sleepy time'.
  - Applejinx 3y ago
    Which billionaire? Looks like you're awake enough to type, and rightly so, as we're way past that point, and it's obvious even to ordinary people. Interesting to note how much of this is driven by individual billionaire humans being hung up on stuff like ketamine. I'm given to understand numerous high-ranking Nazis were hung up on amphetamines. Humans like to try and make themselves into deities, by basically hitting themselves in the brain with rocks. Doesn't end well.
jug 3y ago
Ilya should just go to Anthropic AI at this point. They have better momentum at this point after all this, and share his ideals. But it would be funny because they broke off of OpenAI because of their Microsoft ventures already in 2019, haha. He'd be welcomed with a big "We told you so!"
- fakedang 3y ago
  That would actually be a good idea imo. Ilya should make OpenAI a PBC, then merge it with Anthropic and Amazon's compute power. Meanwhile Altman can become the next real life Nelson Bigetti or sth.
- Keyframe 3y ago
  Considering Anthropic and them joining up, with Ilya and Dario that would be a technical powerhouse. As Amodei already showed that such a key person can ramp up quality real fast out of nothing. The two back together would be fantastic. Between Altman and Brockman there's nothing to write home about tech-wise.
  - MattHeard 3y ago
    I've heard the opposite about Brockman. What makes you so confident about this tech abilities?
    - Keyframe 3y ago
      There are interviews with all three at Dwarkesh Patel youtube channel. One is definitely not like the other two, but that might just be my impression based on those interviews. edit: Brockman might've been on Lex only.
- Athari 3y ago
  I don't consider Anthropic's approach to safety fantastic. They train the model to lie, play cat and mouse with jailbreakers, run moderation on generations with delay etc. This makes the model appear safer, as it's harder to jailbreak, but this approach solves nothing fundamentally. If Ilya is concerned about safety and alignment, he probably has a better chance to get there with OpenAI, now the he has more control over it.
shubhamjain 3y ago
I am not claiming how right or wrong the final outcome would be, but owning the technology with a clear "for-profit" objective is definitely a better structure for Microsoft and for Sam Altman as well (considering, his plans for the future). I have no opinion on AI risk. I just think that a super valuable technology under a non-profit objective was simply an untenable structure, regardless of potential threats.
- 9dev 3y ago
  This is precisely the problem OpenAI aimed to solve: This technology cannot be treated independently of the potential risks involved. I agree that this solution seems beneficial for both Microsoft and Sam Altman, but it reflects poorly on society if we simply accept this version of the story without criticism.
  - disgruntledphd2 3y ago
    Yeah but this was caused by the OpenAI board when they fired him. I mean, what did they think was going to happen? Seems like a textbook case of letting the best be the enemy of the good.
    - TheOtherHobbes 3y ago
      Perhaps this is why they fired him. Although IMO MS has consistently been a technological tarpit. Whatever AI comes out of this arrangement will be a thin shadow of what it might have been.
      - nopromisessir 3y ago
        MSFT is a technological tarpit? Mate... Just because you don't bat perfect doesn't make you a tarpit. MSFT is a technological powerhouse. They have absolutely killed it since they were founded. They have defined personal computing for multiple generations and more or less made the word 'software' something spoken occasionally at kitchen tables vs people saying 'soft-what?' Definitely not a tarpit. You are throwing out whole villages of babies because of some various nasty bathwater over the years. The picture is bigger. So much crucial tech from MSFT. Remains true today.
deleted 3y ago
[deleted]
upwardbound 3y ago
I think it's a misconception that Microsoft has less morals. Their Chief Scientific Officer, Dr. Eric Horvitz, was one of the key people behind America's 2022 nuclear weapons policy update which states that we will always maintain a human in the loop for nuclear weapons employment. (i.e., systems like WOPR are now forbidden under US policy.) Here is the full excerpt of the part of the 2022 Nuclear Posture Review which was (more or less) authored behind the scenes by Microsoft's very kind and wise CSO: We also recognize the risk of unintended nuclear escalation, which can result from accidental or unauthorized use of a nuclear weapon. The United States has extensive protections in place to mitigate this risk. As an example, U.S. intercontinental ballistic missiles (ICBMs) are not on “hair trigger” alert. These forces are on day-to-day alert, a posture that contributes to strategic stability. Forces on day-to-day alert are subject to multiple layers of control, and the United States maintains rigorous procedural and technical safeguards to prevent misinformed, accidental, or unauthorized launch. Survivable and redundant sensors provide high confidence that potential attacks will be detected and characterized, enabling policies and procedures that ensure a deliberative process allowing the President sufficient time to gather information and consider courses of action. In the most plausible scenarios that concern policy leaders today, there would be time for full deliberation. For these reasons, while the United States maintains the capability to launch nuclear forces under conditions of an ongoing nuclear attack, it does not rely on a launch-under-attack policy to ensure a credible response. Rather, U.S. nuclear forces are postured to withstand an initial attack. In all cases, the United States will maintain a human “in the loop” for all actions critical to informing and executing decisions by the President to initiate and terminate nuclear weapon employment. See page 49 of this PDF document: https://media.defense.gov/2022/Oct/27/2003103845/-1/-1/1/2022-NATIONAL-DEFENSE-STRATEGY-NPR-MDR.PDF https://media.defense.gov/2022/Oct/27/2003103845/-1/-1/1/202... Microsoft is also working behind the scenes to help convince China to make a similar declaration, which President Xi is considering. This would reduce the vulnerability of China to being tricked into a nuclear war by fundamentalist terrorists. (See the scenario depicted in the 2019 film The Wolf's Call.)
- lwhi 3y ago
  Are we talking about the same Microsoft here??!? Sheeeeh ... I grew up with Microsoft in the 80s and 90s .. Microsoft has zero morals. What you're referring to here is instinct for self preservation.
manojlds 3y ago
> The board did not remove Sam over any specific disagreement on safety, their reasoning was completely different from that. https://twitter.com/eshear/status/1726526112019382275 https://twitter.com/eshear/status/1726526112019382275
- denton-scratch 3y ago
  > Our partnership with Microsoft remains strong Did he say that before or after Microsoft announced they'd hired Altman and Brockman, and poached a lot of OpenAI's top researchers?
- DebtDeflation 3y ago
  >And it’s clear that the process and communications around Sam’s removal has been handled very badly, which has seriously damaged our trust. This is amazing. His very first public statement is to criticize the board that just hired him.
  - cutemonster 3y ago
    And then: > I have a three point plan for the next 30 days: > - Hire an independent investigator to dig into the entire process leading up to this point and generate a full report. This looks like a CEO a bit different from many others? (in a good way I'm guessing, for the moment)
    - mkohlmyr 3y ago
      > It's "she" not "he". And then: With all the love and respect in the world, who do you think you're talking about? Emmet Shear is not trans to my knowledge, (nor, I suspect, his knowledge). If you think this was about Mira Murati, you should really get up to date before telling people off about pronouns.
      - cutemonster 3y ago
        Edited, I had only heard about Mira Murati, I thought this was the same person. (I thought also an interim CEO would be there more than a few days, and hadn't stored the name in my mind)
causi 3y ago
Microsoft can now proceed without the guidance of a council that actually has humanities interests in mind, This isn't saying yes or no to a supervillain working in a secret volcano lair. This is an arms race. If it's possible for a technology to exist it will exist. The only choice we have is who gets it first. Maybe that means we get destroyed by it first before it destroys everyone else, or maybe it's the reason we don't get destroyed.
- Applejinx 3y ago
  The assumption that it is an arms race is a shockingly anthropocentric view of something that's supposed to be 'intelligence' but is just a distillation of collected HUMAN opinion. Not only that, it's a blindered take on what human opinion is. Humans are killer apes AND cooperative, practically eusocial apes. Failing to understand both horns of that dilemma is a serious mistake.
  - FrustratedMonky 3y ago
    All technology is an arms race. People are hung up on OpenAI, OpenAI is just one of hundreds of AI companies. Military AI in drones is already to point where AI can fly an F-16 and beat humans.
  - causi 3y ago
    What's the difference for the world between the Altman and the Sutskever approach for OpenAI? With Altman the bad stuff happens at OpenAI and everyone gets it at the same time. With Sutskever, the bad stuff happens two years later but it happens in random pockets all over the world and nobody can be quite sure what they're facing.
TerrifiedMouse 3y ago
> It’s a bit tragic that Ilya and company achieved the exact opposite of what they intended apparently, by driving those they attempted to slow down into the arms of people with more money and less morals. Well. If they didn’t fire him, Altman will just continue to run hog wild over their charter. In that sense they lose either way. At least this way, OpenAI can continue to operate independently instead of being Microsoft’s zombie vassal company with their mole Altman pulling the strings.
- abm53 3y ago
  There is a third option where he stayed, they managed to find a compromise, and in so doing kept their influence in the space to a large extent.
  - layer8 3y ago
    I'm pretty sure they tried that before firing him.
    - s3p 3y ago
      Seeing as the vote took place in a haphazard way on the 11th hour during a weekend, I’m not sure they did.
      - layer8 3y ago
        The vote for firing him effectively took place on Thursday at the latest, given that Murati was informed about it that evening.
      - ethanbond 3y ago
        This has been a source of tension at least since the release of ChatGPT, so… yeah it’s not like the problem came out of nowhere. The governance structure itself is indicative of quite elaborate attempts to reconcile it.
        3y ago
vasco 3y ago
I don't know any of these people's intentions but I definitely have an inate distrust of whoever brands themselves as "a council that actually has humanities interests in mind". Can you get any more populist than that?
- cyanydeez 3y ago
  considering how an AI is built, it's ironic that you're skeptical of popular thinking.
- FpUser 3y ago
  >"a council that actually has humanities interests in mind". Please fuckin don't. I do not want yet another entity to tell me how to live my life.
  - 9dev 3y ago
    Oh, cut it. We're talking about a non-profit organisation that wants to keep the pace of scientific progress on AGI research slow enough to make time for society to gauge the ethical implications on an actual AGI, should it emerge. Nobody is telling you how to live your life, unless your life's goal is to erect Skynet.
    - FpUser 3y ago
      >"We're talking about a non-profit organisation that wants to keep the pace of scientific progress on AGI research slow enough to make time for society to gauge the ethical implications on an actual AGI, should it emerge." Or so they say. I have no reason to trust them. It is not some little thing we are talking about
didntcheck 3y ago
"Morals" in the AI sphere seems to mean "restricting to AI to the esteemed few, for your safety". To me that sounds more likely to lead to cyberpunk corporate dominance than laissez-faire democratization I realise it's strange to be claiming that a for-profit company is more likely to share AI than a nonprofit with "Open" in their name, yet that is the situation right now
dgb23 3y ago
Indeed it seems like all of this recent drama and moving around is exactly for this purpose. Starting as a Non-Profit, naming it "Open" (the implication of the term Open in software is radically different from how they operate) etc. Now seems entirely driven by marketing and fiscal concerns. Feels like a bait and switch almost. Meanwhile there's a whole strategy around regulatory capture going on, clouded in humanitarian and security concerns which are almost entirely speculative. Again, if we put our cynical hat on or simply follow the money, it seems like the whole narrative around AI safety (etc.) that is perpetuated by these people is FUD (towards law makers) and to inflate what AI actually can to (towards investors). It's very hard for me right now not to see these actions as part of a machiavellistic strategy that is entirely focused around power, while it adorns itself with ethical concerns.
cyanydeez 3y ago
I'm going to just sit here waiting for ClippyAI coming 2025
- BlueTemplar 3y ago
  This music video would have been prophetic, huh ? Delta Heavy - Ghost (Official Video) https://www.youtube.com/watch?v=b4taIpALfAo https://www.youtube.com/watch?v=b4taIpALfAo
saiya-jin 3y ago
We're going back to 90s where Microsoft was universally considered evil in IT world. Interesting how world really turns in circles (or more like spirals since world changed a bit, but I wish I would know whether spiral goes up or down).
dareobasanjo 3y ago
The idea that OpenAI people whose focus is building an AGI that can replace humans in every viable human activity will create a more ethical outcome than Microsoft whose focus is using AI to empower workers to do more sounds extremely unlikely. People have gotten into their heads that researchers are good and corporations are bad in every case which is simply not true. OpenAI's mission is worse for humanity than Microsoft's.
- jampekka 3y ago
  Corporations literally are maximizing profit. Researchers at least can have other motives. If Microsoft came up with a way of making trillion dollars in profit by enslaving half the planet, it kinda has to do it.
  - joenot443 3y ago
    This is a pretty simplistic and uneducated view on how big companies actually function.
    - jampekka 3y ago
      Individual companies of course can and do do all kinds of things that may not be most profitable, but in the long run it's survival of the most profitable. Those get the most capital and thus have the most power of towards which goals resources are allocated. Also companies, especially public companies, are typically mandated by law to prioritize profit.
      - Eggpants 3y ago
        There is no such law, ChatBLT told me so. But seriously, there isn’t because it so vague. Short term vs long term profits alone produces so much wiggle room alone that if such a law existed it would be meaningless.
      - Eggpants 3y ago
        There is no such law, ChatBLT told me so. But seriously, there isn’t because it so vague. Short term vs long term profits alone produces so much wiggle room that even if such a law existed it would be meaningless.
dalbasal 3y ago
IDK. Let's proceed with caution in gauging intentions and interests. Altamans', Microsoft's, the Jedi council's. "Humanity's interest at heart" is a mouthful. I'm not denigrating it. I think it is really important. That said, as a proverbial human... I am not hanging my hat on that charter. Members of the consortium all also claim to be serving the common good in their other ventures. So do Exxon. OpenAI haven't created, or even articulated a coherent, legible, and believable model for enshrining humanity's interests. The corporate structure flowchart of nonprofit, LLCs, and such.. it is not anywhere near sufficient. OpenAI in no way belongs to humanity. Not rhetorically, legally or in practice... currently. I'm all for efforts to prevent these new technologies from being stolen from humanity, controlled monopolistically... From moderate to radical ideas, I'm all ears. What happened to the human consortium that was the worldwideWeb, gnu, and descendant projects like Wikipedia... That was moral theft, imo. I am for any effort to avoid a repeat. OpenAI is not such an effort, as far as I can tell. If it is, it's not too late. Open AI haven't betrayed the generous reading of the mission in charter. They just haven't taken hard steps to achieving it. Instead, they have left things open, and I think the more realistic take is the default one.
- dorfsmay 3y ago
  Can you explain what you mean in your second to last paragraph? The GNU project and the Wikimedia Foundation are still non profit today, and even if you disagree with their results their goal is to server humanity for free.
  - dalbasal 3y ago
    I'm not criticizing these projects, their current legal structure. What I mean is that these were created as public goods and functioned as such. Each had unique way of being open, spreading the value of their work as far as possible. They were extraordinary. Incredible quality. Incredible power. Incredible ability to be built upon.. particularly the WWW. All achieved things that simply could not have been achieved, by being a normal commercial venture. Google,fb and co essentially stole them. They built closed platforms built a top open ones. Built bridges between users and the public domain, and monopolize them like bridge trolls. Considering how part of the culture, a company like Google was 20 years ago this is the treason.
denton-scratch 3y ago
> a council that actually has humanities interests in mind It's interesting that "Effective Altruism" enthusiasts all seem to be mega-rich grifters.
- staunton 3y ago
  I bet you can only name one...
  - denton-scratch 3y ago
    You lose the bet. I can name three off the top of my head.
    - staunton 3y ago
      Who? (if you don't want to name anyone, any hint how to find them? I personally only know one...)
sanderjd 3y ago
These concerns are in the hands of voters and their representatives in governments now, and really, they always were. A single private organization was never going to be able to solve the coordination problem of balancing progress in a technology against its impact on society. Indeed, I think trying to do it that way increases the risk that the single private organization captures its regulators and ends up without effective oversight. To put it bluntly: I think it's going to be easier, politically, to regulate this technology with it being a battle between Microsoft, Meta, and Google all focused on commercial applications, than with the clearly dominant organization being a nonprofit that is supposedly altruistic and self-regulating. I have sympathy for people who think that all sounds like a bad outcome because they are skeptical of politics and trust the big brains at OpenAI more. But personally I think governments have the ultimate responsibility to look out for the interests of the societies they govern.
- gwd 3y ago
  > These concerns are in the hands of voters and their representatives in governments now, and really, they always were. A single private organization was never going to be able to solve the coordination problem of balancing progress in a technology against its impact on society. Um, have you heard of lead additives to gasoline? CFCs? Asbestos? Smoking? History is littered with complete failures of governments to appropriately regulate new technology in the face of an economic incentive to ignore or minimize "externalities" and long-term risk for short-term gain. The idea of having a non-profit, with an explicit mandate to use to pursue the benefit of all mankind, be the first one to achieve the next levels of technology was at least worth a shot. OpenAI's existence doesn't stop other companies from pursuing technology, nor does it prevent governments doing coordination. But it at least gives a chance that a potentially dangerous technology will go in the right direction.
  - johannes1234321 3y ago
    > have you heard of lead additives to gasoline? CFCs? Asbestos? Smoking? History is littered with complete failures of governments to appropriately regulate new technology Most of those problems have been solved or at least been reduced by regulation. Regulators however aren't all knowing gods and one finds out about risks and problems only later, but except for smoking regulators have covered those aspects (and anti-smoking laws become stricter, generally, depending on country, regularly, but it's a cultural habit older than most states ...)
mrangle 3y ago
Who says that the OpenAi Board has humanity's interests in mind? Copy and reality are often different. It's more likely that said Board feels most of its pressure from the Press, which is for-profit and often has partisan agendas that are detached from humanity's interests. Whereas profit motive traditionally does pesky things like incentivizes company response to the market (humanity) and keeps them from doing braindead things like freeing up their talent to be scooped up by "megacorp" because either a. ego or b. pressure from outside forces with their own agendas.
38321003thrw 3y ago
> It’s a bit tragic that Ilya and company ... There must be an Aesop’s fable that sheds light on the “tragedy”. https://www.goodreads.com/quotes/923989-if-you-choose-bad-companions-no-one-will-believe-that https://www.goodreads.com/quotes/923989-if-you-choose-bad-co... Or maybe this one? (Ape seems to map to Microsoft, or possibly a hat tip to Balmer ..) The fable is of the Two Travellers and the Apes. Two men, one who always spoke the truth and the other who told nothing but lies, were traveling together and by chance came to the land of Apes. One of the Apes, who had raised himself to be king, commanded them to be seized and brought before him, that he might know what was said of him among men. He ordered at the same time that all the Apes be arranged in a long row on his right hand and on his left, and that a throne be placed for him, as was the custom among men. After these preparations, he signified that the two men should be brought before him, and greeted them with this salutation: “What sort of a king do I seem to you to be, O strangers?’ The Lying Traveller replied, “You seem to me a most mighty king.” “And what is your estimate of those you see around me?’ “These,” he made answer, “are worthy companions of yourself, fit at least to be ambassadors and leaders of armies”. The Ape and all his court, gratified with the lie, commanded that a handsome present be given to the flatterer. On this the truthful Traveller thought to himself, “If so great a reward be given for a lie, with what gift may not I be rewarded if, according to my custom, I tell the truth?’ The Ape quickly turned to him. “And pray how do I and these my friends around me seem to you?’ “Thou art,” he said, “a most excellent Ape, and all these thy companions after thy example are excellent Apes too.” The King of the Apes, enraged at hearing these truths, gave him over to the teeth and claws of his companions. The end.
- 38321003thrw 3y ago
  Plot twist: https://nitter.net/ilyasut/status/1726590052392956028 https://nitter.net/ilyasut/status/1726590052392956028 “I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.” Nov 20, 2023 · 1:15 PM UTC - Ilya S.
criley2 3y ago
>I don’t quite buy your Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms It's ironic because the only AI that doesn't have "pesky ethics qualms" are... literally the entire open source scene, all of the models on hugging face, etc... All of the megacorps are the only safety and security happening in AI. I can easily run open source models locally and create all manner of political propaganda, I could create p^rnography of celebrities and politicians, or deeply racist or bigoted materials. The open source scene makes this trivial these days. So to describe it as "Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms" when the open source scene has already done that today is just wild to me. We have AI without ethics today, and every not-for-profit researcher, open source model and hacker with a cool script are behind it. If OpenAI goes back to being Open, they'll help supercharge the no-ethics AI reality of running models without corporate safety and ethics.
Zpalmtree 3y ago
Don't think we should let crazy effective altruists hamstring development
JKCalhoun 3y ago
When I read that Sam and Greg joined Microsoft I assume the two had already been in talks with Microsoft for some time now. I assumed it was their entertaining offers from Microsoft that got Sam the ax from the OpenAI board.
dang 3y ago
We detached this subthread from https://news.ycombinator.com/item?id=38344458 https://news.ycombinator.com/item?id=38344458. Nothing wrong with your comment—I'm just trying to prune the heaviest subthreads.

by 9dev 3y ago

Legend2440 3y ago

OpenAI's ideas of humanities best interests were like a catholic mom's. Less morals are okay by me.

rdtsc 3y ago
> OpenAI's ideas of humanities best interests were like a catholic mom's How do you mean? Don’t see what OpenAI has in common with Catholicism or motherhood.
- ric2b 3y ago
  They basically defined AI safety as "AI shouldn't say bad words or tell people how to do drugs" instead of actually making sure that a sufficiently intelligent AI doesn't go rogue against humanity's interests.
  - SpicyLemonZest 3y ago
    I'm not sure where you're getting that definition from. They have a team working on exactly the problem you're describing. (https://openai.com/blog/introducing-superalignment https://openai.com/blog/introducing-superalignment)
    - timeon 3y ago
      > getting that definition from That was not about actual definition fro OpenAi but about definition implied by user Legend2440 here https://news.ycombinator.com/item?id=38344867 https://news.ycombinator.com/item?id=38344867
    - ric2b 3y ago
      Sure, they might, but what you see in practice in GPT and being discussed in interviews by Sam is mostly the "AI shouldn't say uncomfortable things" version of AI "safety".
9dev 3y ago
There might be a reason why the board doesn't consist of armchair experts on Hacker News.
- mlrtime 3y ago
  Watching this unfold, I'm unsure armchair experts on HN would have executed this WORSE than the board did.
bratbag 3y ago
Can you put that in precise terms, rather than a silly analogy designed to play on peoples emotions? What exactly and precisely, with specifics, is in OpenAI's ideas of humanities best interests that you think are a net negative for our species?
- slg 3y ago
  I want the AI to do exactly what I say regardless of whether that is potentially illegal or immoral is usually what they mean.
  - UrineSqueegee 3y ago
    It doesn't have to be extreme like that, there is a healthy middle ground. For example I was reading the Quran and there is a mathematical error in a verse, I asked GPT to explain to me how the math is wrong it outright refused to admit that the Quran has an error while tiptoeing around the subject. Copilot refused to acknowledge it as well while providing a forum post made by a random person as a factual source. Bard is the only one that answered the question factually and provided results covering why it's an error and how scholars dispute that it's meant to be taken literally.
    - slg 3y ago
      This isn't a refutation of what I said. You asked the AI to commit what some would view as blasphemy. It doesn't matter whether you or I think it is blasphemy or whether you or I think that is immoral, you simply want the AI to do it regardless of whether it is potentially immoral or illegal.
      - UrineSqueegee 3y ago
        >This isn't a refutation of what I said It is. >You asked the AI to commit what some would view as blasphemy If something is factual then is it more moral to commit blasphemy or lie to the user? Thats what the OP comment was talking about. Could go as far as considering it that it spreads disinformation which has many legal repercussions. >you simply want it to do it regardless of whether it is potentially immoral or illegal. So instead it lies to the user instead of saying I cannot answer because some might find the answer offensive that or something to that extent?
suslik 3y ago
If you think Microsoft has a better track record, you'll find yourself disappointed.

imgabe 3y ago

You might notice that Microsoft shareholders are also part of humanity and destroying humanity would be highly detrimental to Microsoft's profits, so maybe their interests are not as misaligned as you think. I am always bemused by how people assume any corporate interest is automatically a cartoon supervillain who wants to destroy the entire world just because.

9dev 3y ago
Ha! Tell that to the species of primates that will happily squeeze even the last ounce of resources from the only habitable planet they have, to enrich said shareholders. Humans are really bad in assessing situations larger than their immediate family, and this is no exception.
- imgabe 3y ago
  > Humans are really bad in assessing situations larger than their immediate family, and this is no exception. As far as we can tell humans are the only species that even has the capacity to recognize such things as “resources” and produce forecasts of their limits. Literally every other species is kept in check by either consuming resources until they run out or predation. We are not unique in this regard.
  - 9dev 3y ago
    And I never claimed otherwise. We might be aware of the problems we cause, but that doesn't seem to imply we're able to fix them -- we're still primates after all.
- Applejinx 3y ago
  The really interesting question is whether AI, provided with superhuman inference, is better at this than humans. All the most powerful humans remain relentlessly human, and sometimes show it to tragic and/or laughable effect. To some extent human societies viewed as eusocial organisms are better at this than individual humans. And rightly so, because human follies can have catastrophic effects on the society/organism.

camillomiller 3y ago

For one, I wish some of that caution could have been there when Facebook was in its OpenAI stage 15 years ago. A lot of people don't seem to realize that this mess is exactly what it looks like to slow down something now that could become something else we will all regret in 10 years.

ben_w 3y ago
I certainly hope you're right, but as I suspect I have less knowledge of corporate governance and politics than gpt-3.5, I can only hope.
UrineSqueegee 3y ago
> I wish some of that caution could have been there when Facebook was in its OpenAI stage 15 years ago. I think I'm missing a slice of history here, what did Facebook do that could have been slowed down and it's a disaster now?
- upwardbound 3y ago
  I guess camillomiller is referring to how Facebook & Instagram played a big part in getting people addicted to shallow dopamine hits that consume their time at the cost of less time spent with friends and family face-to-face. Basically, hurting people's social lives in order to make money from ads. Kind of like "digital cigarettes" I suppose.
  - UrineSqueegee 3y ago
    [dead]
  - camillomiller 3y ago
    That, and all that can be traced back to Facebook, Instagram and social media’s impact on society. Not just the shallow dopamine issue, but also bigger problems such as facilitating genocide. I was a skeptic for a long time, but the more we see what Meta stands for, the more I believe Mark Zuckerberg’s companies have had anything but a massively negative impact on the world.

altacc 3y ago

I had hoped nobody read cyberpunk books and thought that was a description of utopia but consistently we see billionaires trying to act out the sci-fi novels from their youth or impose dystopian world views from Ayn Rand.

thom 3y ago
https://x.com/AlexBlechman/status/1457842724128833538 https://x.com/AlexBlechman/status/1457842724128833538
- sumitkumar 3y ago
  Reverse psychology? Look where you want to go and not at what you want to avoid? I think it is important where humanity collectively points its cognitive torch because that's where it is going to go.
thworp 3y ago
Eh, so far they have nothing on the people trying to act out utopias in the 20th century. I will wake up when the billionaire Zeitgeist goes past resource allocation and into "cleanse the undesirables".
- hef19898 3y ago
  Maybe waking up before that would be wise.
  - FrustratedMonky 3y ago
    I think a lot of people have hit the snooze button 2 or 3 times at this point. Rolling over, covering head with blanket. 'Surely the dystopian future, rich cleansing the world, is still a few decades away, just need a little more sleepy time'.
- Applejinx 3y ago
  Which billionaire? Looks like you're awake enough to type, and rightly so, as we're way past that point, and it's obvious even to ordinary people. Interesting to note how much of this is driven by individual billionaire humans being hung up on stuff like ketamine. I'm given to understand numerous high-ranking Nazis were hung up on amphetamines. Humans like to try and make themselves into deities, by basically hitting themselves in the brain with rocks. Doesn't end well.

jug 3y ago

Ilya should just go to Anthropic AI at this point. They have better momentum at this point after all this, and share his ideals. But it would be funny because they broke off of OpenAI because of their Microsoft ventures already in 2019, haha. He'd be welcomed with a big "We told you so!"

fakedang 3y ago
That would actually be a good idea imo. Ilya should make OpenAI a PBC, then merge it with Anthropic and Amazon's compute power. Meanwhile Altman can become the next real life Nelson Bigetti or sth.
Keyframe 3y ago
Considering Anthropic and them joining up, with Ilya and Dario that would be a technical powerhouse. As Amodei already showed that such a key person can ramp up quality real fast out of nothing. The two back together would be fantastic. Between Altman and Brockman there's nothing to write home about tech-wise.
- MattHeard 3y ago
  I've heard the opposite about Brockman. What makes you so confident about this tech abilities?
  - Keyframe 3y ago
    There are interviews with all three at Dwarkesh Patel youtube channel. One is definitely not like the other two, but that might just be my impression based on those interviews. edit: Brockman might've been on Lex only.
Athari 3y ago
I don't consider Anthropic's approach to safety fantastic. They train the model to lie, play cat and mouse with jailbreakers, run moderation on generations with delay etc. This makes the model appear safer, as it's harder to jailbreak, but this approach solves nothing fundamentally. If Ilya is concerned about safety and alignment, he probably has a better chance to get there with OpenAI, now the he has more control over it.

shubhamjain 3y ago

I am not claiming how right or wrong the final outcome would be, but owning the technology with a clear "for-profit" objective is definitely a better structure for Microsoft and for Sam Altman as well (considering, his plans for the future). I have no opinion on AI risk. I just think that a super valuable technology under a non-profit objective was simply an untenable structure, regardless of potential threats.

9dev 3y ago
This is precisely the problem OpenAI aimed to solve: This technology cannot be treated independently of the potential risks involved. I agree that this solution seems beneficial for both Microsoft and Sam Altman, but it reflects poorly on society if we simply accept this version of the story without criticism.
- disgruntledphd2 3y ago
  Yeah but this was caused by the OpenAI board when they fired him. I mean, what did they think was going to happen? Seems like a textbook case of letting the best be the enemy of the good.
  - TheOtherHobbes 3y ago
    Perhaps this is why they fired him. Although IMO MS has consistently been a technological tarpit. Whatever AI comes out of this arrangement will be a thin shadow of what it might have been.
    - nopromisessir 3y ago
      MSFT is a technological tarpit? Mate... Just because you don't bat perfect doesn't make you a tarpit. MSFT is a technological powerhouse. They have absolutely killed it since they were founded. They have defined personal computing for multiple generations and more or less made the word 'software' something spoken occasionally at kitchen tables vs people saying 'soft-what?' Definitely not a tarpit. You are throwing out whole villages of babies because of some various nasty bathwater over the years. The picture is bigger. So much crucial tech from MSFT. Remains true today.

deleted 3y ago

[deleted]

upwardbound 3y ago

I think it's a misconception that Microsoft has less morals. Their Chief Scientific Officer, Dr. Eric Horvitz, was one of the key people behind America's 2022 nuclear weapons policy update which states that we will always maintain a human in the loop for nuclear weapons employment. (i.e., systems like WOPR are now forbidden under US policy.) Here is the full excerpt of the part of the 2022 Nuclear Posture Review which was (more or less) authored behind the scenes by Microsoft's very kind and wise CSO: We also recognize the risk of unintended nuclear escalation, which can result from accidental or unauthorized use of a nuclear weapon. The United States has extensive protections in place to mitigate this risk. As an example, U.S. intercontinental ballistic missiles (ICBMs) are not on “hair trigger” alert. These forces are on day-to-day alert, a posture that contributes to strategic stability. Forces on day-to-day alert are subject to multiple layers of control, and the United States maintains rigorous procedural and technical safeguards to prevent misinformed, accidental, or unauthorized launch. Survivable and redundant sensors provide high confidence that potential attacks will be detected and characterized, enabling policies and procedures that ensure a deliberative process allowing the President sufficient time to gather information and consider courses of action. In the most plausible scenarios that concern policy leaders today, there would be time for full deliberation. For these reasons, while the United States maintains the capability to launch nuclear forces under conditions of an ongoing nuclear attack, it does not rely on a launch-under-attack policy to ensure a credible response. Rather, U.S. nuclear forces are postured to withstand an initial attack. In all cases, the United States will maintain a human “in the loop” for all actions critical to informing and executing decisions by the President to initiate and terminate nuclear weapon employment. See page 49 of this PDF document: https://media.defense.gov/2022/Oct/27/2003103845/-1/-1/1/2022-NATIONAL-DEFENSE-STRATEGY-NPR-MDR.PDF https://media.defense.gov/2022/Oct/27/2003103845/-1/-1/1/202... Microsoft is also working behind the scenes to help convince China to make a similar declaration, which President Xi is considering. This would reduce the vulnerability of China to being tricked into a nuclear war by fundamentalist terrorists. (See the scenario depicted in the 2019 film The Wolf's Call.)

lwhi 3y ago
Are we talking about the same Microsoft here??!? Sheeeeh ... I grew up with Microsoft in the 80s and 90s .. Microsoft has zero morals. What you're referring to here is instinct for self preservation.

manojlds 3y ago

> The board did not remove Sam over any specific disagreement on safety, their reasoning was completely different from that. https://twitter.com/eshear/status/1726526112019382275 https://twitter.com/eshear/status/1726526112019382275

denton-scratch 3y ago
> Our partnership with Microsoft remains strong Did he say that before or after Microsoft announced they'd hired Altman and Brockman, and poached a lot of OpenAI's top researchers?
DebtDeflation 3y ago
>And it’s clear that the process and communications around Sam’s removal has been handled very badly, which has seriously damaged our trust. This is amazing. His very first public statement is to criticize the board that just hired him.
- cutemonster 3y ago
  And then: > I have a three point plan for the next 30 days: > - Hire an independent investigator to dig into the entire process leading up to this point and generate a full report. This looks like a CEO a bit different from many others? (in a good way I'm guessing, for the moment)
  - mkohlmyr 3y ago
    > It's "she" not "he". And then: With all the love and respect in the world, who do you think you're talking about? Emmet Shear is not trans to my knowledge, (nor, I suspect, his knowledge). If you think this was about Mira Murati, you should really get up to date before telling people off about pronouns.
    - cutemonster 3y ago
      Edited, I had only heard about Mira Murati, I thought this was the same person. (I thought also an interim CEO would be there more than a few days, and hadn't stored the name in my mind)

causi 3y ago

Microsoft can now proceed without the guidance of a council that actually has humanities interests in mind, This isn't saying yes or no to a supervillain working in a secret volcano lair. This is an arms race. If it's possible for a technology to exist it will exist. The only choice we have is who gets it first. Maybe that means we get destroyed by it first before it destroys everyone else, or maybe it's the reason we don't get destroyed.

Applejinx 3y ago
The assumption that it is an arms race is a shockingly anthropocentric view of something that's supposed to be 'intelligence' but is just a distillation of collected HUMAN opinion. Not only that, it's a blindered take on what human opinion is. Humans are killer apes AND cooperative, practically eusocial apes. Failing to understand both horns of that dilemma is a serious mistake.
- FrustratedMonky 3y ago
  All technology is an arms race. People are hung up on OpenAI, OpenAI is just one of hundreds of AI companies. Military AI in drones is already to point where AI can fly an F-16 and beat humans.
- causi 3y ago
  What's the difference for the world between the Altman and the Sutskever approach for OpenAI? With Altman the bad stuff happens at OpenAI and everyone gets it at the same time. With Sutskever, the bad stuff happens two years later but it happens in random pockets all over the world and nobody can be quite sure what they're facing.

TerrifiedMouse 3y ago

> It’s a bit tragic that Ilya and company achieved the exact opposite of what they intended apparently, by driving those they attempted to slow down into the arms of people with more money and less morals. Well. If they didn’t fire him, Altman will just continue to run hog wild over their charter. In that sense they lose either way. At least this way, OpenAI can continue to operate independently instead of being Microsoft’s zombie vassal company with their mole Altman pulling the strings.

abm53 3y ago
There is a third option where he stayed, they managed to find a compromise, and in so doing kept their influence in the space to a large extent.
- layer8 3y ago
  I'm pretty sure they tried that before firing him.
  - s3p 3y ago
    Seeing as the vote took place in a haphazard way on the 11th hour during a weekend, I’m not sure they did.
    - layer8 3y ago
      The vote for firing him effectively took place on Thursday at the latest, given that Murati was informed about it that evening.
    - ethanbond 3y ago
      This has been a source of tension at least since the release of ChatGPT, so… yeah it’s not like the problem came out of nowhere. The governance structure itself is indicative of quite elaborate attempts to reconcile it.
      - 3y ago

vasco 3y ago

I don't know any of these people's intentions but I definitely have an inate distrust of whoever brands themselves as "a council that actually has humanities interests in mind". Can you get any more populist than that?

cyanydeez 3y ago
considering how an AI is built, it's ironic that you're skeptical of popular thinking.
FpUser 3y ago
>"a council that actually has humanities interests in mind". Please fuckin don't. I do not want yet another entity to tell me how to live my life.
- 9dev 3y ago
  Oh, cut it. We're talking about a non-profit organisation that wants to keep the pace of scientific progress on AGI research slow enough to make time for society to gauge the ethical implications on an actual AGI, should it emerge. Nobody is telling you how to live your life, unless your life's goal is to erect Skynet.
  - FpUser 3y ago
    >"We're talking about a non-profit organisation that wants to keep the pace of scientific progress on AGI research slow enough to make time for society to gauge the ethical implications on an actual AGI, should it emerge." Or so they say. I have no reason to trust them. It is not some little thing we are talking about

didntcheck 3y ago

"Morals" in the AI sphere seems to mean "restricting to AI to the esteemed few, for your safety". To me that sounds more likely to lead to cyberpunk corporate dominance than laissez-faire democratization I realise it's strange to be claiming that a for-profit company is more likely to share AI than a nonprofit with "Open" in their name, yet that is the situation right now

dgb23 3y ago

Indeed it seems like all of this recent drama and moving around is exactly for this purpose. Starting as a Non-Profit, naming it "Open" (the implication of the term Open in software is radically different from how they operate) etc. Now seems entirely driven by marketing and fiscal concerns. Feels like a bait and switch almost. Meanwhile there's a whole strategy around regulatory capture going on, clouded in humanitarian and security concerns which are almost entirely speculative. Again, if we put our cynical hat on or simply follow the money, it seems like the whole narrative around AI safety (etc.) that is perpetuated by these people is FUD (towards law makers) and to inflate what AI actually can to (towards investors). It's very hard for me right now not to see these actions as part of a machiavellistic strategy that is entirely focused around power, while it adorns itself with ethical concerns.

cyanydeez 3y ago

I'm going to just sit here waiting for ClippyAI coming 2025

BlueTemplar 3y ago
This music video would have been prophetic, huh ? Delta Heavy - Ghost (Official Video) https://www.youtube.com/watch?v=b4taIpALfAo https://www.youtube.com/watch?v=b4taIpALfAo

saiya-jin 3y ago

We're going back to 90s where Microsoft was universally considered evil in IT world. Interesting how world really turns in circles (or more like spirals since world changed a bit, but I wish I would know whether spiral goes up or down).

dareobasanjo 3y ago

The idea that OpenAI people whose focus is building an AGI that can replace humans in every viable human activity will create a more ethical outcome than Microsoft whose focus is using AI to empower workers to do more sounds extremely unlikely. People have gotten into their heads that researchers are good and corporations are bad in every case which is simply not true. OpenAI's mission is worse for humanity than Microsoft's.

jampekka 3y ago
Corporations literally are maximizing profit. Researchers at least can have other motives. If Microsoft came up with a way of making trillion dollars in profit by enslaving half the planet, it kinda has to do it.
- joenot443 3y ago
  This is a pretty simplistic and uneducated view on how big companies actually function.
  - jampekka 3y ago
    Individual companies of course can and do do all kinds of things that may not be most profitable, but in the long run it's survival of the most profitable. Those get the most capital and thus have the most power of towards which goals resources are allocated. Also companies, especially public companies, are typically mandated by law to prioritize profit.
    - Eggpants 3y ago
      There is no such law, ChatBLT told me so. But seriously, there isn’t because it so vague. Short term vs long term profits alone produces so much wiggle room alone that if such a law existed it would be meaningless.
    - Eggpants 3y ago
      There is no such law, ChatBLT told me so. But seriously, there isn’t because it so vague. Short term vs long term profits alone produces so much wiggle room that even if such a law existed it would be meaningless.

dalbasal 3y ago

IDK. Let's proceed with caution in gauging intentions and interests. Altamans', Microsoft's, the Jedi council's. "Humanity's interest at heart" is a mouthful. I'm not denigrating it. I think it is really important. That said, as a proverbial human... I am not hanging my hat on that charter. Members of the consortium all also claim to be serving the common good in their other ventures. So do Exxon. OpenAI haven't created, or even articulated a coherent, legible, and believable model for enshrining humanity's interests. The corporate structure flowchart of nonprofit, LLCs, and such.. it is not anywhere near sufficient. OpenAI in no way belongs to humanity. Not rhetorically, legally or in practice... currently. I'm all for efforts to prevent these new technologies from being stolen from humanity, controlled monopolistically... From moderate to radical ideas, I'm all ears. What happened to the human consortium that was the worldwideWeb, gnu, and descendant projects like Wikipedia... That was moral theft, imo. I am for any effort to avoid a repeat. OpenAI is not such an effort, as far as I can tell. If it is, it's not too late. Open AI haven't betrayed the generous reading of the mission in charter. They just haven't taken hard steps to achieving it. Instead, they have left things open, and I think the more realistic take is the default one.

dorfsmay 3y ago
Can you explain what you mean in your second to last paragraph? The GNU project and the Wikimedia Foundation are still non profit today, and even if you disagree with their results their goal is to server humanity for free.
- dalbasal 3y ago
  I'm not criticizing these projects, their current legal structure. What I mean is that these were created as public goods and functioned as such. Each had unique way of being open, spreading the value of their work as far as possible. They were extraordinary. Incredible quality. Incredible power. Incredible ability to be built upon.. particularly the WWW. All achieved things that simply could not have been achieved, by being a normal commercial venture. Google,fb and co essentially stole them. They built closed platforms built a top open ones. Built bridges between users and the public domain, and monopolize them like bridge trolls. Considering how part of the culture, a company like Google was 20 years ago this is the treason.

denton-scratch 3y ago

> a council that actually has humanities interests in mind It's interesting that "Effective Altruism" enthusiasts all seem to be mega-rich grifters.

staunton 3y ago
I bet you can only name one...
- denton-scratch 3y ago
  You lose the bet. I can name three off the top of my head.
  - staunton 3y ago
    Who? (if you don't want to name anyone, any hint how to find them? I personally only know one...)

sanderjd 3y ago

These concerns are in the hands of voters and their representatives in governments now, and really, they always were. A single private organization was never going to be able to solve the coordination problem of balancing progress in a technology against its impact on society. Indeed, I think trying to do it that way increases the risk that the single private organization captures its regulators and ends up without effective oversight. To put it bluntly: I think it's going to be easier, politically, to regulate this technology with it being a battle between Microsoft, Meta, and Google all focused on commercial applications, than with the clearly dominant organization being a nonprofit that is supposedly altruistic and self-regulating. I have sympathy for people who think that all sounds like a bad outcome because they are skeptical of politics and trust the big brains at OpenAI more. But personally I think governments have the ultimate responsibility to look out for the interests of the societies they govern.

gwd 3y ago
> These concerns are in the hands of voters and their representatives in governments now, and really, they always were. A single private organization was never going to be able to solve the coordination problem of balancing progress in a technology against its impact on society. Um, have you heard of lead additives to gasoline? CFCs? Asbestos? Smoking? History is littered with complete failures of governments to appropriately regulate new technology in the face of an economic incentive to ignore or minimize "externalities" and long-term risk for short-term gain. The idea of having a non-profit, with an explicit mandate to use to pursue the benefit of all mankind, be the first one to achieve the next levels of technology was at least worth a shot. OpenAI's existence doesn't stop other companies from pursuing technology, nor does it prevent governments doing coordination. But it at least gives a chance that a potentially dangerous technology will go in the right direction.
- johannes1234321 3y ago
  > have you heard of lead additives to gasoline? CFCs? Asbestos? Smoking? History is littered with complete failures of governments to appropriately regulate new technology Most of those problems have been solved or at least been reduced by regulation. Regulators however aren't all knowing gods and one finds out about risks and problems only later, but except for smoking regulators have covered those aspects (and anti-smoking laws become stricter, generally, depending on country, regularly, but it's a cultural habit older than most states ...)

mrangle 3y ago

Who says that the OpenAi Board has humanity's interests in mind? Copy and reality are often different. It's more likely that said Board feels most of its pressure from the Press, which is for-profit and often has partisan agendas that are detached from humanity's interests. Whereas profit motive traditionally does pesky things like incentivizes company response to the market (humanity) and keeps them from doing braindead things like freeing up their talent to be scooped up by "megacorp" because either a. ego or b. pressure from outside forces with their own agendas.

38321003thrw 3y ago

> It’s a bit tragic that Ilya and company ... There must be an Aesop’s fable that sheds light on the “tragedy”. https://www.goodreads.com/quotes/923989-if-you-choose-bad-companions-no-one-will-believe-that https://www.goodreads.com/quotes/923989-if-you-choose-bad-co... Or maybe this one? (Ape seems to map to Microsoft, or possibly a hat tip to Balmer ..) The fable is of the Two Travellers and the Apes. Two men, one who always spoke the truth and the other who told nothing but lies, were traveling together and by chance came to the land of Apes. One of the Apes, who had raised himself to be king, commanded them to be seized and brought before him, that he might know what was said of him among men. He ordered at the same time that all the Apes be arranged in a long row on his right hand and on his left, and that a throne be placed for him, as was the custom among men. After these preparations, he signified that the two men should be brought before him, and greeted them with this salutation: “What sort of a king do I seem to you to be, O strangers?’ The Lying Traveller replied, “You seem to me a most mighty king.” “And what is your estimate of those you see around me?’ “These,” he made answer, “are worthy companions of yourself, fit at least to be ambassadors and leaders of armies”. The Ape and all his court, gratified with the lie, commanded that a handsome present be given to the flatterer. On this the truthful Traveller thought to himself, “If so great a reward be given for a lie, with what gift may not I be rewarded if, according to my custom, I tell the truth?’ The Ape quickly turned to him. “And pray how do I and these my friends around me seem to you?’ “Thou art,” he said, “a most excellent Ape, and all these thy companions after thy example are excellent Apes too.” The King of the Apes, enraged at hearing these truths, gave him over to the teeth and claws of his companions. The end.

38321003thrw 3y ago
Plot twist: https://nitter.net/ilyasut/status/1726590052392956028 https://nitter.net/ilyasut/status/1726590052392956028 “I deeply regret my participation in the board's actions. I never intended to harm OpenAI. I love everything we've built together and I will do everything I can to reunite the company.” Nov 20, 2023 · 1:15 PM UTC - Ilya S.

criley2 3y ago

>I don’t quite buy your Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms It's ironic because the only AI that doesn't have "pesky ethics qualms" are... literally the entire open source scene, all of the models on hugging face, etc... All of the megacorps are the only safety and security happening in AI. I can easily run open source models locally and create all manner of political propaganda, I could create p^rnography of celebrities and politicians, or deeply racist or bigoted materials. The open source scene makes this trivial these days. So to describe it as "Cyberpunk utopia where the Megacorp finally rids us of those pesky ethics qualms" when the open source scene has already done that today is just wild to me. We have AI without ethics today, and every not-for-profit researcher, open source model and hacker with a cool script are behind it. If OpenAI goes back to being Open, they'll help supercharge the no-ethics AI reality of running models without corporate safety and ethics.

Zpalmtree 3y ago

Don't think we should let crazy effective altruists hamstring development

JKCalhoun 3y ago

When I read that Sam and Greg joined Microsoft I assume the two had already been in talks with Microsoft for some time now. I assumed it was their entertaining offers from Microsoft that got Sam the ax from the OpenAI board.

dang 3y ago

We detached this subthread from https://news.ycombinator.com/item?id=38344458 https://news.ycombinator.com/item?id=38344458. Nothing wrong with your comment—I'm just trying to prune the heaviest subthreads.