14 ms·

Helen Toner shares her side

150 points by Leary 3y ago

purrgers 3y ago
https://archive.is/gAkKC https://archive.is/gAkKC
isubkhankulov 3y ago
It’s interesting that she was threatened with being held responsible for violating a fiduciary duty for the for-profit entity while she was sitting on the non-profit board. Going fwd, I wonder why they cannot convert the structure to a traditional c-corp. Supposedly: “tax issues”. Whatever problems OpenAI dealt with last week, the current non-profit structure will continue to cause future problems IMO.
- mrkramer 3y ago
  Microsoft should acquire them and cut the non-profit crap.
  - happytiger 3y ago
    Isn’t that basically what is happening with a few extra steps?
    - mrkramer 3y ago
      Yes....it's actually Microsoft vs Google AI arms race, that's why Google went all in with their LLM technology.
      - mistrial9 3y ago
        > cut the non-profit crap Great learning opportunity for those that have not taken time to learn and understand non-profit structures in the USA. this is clearly an opportunity for the uninformed to study the US non-profit system, unique in the world. For example, I recall a delegation from Japan touring the HQ of a non-profit educational group in San Francisco. Like the p-poster here, they apparently had no clue why or how capable people could come together and form stable work culture aside from "hierarchical, profit motive in USD$" (.. and people did actual work, unlike many management level stock manipulators).
        mrkramer 3y ago
        I'm from southeast Europe and I graduated from IB high school programme[0], it is an international Swiss based educational programme that prepares you to study abroad and it is a non-profit. As a non-profit they are efficient af at what they do but in the world of technological innovations, incentives are so much higher and so much more impactful when an organization is for-profit. [0] https://en.wikipedia.org/wiki/International_Baccalaureate https://en.wikipedia.org/wiki/International_Baccalaureate
        mistrial9 3y ago
        Mark Surman writes: As I wrote in Fast Company earlier this week, we risk missing the real lesson from the OpenAI governance debacle. ... "In the aftermath of the OpenAI governance battle, the narrative has been dominated by the idea that “the capitalists won.” While this conclusion is catchy, it misses the point. The real (and bigger) story is: The founders of OpenAI set out to create a nonprofit public institution to ensure that AI’s evolution aligns with the interests of humanity at large. Last month, it became clear that they failed. What is also clear: Public institutions that prioritize the public good over short-term profits are desperately needed for the AI era."
- Havoc 3y ago
  > fiduciary duty for the for-profit entity I read it as both. If OpenAI collapses operationally then the non profit fails at its mission too. That’s the board she is on and that’s the fiduciary duty relevant (She may also be on the for profit board idk)
  - stcredzero 3y ago
    In the current legal framework, isn't the reality, fiduciary uber alles?
  - webmaven 3y ago
    Wait, does the for-profit even have a board?
    - Havoc 3y ago
      Had a quick look at structure and I'd say probably not
  - 0xDEAFBEAD 3y ago
    >If OpenAI collapses operationally then the non profit fails at its mission too. Not necessarily true. "We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome [...] if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project." https://openai.com/charter https://openai.com/charter
- dragonwriter 3y ago
  > Going fwd, I wonder why they cannot convert the structure to a traditional c-corp. Supposedly: “tax issues”. Because, IIRC, when a charity nonprofit like OpenAI, Inc.-- a 501(c)(3), other nonprofits are different -- converts to a for-profit (C-corp or otherwise doesn't matter), all the assets acquired while it was a charity must either remain dedicated to and used exclusively for the charitable purpose, be returned, or be donated to charity (or sold, and then the proceeds treated the same way.) For the charity OpenAI, Inc., this would include its interest in OpenAI, LLC (the wholly owned subsidiary through which it exercises control of OpenAI, LP and OpenAI Global, LLC -- the latter being the for profit entity that actually sells product, etc.), and OpenAI, LP (the holding company which in turn is majority owner of OpenAI Global, LLC).
  - gnulinux 3y ago
    Just curious: can they legally sell all their assets to themselves (i.e. OpenAI nonprofit sells it to for-profit OpenAI Inc. for market price real $$$) and then use that $$$ for charitable purposes. Would this satisfy IRS by letting them retain the intellectual property rights?
    - yieldcrv 3y ago
      self dealing regulations are to prevent that. also OpenAI Inc. for profit doesnt exist, a wholly owned LLC subsidiary does. so they would need to divest of that LLC first (by selling the shares) the way around the self dealing regulations are to reduce their ownership below 20% - 35% (yes, its that range, for multiple reasons) good thing here is that most venture backed companies subvert this already by simply share dilution and youre breaking into one the best secrets of Silicon Valley. Non-profits pump the value of startups and launder the reputation of the founders because their ideas get so much “validation” so quickly. (non profits they partially control act as lead investors for startups they control. they both sell shares at higher price, ideally: founders get QSBS tax free treatment, non profit retains tax free capital gains too. nothing wrong with that, you just probably cant do it too) so open ai for profit llc needs to get rid of the PPU’s and bring in outside investors to dilute ownership enough, but two birds one stone: the outside investors have to get a higher valuation
- WillPostForFood 3y ago
  It’s interesting that she was threatened with being held responsible for violating a fiduciary duty for the for-profit entity while she was sitting on the non-profit board. Fiduciary doesn't mean financial or monetary duty, it means holding something in trust. Violating a fiduciary duty could be financial, social, reputational, or in this case, literally destroying the operation of the company you were entrusted with.
- jacquesm 3y ago
  Tax issues and regulatory issues both.
- JKCalhoun 3y ago
  > It’s interesting that she was threatened with being held responsible for violating a fiduciary duty Yeah, that smells like a bullshit threat. Anyone would do well to ignore it.
nerdo 3y ago
> Toner maintains that safety wasn’t the reason the board wanted to fire Altman. Rather, it was a lack of trust > Toner declined to provide specific details on why she and the three others voted to fire Altman from OpenAI Does she see herself as more trustworthy? She can't even be bothered to give an excuse for the firing. Someone like this couldn't be trusted in literally any function, how did she get a board seat on OpenAI?
- deleted 3y ago
  [deleted]
- gnulinux 3y ago
  > how did she get a board seat on OpenAI? She's the leading academic on the issue of AI safety. It's really ridiculous people don't even know her name and say random things about her, not realizing she's a rock star in her field. Her leaving the board is a tremendous loss for OpenAI, about as terrible as Ilya leaving the board. These are two giants of AI currently.
  - edgyquant 3y ago
    > the leading academic on the issue of AI safety. Sounds like a made up position to me. Just another bureaucrat with a cool sounding name
    - reducesuffering 3y ago
      Why don't you reserve your ignorant judgement until you do due diligence? https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en
      - PeterisP 3y ago
        As an academic from a different field, this is not a publication record that would be anywhere close to someone being "the leading academic on the issue of AI safety". It indicates that she has had been one of very many participants at two impactful papers (with zero indication of what her individual contribution was, with so many authors it might as well be being part of the funding organization), has done a few collaborations with strong researchers, and that her personal first-author or sole-author research is barely relevant. Like, it's not a bad publication record, especially for a junior researcher (heck, her first participation in other people's papers is just in 2018 and her own work starts in 2021-2022, so she's effectively just getting started and may have many opportunities to prove herself in the future), it does indicate doing a couple years of research and might qualify for an okayish faculty job when (if?) she gets her PhD, but it's not also something indicating a senior researcher that's the pillar of anything and has had some impact - a grad student who's lucky to study in a strong leading researcher's lab and does some work on two of the advisor's papers might have a similar publication record already at their graduation before their career has even seriously started. Many of her co-authors have significantly larger research impact and personal research (https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ or https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ for example or https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ) - which also raises a question of how much each co-author did in a two-author paper where the leading author is a strong researcher and the second author is a "Director of Strategy and Foundational Research Grants" i.e. getting money for that research, so labeling her as a leading researcher (much less the leading researcher) of some field seems misleading as the publication record would indicate that she's not even on the same level as they are.
chewmieser 3y ago
She specifically declines in the article to provide the additional context that would probably clarify everything. This is just more non-answers and only a tad more information than we had before (threats of violating a fiduciary duty). Nobody involved is being transparent about what went down here. Not sure we'll find out the full story here...
- tfehring 3y ago
  It's frustrating, but she's exposed to potentially massive personal liability for the board's actions. I'm surprised that she agreed to an interview at all - frankly, I probably wouldn't if I were in her position - and I'm sure her answers were carefully scripted and run by her attorney(s). I think this Reddit comment captures the most plausible story at this point: https://www.reddit.com/r/MachineLearning/comments/1812w04/comment/kabk73s/ https://www.reddit.com/r/MachineLearning/comments/1812w04/co...
  - chewmieser 3y ago
    That’s great, thanks for the link there. Seems plausible enough to fill in the blanks.
  - alvah 3y ago
    She wouldn't be the first high-profile member of the EA cult to talk to the press after a scandal, against the advice of any sane lawyer.
- calf 3y ago
  Article does give context as to what "inconsistently candid" meant: Altman allegedly went to each board member to say some negative things about Toner, but when board members got together to verify consistency they found out Altman said a false or misleading thing to at least one board member about Toner. That's not a good look on Altman at all, it looks like Altman was using divide-and-conquer in a skeezy way, spreading around negative impression, even rumors, about his bosses, as a means dispose of a problem.
  - 3y ago
Leary 3y ago
Key passage: "Altman approached other board members, trying to convince each to fire Toner. Later, some board members swapped notes on their individual discussions with Altman. The group concluded that in one discussion with a board member, Altman left a misleading perception that another member thought Toner should leave, the people said. By this point, several of OpenAI’s then-directors already had concerns about Altman’s honesty, people familiar with their thinking said."
- TeeWEE 3y ago
  Wow, there is no way around it. Sam Altman is basically getting his way by misleading people. The board did the right thing. Honesty is crucial as a leader. I think for Sam Altman his goal is more important than ethics and he will do whatever to get his way.
  - jfengel 3y ago
    Honesty is great, in moderation, but it's not the be-all and end-all of leadership. Leadership is fundamentally about manipulating people. If the people already agreed, they wouldn't need leadership. Your job as leader is to get them all working in more or less the same direction, even though they don't really want to. Honesty is a great way to do that, when it works. Honesty has the virtue of not having to keep track of your lies, and you don't get caught in an inconsistency. But it doesn't always work, and then you have to consider alternatives. Sometimes people are dumb. Sometimes people are wrong. Sometimes people are hostile. Sometimes people are right, but in a way that can't actually be implemented, and you need to go with a sub-optimal path despite that. At core, you can't lead if you don't believe that you've listened to all of the concerns and reached the solution that is best for the group as a whole, according to whatever definition of "best" and "whole" you were hired for. Sometimes that's going to force you to use other means that just telling people the truth and hoping they reach the same conclusion. Leadership is fundamentally a people problem, not a technical one. And sometimes that's going to mean being dishonest. A good leader does it as little as possible, but simply dismissing it as a tactic is a recipe for failure because it hides a fundamental truth about why and how people behave in groups.
YetAnotherNick 3y ago
> Toner maintains that safety wasn’t the reason the board wanted to fire Altman. This is the biggest news of it all So all the Q* thing and Ilya's safety position and other theories were incorrect?
gregatragenet3 3y ago
Anyone else come to the realization that when spokespeople talk about "AI Safety" they aren't concerned with the skynet-esque enslavement of mankind or paperclip maximizing, but that controls be in place that prevent people from using the technology in a way that is misaligned with the maximum extraction of profit?
- wait_a_minute 3y ago
  We need open source models that are trained on the same data and that can be built from source and run on lightweight machines like mobile devices.
  - timschmidt 3y ago
    This seems relevant: https://arxiv.org/abs/2311.08105 https://arxiv.org/abs/2311.08105
- justrealist 3y ago
  Please stop framing your own personal cynicism as a grand narrative revalation.
- reducesuffering 3y ago
  Your "realization" is counter to everyone on the anti-safety side being monetarily incentivized (VC's, OpenAI employees with $1m+ pay packages, startup founders seeking to get rich) compared to the people that never aimed to profit at all like Helen Toner, Yudkowsky, Bengio, and Hinton.
- silenced_trope 3y ago
  I've mentioned in past posts that the terms "safety" when applied to the AI discussion are never quantified or qualified in any way. It's more of a smear. If you're the one arguing from an AI safety standpoint, it means your opponent is not being safe and is dangerous. You should be expected to articulate why your position is "safer" and how "safe" is defined. In my opinion most AI safety arguments are about vaguely defined variables like speed: "it's going too fast" - to who? who defines what fast is? do you slow down 2x or 10x? do competitors and/or rival nations less concerned zip by and the end result is still the same? There is some validity to the discussion if AI is being "racist" or "biased" (again however you define these) but again competitors and/or rival nations may be less concerned and the end result may still be the same. If anything the "safest" option is to define what your concerns are, then race ahead to try to be the canonical solution or defined standard in order to set standards for the rest to follow or be bound by.
lacker 3y ago
The most embarassing thing about this isn't that the board decided to fire the CEO. That happens all the time. Many startups replace their CEO, it isn't always clear whether it's the right decision, but that's the responsibility of the board. The embarassing part is that the board decided to fire the CEO, announced their decision, refused to say why, attempted to put in place a new CEO but had to immediately demote the new CEO (Mira) after she rejected their plan, upset and alienated their core partners, along with almost all of their employees, and then publicly backtracked to undo the firing that led to this all happening. Once you screw this up to such an incredible degree, how can anyone really trust that you were doing the rest of your job well?
- brindlejim 3y ago
  [flagged]
- caesil 3y ago
  Interesting how an intellectual movement claiming to know better than anyone else how development of an unpredictable technology might pan out over the course of years failed to predict how one decision would pan out over the course of a single weekend. Perhaps there's some level of overconfidence at play from systems thinkers who overintellectualize their ability to conceptualize and extrapolate forward an impossibly complex system.
  - jacquesm 3y ago
    > Perhaps there's some level of overconfidence at play from systems thinkers who overintellectualize their ability to conceptualize and extrapolate forward an impossibly complex system. You may well be on to something. I'd trust a cabal of science fiction writers more than that I would trust these self appointed governors of our collective future. They lack imagination, for starters.
    - OccamsMirror 3y ago
      Science fiction writers like L. Ron Hubbard?
COGlory 3y ago
The more I learn about Altman, the more I begin to think he is a pathological liar.
- black6 3y ago
  He and all his contemporaries are. You can't rise to that position without being a sociopath. https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
  - alvah 3y ago
    If you're still young and idealistic, it's maybe ok to think like this, as you've likely had nonsense poured into your head for years & little real-world experience. If you're as old as me and you still think this way, there's little hope for you. I have worked for many CEOs, and (surprise!) fewer than 100% of them are sociopaths. Good grief.
- Vvector 3y ago
  He's not lying, he's hallucinating...
  - consumer451 3y ago
    This comment is potentially so much deeper than it first appears. I am not saying this necessarily as a criticism, but self-delusion appears to be an important part of many founder and executive stories.
- whynotminot 3y ago
  “I’m starting to think this CEO guy is like all the other CEO guys.” Insightful.
yinser 3y ago
I think the OpenAI shenanigans are far from over and the next act is likely to play out. A couple of my personal opinions are: - As PG pointed out Sam would find his way to the top of an island of cannibals. If you're generous to Toner's side in this article she likely _was_ alarmed by how Sam tried to get other board members to turn against her. When she walked into the arena with Altman to try and remove him however he had much more experience and orchestrated an incredible counter-coup. - There must be an army of attorneys representing the board members, and investors because there are huge potential legal pitfalls to destroying value, or failing fiduciary duties. - We don't know a lot because anything put out in public has to be spotless legally. If you're a board member and you know the truth, the only upside to leaking more info is clout, but the downsides are being sued and tied up in court losing a lot of time and money. - People seem dissatisfied with what was revealed in this article but Toner has likely been advised by lots of counsel on exactly what can be said, and the list of those things is probably miniscule. I'd say she's doing the best she can to give the public more info and I'm happy she's taken the effort to do so.
- tivert 3y ago
  > When she walked into the arena with Altman to try and remove him however he had much more experience and orchestrated an incredible counter-coup. Nit: Altman may have orchestrated a coup, but the board certainly did not. The board is the legitimate authority, and in no way is a board firing its subordinate a "coup." Rather, it's executing its legitimate power. Coups triggered when subordinate feels threatened is a pretty common pattern. IIRC, the recent coup in Niger was triggered when the leader of the presidential guard thought the president was about to fire him, too power in a coup rather than lose his job.
  - jacquesm 3y ago
    Correct, the board acted rashly and without forethought but well within their authority. Their big failure was that they didn't make it stick.
    - lukasb 3y ago
deleted 3y ago
[deleted]
boiler_up800 3y ago
She didn’t think there would be employee backlash when they fired the guy who was almost finished closing a financing that would make most senior employees millionaires? I respect that she and other board members didn’t like that Sam was trying to manipulate them into ousting her but their counterattack was poorly thought out.
sertbdfgbnfgsd 3y ago
> The board’s mandate is to “humanity,” not investors. Somebody put her there, told her "you know, say it's for humanity or something" and she actually believed it. No, it's for people to get rich.
- pseudonom- 3y ago
  Are you aware of the history and governance structure of OpenAI?
  - sertbdfgbnfgsd 3y ago
    No, and I don't think it particularly matters. In practice, the people who signal they'll make their friends rich will have the most backers. The board is just there to give the appearance of checks and balances. If these two claims sound outrageous consider that this is exactly how it's playing out. But you seem to have something in mind. Tell me.
    - pseudonom- 3y ago
      OpenAI was formed as a nonprofit with a specific charter ("OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.") and the capped-profit entity under which daily operations occur formed years later with the claim that it was instrumentally useful for pursuing that charter. The capped-profit entity remains a subsidiary of the nonprofit. The board in the dispute is the board of the overseeing nonprofit. So there are many particulars that mean pattern matching to a standard board dispute will lose something. I think it's likely many of the primary actors have, at various times, had non-strictly-pecuniary motives. That one side won doesn't mean that the other side was always a farce.
      - sertbdfgbnfgsd 3y ago
blueyes 3y ago
Any board member who believes it is her duty to destroy the organization she pledged to serve should simply step down. The folks who argue that it is a CEO's duty to serve the board do not understand governance or power. Open AI would have wallowed in irrelevance had Altman not raised the billions from Microsoft to fund its research. Because he was able to do that, and built the relationship with Nadella, he had and has power. Toner's behavior seems naive in that context. But naiveté among academics does not surprise me.
- Kinrany 3y ago
  Your argument is "might makes right".
  - blueyes 3y ago
    No, my argument is: if you do not know how to navigate power, you have no business on a board that purports to govern a fast-growing, prominent and critical organization. This board does not exist to serve itself, its abstractions or its ungrounded anxieties. It exists to practically guide a real organization employing many hundreds of people and serving many millions. There is no room for naiveté.
- jacquesm 3y ago
  My reading: They weren't picked for their ethics, they were picked for being malleable, to serve as a fig-leaf and to be able to mislead regulators. To Sam's surprise some of them took their job serious, so he tried to get rid of them because he saw the confrontation as inevitable. That backfired, he got terminated, and then that backfired so the boardmembers got the choice to jump on command or vacate the premises. You can be fairly certain that the board will be stocked with 'yes-men' (and women) beholden to Altman. He will not make that same mistake twice.
- kippinitreal 3y ago
  I mean, it's totally fine for a non-profit to "wallow in irrelevance"? I think the fundamental issue here is that a tax-exempt non-profit organization like these board members are leading _shouldn't_ necessarily be chasing fame/fortune. By definition they've put mission above profits (with tax writeoffs as a benefit). The naivety seems to be that the company is trying to be altruistically mission driven while also acting like a typical ambitious/profit seeking startup. Startups are great! Just weird to wrap one in a non-profit which brings different incentives
MicolashKyoka 3y ago
openai is where it's at because of sama & greg (& ofc contributions of employees, but funding & solid engineering leadership is a big part in its success), to think that she could just push them away without it backfiring is mindblowing.
angarg12 3y ago
> Helen Toner was a relatively unknown 31-year-old academic from Australia—until she became one of the four board members who fired Sam Altman > Toner graduated from the University of Melbourne, Australia, in 2014 with a degree in chemical engineering [...] In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, [...] She succeeded her former manager from Open Philanthropy, Holden Karnofsky, on the OpenAI board in 2021 after he stepped down. Honest question, how do people find themselves in the board of one of the hottest startups at the tender age of 31? Are they geniuses or is it all about connections?
- Angostura 3y ago
  It wasn't very hot in 2021
- reducesuffering 3y ago
  These people took chances on their belief in the future, that AGI was coming and monumentally impactful, back in 2015 when the entirety of people on HN mocked and derided it. Even in 2023, the majority do.
- mrlatinos 3y ago
  Her father: http://www.toner-assoc.com.au/Consultants.htm http://www.toner-assoc.com.au/Consultants.htm Her brothers: https://www.vesparum.com/origins https://www.vesparum.com/origins So the short answer is wealth and connections, or daddy.
  - rurban 3y ago
    Or intelligence. Looking at her career and papers sounds like being in the intelligence business, not nepo. AI safety is military intelligence only
- kj4211cash 3y ago
  Right place, right time. She was one of very few people "working" on AI Safety. I made another comment about this elsewhere on this post, but to me this debacle is a function of the fact that there is no there there when it comes to AI Safety "research".
m3kw9 3y ago
I’m glad she’s off the board. You cannot have a trigger happy fire an important ceo over some to me is a small trust issue, infact it seems like she’s over using her powers and “hold principle” and destroy everything even to achieve it. It proves she cannot think inbetween, saying it’s “protect humanity at all costs”, you really need a brain to execute that and she has proven it was a massive f-up
- jacquesm 3y ago
  > You cannot have a trigger happy fire an important ceo over some to me is a small trust issue Well, if that was the case it actually was fine but there is no hard proof that it was like that and if it was they messed it up. So failure on all counts.
- lukeschlather 3y ago
  This doesn't sound like a small trust issue. It sounds like either Altman had to go or she had to go, and Altman made that decision himself, he really only gave her two options: get Altman fired or resign.
m3kw9 3y ago
Allowing the board to control AI safety is BS. Is like ethics, the engineer is usually imbued with ethics when they work at any firm. It’s that simple, you work at an airline manufacturer, your duty is to not cause harm, test properly, it’s no different with AI. The engineer understands this more than any board member getting paid to do nothing.
- 0xDEAFBEAD 3y ago
  >the engineer is usually imbued with ethics when they work at any firm. It’s that simple, you work at an airline manufacturer, your duty is to not cause harm, test properly, it’s no different with AI. One point here is that there's no licensing requirement for AI engineers like you see in some other fields of engineering. As someone recently quipped on Twitter: > did you know that there are currently no licensing requirements in the US for training potentially world-threatening AI models, but there are in the state of Louisiana for selling flower arrangements https://nitter.net/littIeramblings/status/1730925237326565425#m https://nitter.net/littIeramblings/status/173092523732656542... In any case, what's been happening in AI engineering is significantly different than in other fields of engineering. There is no consensus as to the magnitude of AI risk, although extinction probability estimates are generally high: https://nitter.net/AISafetyMemes/status/1729892336782524676#m https://nitter.net/AISafetyMemes/status/1729892336782524676#... In practice, AI engineers self-select into different places of employment depending on how worried they are. If you're not very worried, you work at Meta. If you're significantly worried, you work at Anthropic or even someplace like MIRI. Most AI labs have a safety culture that's somewhere in between those extremes. [Obviously, from the point of view of preventing extinction, this situation seems unsatisfactory.] With regard to the point about the board -- their job was to supervise the CEO, and they say Sam was making that difficult. See https://nitter.net/hlntnr/status/1730034020140912901#m https://nitter.net/hlntnr/status/1730034020140912901#m
kj4211cash 3y ago
It seems to me as if Toner was just in the right place at the right time in order to get a seat on the board of Open AI. She has an undergrad degree in chemical engineering and a master's in "security studies." Her work on AI Safety opened this door for her but seems, at least to me, to be ... superficial. I'm no expert but I have worked in tech for awhile and at a public policy think tank. So I guess my question is there a real scientific field of AI Safety? Are there any real experts? Are there any real insights? I dislike the idea of trusting giant tech companies with breakthrough technologies with minimal oversight / regulation. But it just seems like there is no real science regarding AI x public policy. Like the policy experts have no clue what they are talking about. And after this debacle they probably won't be lucky enough to find themselves on the boards of organizations like Open AI.
- 0xDEAFBEAD 3y ago
  >So I guess my question is there a real scientific field of AI Safety? Are there any real experts? Are there any real insights? I'd say so, and in fact it's older than you might think. Here's a lit review from 2018 for example: https://arxiv.org/pdf/1805.01109.pdf https://arxiv.org/pdf/1805.01109.pdf This post is more recent, but understanding it might require some context: https://www.alignmentforum.org/posts/zaaGsFBeDTpCsYHef/shallow-review-of-live-agendas-in-alignment-and-safety https://www.alignmentforum.org/posts/zaaGsFBeDTpCsYHef/shall... A group from Cambridge has been running an online course you can take if you're interested in getting up to speed: https://aisafetyfundamentals.com/ https://aisafetyfundamentals.com/ Application deadline is early February. I took the course a couple years ago; happy to answer any questions.
- sytelus 3y ago
  There are real experts on AI safety. the way to distinguish between them and the others is simply ask where is the code for your research?
trace_id 3y ago
She is a front for you know who.
benreesman 3y ago
AI Safety or lack thereof, to the extent that it emerges as a real threat, will be a matter for law enforcement. Just like “terrorism” and “drugs” and all the other vaguely menacing things that ended up being worth a pile of money to some incumbent interest, the real story will be boring: some crooks used a fancy-ass LLM to take a bank apart or something. The bank calls the cops, this happens about twice before the cops have a division for this sort of thing, and sooner or later someone is in a courtroom. It would be nauseatingly boring to get into how many times this has happened. Also predictably depressing, the real story, which is yet another brick in the wall of “we’re handing the reigns to people who make movie Zuck look flat fucking normal”, goes largely unreported. We used to complain about dual-class share structures. Sam just eviscerated a board of an ostensible charity with the full backing of MSFT and got himself installed as Ungovernable God Emperor of Arrakis for life. Still doing the eyeball scanner thing. Still has never shown the public he can handle numpy to the tune of MNIST. This is all in broad daylight and the Gemini launch gets clobbered because the 4-series comparable one comes out in January. Sam’s a smart guy, I’m quite sure he’d pay attention if the braintrust (this site) complained loudly. But the message is: “more, not less”.
gardenhedge 3y ago
Helen Toner has proven herself to be a poor communicator. This is what will happen if you give inexperienced people, that have not earned it, board seats.
siliconc0w 3y ago
Missing the quarter, sure that's grounds. Lying and backstabbing is SOP.
patcon 3y ago
> Some of Altman’s backers, including OpenAI investor Vinod Khosla, publicly expressed derision specifically toward Toner and Tasha McCauley, another former OpenAI board member who voted to fire Altman and is connected to organizations that promote effective altruism. > “Fancy titles like ‘Director of Strategy at Georgetown’s Center for Security and Emerging Technology’ can lead to a false sense of understanding of the complex process of entrepreneurial innovation,” Khosla wrote in an essay in tech-news publication the Information, referring to Toner and her current position. Strong words from the infamous beach villain and resident asshole, Vinod Khosla... Every Generation Gets the Beach Villain It Deserves (NYT) https://archive.is/6mss9 https://archive.is/6mss9

150 points by Leary 3y ago

https://archive.is/gAkKC https://archive.is/gAkKC

It’s interesting that she was threatened with being held responsible for violating a fiduciary duty for the for-profit entity while she was sitting on the non-profit board. Going fwd, I wonder why they cannot convert the structure to a traditional c-corp. Supposedly: “tax issues”. Whatever problems OpenAI dealt with last week, the current non-profit structure will continue to cause future problems IMO.

mrkramer 3y ago
Microsoft should acquire them and cut the non-profit crap.
- happytiger 3y ago
  Isn’t that basically what is happening with a few extra steps?
  - mrkramer 3y ago
    Yes....it's actually Microsoft vs Google AI arms race, that's why Google went all in with their LLM technology.
    - mistrial9 3y ago
      > cut the non-profit crap Great learning opportunity for those that have not taken time to learn and understand non-profit structures in the USA. this is clearly an opportunity for the uninformed to study the US non-profit system, unique in the world. For example, I recall a delegation from Japan touring the HQ of a non-profit educational group in San Francisco. Like the p-poster here, they apparently had no clue why or how capable people could come together and form stable work culture aside from "hierarchical, profit motive in USD$" (.. and people did actual work, unlike many management level stock manipulators).
      - mrkramer 3y ago
        I'm from southeast Europe and I graduated from IB high school programme[0], it is an international Swiss based educational programme that prepares you to study abroad and it is a non-profit. As a non-profit they are efficient af at what they do but in the world of technological innovations, incentives are so much higher and so much more impactful when an organization is for-profit. [0] https://en.wikipedia.org/wiki/International_Baccalaureate https://en.wikipedia.org/wiki/International_Baccalaureate
      - mistrial9 3y ago
        Mark Surman writes: As I wrote in Fast Company earlier this week, we risk missing the real lesson from the OpenAI governance debacle. ... "In the aftermath of the OpenAI governance battle, the narrative has been dominated by the idea that “the capitalists won.” While this conclusion is catchy, it misses the point. The real (and bigger) story is: The founders of OpenAI set out to create a nonprofit public institution to ensure that AI’s evolution aligns with the interests of humanity at large. Last month, it became clear that they failed. What is also clear: Public institutions that prioritize the public good over short-term profits are desperately needed for the AI era."
Havoc 3y ago
> fiduciary duty for the for-profit entity I read it as both. If OpenAI collapses operationally then the non profit fails at its mission too. That’s the board she is on and that’s the fiduciary duty relevant (She may also be on the for profit board idk)
- stcredzero 3y ago
  In the current legal framework, isn't the reality, fiduciary uber alles?
- webmaven 3y ago
  Wait, does the for-profit even have a board?
  - Havoc 3y ago
    Had a quick look at structure and I'd say probably not
- 0xDEAFBEAD 3y ago
  >If OpenAI collapses operationally then the non profit fails at its mission too. Not necessarily true. "We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome [...] if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project." https://openai.com/charter https://openai.com/charter
dragonwriter 3y ago
> Going fwd, I wonder why they cannot convert the structure to a traditional c-corp. Supposedly: “tax issues”. Because, IIRC, when a charity nonprofit like OpenAI, Inc.-- a 501(c)(3), other nonprofits are different -- converts to a for-profit (C-corp or otherwise doesn't matter), all the assets acquired while it was a charity must either remain dedicated to and used exclusively for the charitable purpose, be returned, or be donated to charity (or sold, and then the proceeds treated the same way.) For the charity OpenAI, Inc., this would include its interest in OpenAI, LLC (the wholly owned subsidiary through which it exercises control of OpenAI, LP and OpenAI Global, LLC -- the latter being the for profit entity that actually sells product, etc.), and OpenAI, LP (the holding company which in turn is majority owner of OpenAI Global, LLC).
- gnulinux 3y ago
  Just curious: can they legally sell all their assets to themselves (i.e. OpenAI nonprofit sells it to for-profit OpenAI Inc. for market price real $$$) and then use that $$$ for charitable purposes. Would this satisfy IRS by letting them retain the intellectual property rights?
  - yieldcrv 3y ago
    self dealing regulations are to prevent that. also OpenAI Inc. for profit doesnt exist, a wholly owned LLC subsidiary does. so they would need to divest of that LLC first (by selling the shares) the way around the self dealing regulations are to reduce their ownership below 20% - 35% (yes, its that range, for multiple reasons) good thing here is that most venture backed companies subvert this already by simply share dilution and youre breaking into one the best secrets of Silicon Valley. Non-profits pump the value of startups and launder the reputation of the founders because their ideas get so much “validation” so quickly. (non profits they partially control act as lead investors for startups they control. they both sell shares at higher price, ideally: founders get QSBS tax free treatment, non profit retains tax free capital gains too. nothing wrong with that, you just probably cant do it too) so open ai for profit llc needs to get rid of the PPU’s and bring in outside investors to dilute ownership enough, but two birds one stone: the outside investors have to get a higher valuation
WillPostForFood 3y ago
It’s interesting that she was threatened with being held responsible for violating a fiduciary duty for the for-profit entity while she was sitting on the non-profit board. Fiduciary doesn't mean financial or monetary duty, it means holding something in trust. Violating a fiduciary duty could be financial, social, reputational, or in this case, literally destroying the operation of the company you were entrusted with.
jacquesm 3y ago
Tax issues and regulatory issues both.
JKCalhoun 3y ago
> It’s interesting that she was threatened with being held responsible for violating a fiduciary duty Yeah, that smells like a bullshit threat. Anyone would do well to ignore it.

nerdo 3y ago

> Toner maintains that safety wasn’t the reason the board wanted to fire Altman. Rather, it was a lack of trust > Toner declined to provide specific details on why she and the three others voted to fire Altman from OpenAI Does she see herself as more trustworthy? She can't even be bothered to give an excuse for the firing. Someone like this couldn't be trusted in literally any function, how did she get a board seat on OpenAI?

deleted 3y ago
[deleted]
gnulinux 3y ago
> how did she get a board seat on OpenAI? She's the leading academic on the issue of AI safety. It's really ridiculous people don't even know her name and say random things about her, not realizing she's a rock star in her field. Her leaving the board is a tremendous loss for OpenAI, about as terrible as Ilya leaving the board. These are two giants of AI currently.
- edgyquant 3y ago
  > the leading academic on the issue of AI safety. Sounds like a made up position to me. Just another bureaucrat with a cool sounding name
  - reducesuffering 3y ago
    Why don't you reserve your ignorant judgement until you do due diligence? https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en
    - PeterisP 3y ago
      As an academic from a different field, this is not a publication record that would be anywhere close to someone being "the leading academic on the issue of AI safety". It indicates that she has had been one of very many participants at two impactful papers (with zero indication of what her individual contribution was, with so many authors it might as well be being part of the funding organization), has done a few collaborations with strong researchers, and that her personal first-author or sole-author research is barely relevant. Like, it's not a bad publication record, especially for a junior researcher (heck, her first participation in other people's papers is just in 2018 and her own work starts in 2021-2022, so she's effectively just getting started and may have many opportunities to prove herself in the future), it does indicate doing a couple years of research and might qualify for an okayish faculty job when (if?) she gets her PhD, but it's not also something indicating a senior researcher that's the pillar of anything and has had some impact - a grad student who's lucky to study in a strong leading researcher's lab and does some work on two of the advisor's papers might have a similar publication record already at their graduation before their career has even seriously started. Many of her co-authors have significantly larger research impact and personal research (https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ or https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ for example or https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ) - which also raises a question of how much each co-author did in a two-author paper where the leading author is a strong researcher and the second author is a "Director of Strategy and Foundational Research Grants" i.e. getting money for that research, so labeling her as a leading researcher (much less the leading researcher) of some field seems misleading as the publication record would indicate that she's not even on the same level as they are.

chewmieser 3y ago

She specifically declines in the article to provide the additional context that would probably clarify everything. This is just more non-answers and only a tad more information than we had before (threats of violating a fiduciary duty). Nobody involved is being transparent about what went down here. Not sure we'll find out the full story here...

tfehring 3y ago
It's frustrating, but she's exposed to potentially massive personal liability for the board's actions. I'm surprised that she agreed to an interview at all - frankly, I probably wouldn't if I were in her position - and I'm sure her answers were carefully scripted and run by her attorney(s). I think this Reddit comment captures the most plausible story at this point: https://www.reddit.com/r/MachineLearning/comments/1812w04/comment/kabk73s/ https://www.reddit.com/r/MachineLearning/comments/1812w04/co...
- chewmieser 3y ago
  That’s great, thanks for the link there. Seems plausible enough to fill in the blanks.
- alvah 3y ago
  She wouldn't be the first high-profile member of the EA cult to talk to the press after a scandal, against the advice of any sane lawyer.
calf 3y ago
Article does give context as to what "inconsistently candid" meant: Altman allegedly went to each board member to say some negative things about Toner, but when board members got together to verify consistency they found out Altman said a false or misleading thing to at least one board member about Toner. That's not a good look on Altman at all, it looks like Altman was using divide-and-conquer in a skeezy way, spreading around negative impression, even rumors, about his bosses, as a means dispose of a problem.
- 3y ago

Leary 3y ago

Key passage: "Altman approached other board members, trying to convince each to fire Toner. Later, some board members swapped notes on their individual discussions with Altman. The group concluded that in one discussion with a board member, Altman left a misleading perception that another member thought Toner should leave, the people said. By this point, several of OpenAI’s then-directors already had concerns about Altman’s honesty, people familiar with their thinking said."

TeeWEE 3y ago
Wow, there is no way around it. Sam Altman is basically getting his way by misleading people. The board did the right thing. Honesty is crucial as a leader. I think for Sam Altman his goal is more important than ethics and he will do whatever to get his way.
- jfengel 3y ago
  Honesty is great, in moderation, but it's not the be-all and end-all of leadership. Leadership is fundamentally about manipulating people. If the people already agreed, they wouldn't need leadership. Your job as leader is to get them all working in more or less the same direction, even though they don't really want to. Honesty is a great way to do that, when it works. Honesty has the virtue of not having to keep track of your lies, and you don't get caught in an inconsistency. But it doesn't always work, and then you have to consider alternatives. Sometimes people are dumb. Sometimes people are wrong. Sometimes people are hostile. Sometimes people are right, but in a way that can't actually be implemented, and you need to go with a sub-optimal path despite that. At core, you can't lead if you don't believe that you've listened to all of the concerns and reached the solution that is best for the group as a whole, according to whatever definition of "best" and "whole" you were hired for. Sometimes that's going to force you to use other means that just telling people the truth and hoping they reach the same conclusion. Leadership is fundamentally a people problem, not a technical one. And sometimes that's going to mean being dishonest. A good leader does it as little as possible, but simply dismissing it as a tactic is a recipe for failure because it hides a fundamental truth about why and how people behave in groups.

YetAnotherNick 3y ago

> Toner maintains that safety wasn’t the reason the board wanted to fire Altman. This is the biggest news of it all So all the Q* thing and Ilya's safety position and other theories were incorrect?

gregatragenet3 3y ago

Anyone else come to the realization that when spokespeople talk about "AI Safety" they aren't concerned with the skynet-esque enslavement of mankind or paperclip maximizing, but that controls be in place that prevent people from using the technology in a way that is misaligned with the maximum extraction of profit?

wait_a_minute 3y ago
We need open source models that are trained on the same data and that can be built from source and run on lightweight machines like mobile devices.
- timschmidt 3y ago
  This seems relevant: https://arxiv.org/abs/2311.08105 https://arxiv.org/abs/2311.08105
justrealist 3y ago
Please stop framing your own personal cynicism as a grand narrative revalation.
reducesuffering 3y ago
Your "realization" is counter to everyone on the anti-safety side being monetarily incentivized (VC's, OpenAI employees with $1m+ pay packages, startup founders seeking to get rich) compared to the people that never aimed to profit at all like Helen Toner, Yudkowsky, Bengio, and Hinton.
silenced_trope 3y ago
I've mentioned in past posts that the terms "safety" when applied to the AI discussion are never quantified or qualified in any way. It's more of a smear. If you're the one arguing from an AI safety standpoint, it means your opponent is not being safe and is dangerous. You should be expected to articulate why your position is "safer" and how "safe" is defined. In my opinion most AI safety arguments are about vaguely defined variables like speed: "it's going too fast" - to who? who defines what fast is? do you slow down 2x or 10x? do competitors and/or rival nations less concerned zip by and the end result is still the same? There is some validity to the discussion if AI is being "racist" or "biased" (again however you define these) but again competitors and/or rival nations may be less concerned and the end result may still be the same. If anything the "safest" option is to define what your concerns are, then race ahead to try to be the canonical solution or defined standard in order to set standards for the rest to follow or be bound by.

lacker 3y ago

The most embarassing thing about this isn't that the board decided to fire the CEO. That happens all the time. Many startups replace their CEO, it isn't always clear whether it's the right decision, but that's the responsibility of the board. The embarassing part is that the board decided to fire the CEO, announced their decision, refused to say why, attempted to put in place a new CEO but had to immediately demote the new CEO (Mira) after she rejected their plan, upset and alienated their core partners, along with almost all of their employees, and then publicly backtracked to undo the firing that led to this all happening. Once you screw this up to such an incredible degree, how can anyone really trust that you were doing the rest of your job well?

brindlejim 3y ago
[flagged]
caesil 3y ago
Interesting how an intellectual movement claiming to know better than anyone else how development of an unpredictable technology might pan out over the course of years failed to predict how one decision would pan out over the course of a single weekend. Perhaps there's some level of overconfidence at play from systems thinkers who overintellectualize their ability to conceptualize and extrapolate forward an impossibly complex system.
- jacquesm 3y ago
  > Perhaps there's some level of overconfidence at play from systems thinkers who overintellectualize their ability to conceptualize and extrapolate forward an impossibly complex system. You may well be on to something. I'd trust a cabal of science fiction writers more than that I would trust these self appointed governors of our collective future. They lack imagination, for starters.
  - OccamsMirror 3y ago
    Science fiction writers like L. Ron Hubbard?

COGlory 3y ago

The more I learn about Altman, the more I begin to think he is a pathological liar.

black6 3y ago
He and all his contemporaries are. You can't rise to that position without being a sociopath. https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
- alvah 3y ago
  If you're still young and idealistic, it's maybe ok to think like this, as you've likely had nonsense poured into your head for years & little real-world experience. If you're as old as me and you still think this way, there's little hope for you. I have worked for many CEOs, and (surprise!) fewer than 100% of them are sociopaths. Good grief.
Vvector 3y ago
He's not lying, he's hallucinating...
- consumer451 3y ago
  This comment is potentially so much deeper than it first appears. I am not saying this necessarily as a criticism, but self-delusion appears to be an important part of many founder and executive stories.
whynotminot 3y ago
“I’m starting to think this CEO guy is like all the other CEO guys.” Insightful.

yinser 3y ago

I think the OpenAI shenanigans are far from over and the next act is likely to play out. A couple of my personal opinions are: - As PG pointed out Sam would find his way to the top of an island of cannibals. If you're generous to Toner's side in this article she likely _was_ alarmed by how Sam tried to get other board members to turn against her. When she walked into the arena with Altman to try and remove him however he had much more experience and orchestrated an incredible counter-coup. - There must be an army of attorneys representing the board members, and investors because there are huge potential legal pitfalls to destroying value, or failing fiduciary duties. - We don't know a lot because anything put out in public has to be spotless legally. If you're a board member and you know the truth, the only upside to leaking more info is clout, but the downsides are being sued and tied up in court losing a lot of time and money. - People seem dissatisfied with what was revealed in this article but Toner has likely been advised by lots of counsel on exactly what can be said, and the list of those things is probably miniscule. I'd say she's doing the best she can to give the public more info and I'm happy she's taken the effort to do so.

tivert 3y ago
> When she walked into the arena with Altman to try and remove him however he had much more experience and orchestrated an incredible counter-coup. Nit: Altman may have orchestrated a coup, but the board certainly did not. The board is the legitimate authority, and in no way is a board firing its subordinate a "coup." Rather, it's executing its legitimate power. Coups triggered when subordinate feels threatened is a pretty common pattern. IIRC, the recent coup in Niger was triggered when the leader of the presidential guard thought the president was about to fire him, too power in a coup rather than lose his job.
- jacquesm 3y ago
  Correct, the board acted rashly and without forethought but well within their authority. Their big failure was that they didn't make it stick.
  - lukasb 3y ago

deleted 3y ago

[deleted]

boiler_up800 3y ago

She didn’t think there would be employee backlash when they fired the guy who was almost finished closing a financing that would make most senior employees millionaires? I respect that she and other board members didn’t like that Sam was trying to manipulate them into ousting her but their counterattack was poorly thought out.

sertbdfgbnfgsd 3y ago

> The board’s mandate is to “humanity,” not investors. Somebody put her there, told her "you know, say it's for humanity or something" and she actually believed it. No, it's for people to get rich.

pseudonom- 3y ago
Are you aware of the history and governance structure of OpenAI?
- sertbdfgbnfgsd 3y ago
  No, and I don't think it particularly matters. In practice, the people who signal they'll make their friends rich will have the most backers. The board is just there to give the appearance of checks and balances. If these two claims sound outrageous consider that this is exactly how it's playing out. But you seem to have something in mind. Tell me.
  - pseudonom- 3y ago
    OpenAI was formed as a nonprofit with a specific charter ("OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.") and the capped-profit entity under which daily operations occur formed years later with the claim that it was instrumentally useful for pursuing that charter. The capped-profit entity remains a subsidiary of the nonprofit. The board in the dispute is the board of the overseeing nonprofit. So there are many particulars that mean pattern matching to a standard board dispute will lose something. I think it's likely many of the primary actors have, at various times, had non-strictly-pecuniary motives. That one side won doesn't mean that the other side was always a farce.
    - sertbdfgbnfgsd 3y ago

blueyes 3y ago

Any board member who believes it is her duty to destroy the organization she pledged to serve should simply step down. The folks who argue that it is a CEO's duty to serve the board do not understand governance or power. Open AI would have wallowed in irrelevance had Altman not raised the billions from Microsoft to fund its research. Because he was able to do that, and built the relationship with Nadella, he had and has power. Toner's behavior seems naive in that context. But naiveté among academics does not surprise me.

Kinrany 3y ago
Your argument is "might makes right".
- blueyes 3y ago
  No, my argument is: if you do not know how to navigate power, you have no business on a board that purports to govern a fast-growing, prominent and critical organization. This board does not exist to serve itself, its abstractions or its ungrounded anxieties. It exists to practically guide a real organization employing many hundreds of people and serving many millions. There is no room for naiveté.
jacquesm 3y ago
My reading: They weren't picked for their ethics, they were picked for being malleable, to serve as a fig-leaf and to be able to mislead regulators. To Sam's surprise some of them took their job serious, so he tried to get rid of them because he saw the confrontation as inevitable. That backfired, he got terminated, and then that backfired so the boardmembers got the choice to jump on command or vacate the premises. You can be fairly certain that the board will be stocked with 'yes-men' (and women) beholden to Altman. He will not make that same mistake twice.
kippinitreal 3y ago
I mean, it's totally fine for a non-profit to "wallow in irrelevance"? I think the fundamental issue here is that a tax-exempt non-profit organization like these board members are leading _shouldn't_ necessarily be chasing fame/fortune. By definition they've put mission above profits (with tax writeoffs as a benefit). The naivety seems to be that the company is trying to be altruistically mission driven while also acting like a typical ambitious/profit seeking startup. Startups are great! Just weird to wrap one in a non-profit which brings different incentives

MicolashKyoka 3y ago

openai is where it's at because of sama & greg (& ofc contributions of employees, but funding & solid engineering leadership is a big part in its success), to think that she could just push them away without it backfiring is mindblowing.

angarg12 3y ago

> Helen Toner was a relatively unknown 31-year-old academic from Australia—until she became one of the four board members who fired Sam Altman > Toner graduated from the University of Melbourne, Australia, in 2014 with a degree in chemical engineering [...] In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, [...] She succeeded her former manager from Open Philanthropy, Holden Karnofsky, on the OpenAI board in 2021 after he stepped down. Honest question, how do people find themselves in the board of one of the hottest startups at the tender age of 31? Are they geniuses or is it all about connections?

Angostura 3y ago
It wasn't very hot in 2021
reducesuffering 3y ago
These people took chances on their belief in the future, that AGI was coming and monumentally impactful, back in 2015 when the entirety of people on HN mocked and derided it. Even in 2023, the majority do.
mrlatinos 3y ago
Her father: http://www.toner-assoc.com.au/Consultants.htm http://www.toner-assoc.com.au/Consultants.htm Her brothers: https://www.vesparum.com/origins https://www.vesparum.com/origins So the short answer is wealth and connections, or daddy.
- rurban 3y ago
  Or intelligence. Looking at her career and papers sounds like being in the intelligence business, not nepo. AI safety is military intelligence only
kj4211cash 3y ago
Right place, right time. She was one of very few people "working" on AI Safety. I made another comment about this elsewhere on this post, but to me this debacle is a function of the fact that there is no there there when it comes to AI Safety "research".

m3kw9 3y ago

I’m glad she’s off the board. You cannot have a trigger happy fire an important ceo over some to me is a small trust issue, infact it seems like she’s over using her powers and “hold principle” and destroy everything even to achieve it. It proves she cannot think inbetween, saying it’s “protect humanity at all costs”, you really need a brain to execute that and she has proven it was a massive f-up

jacquesm 3y ago
> You cannot have a trigger happy fire an important ceo over some to me is a small trust issue Well, if that was the case it actually was fine but there is no hard proof that it was like that and if it was they messed it up. So failure on all counts.
lukeschlather 3y ago
This doesn't sound like a small trust issue. It sounds like either Altman had to go or she had to go, and Altman made that decision himself, he really only gave her two options: get Altman fired or resign.

m3kw9 3y ago

Allowing the board to control AI safety is BS. Is like ethics, the engineer is usually imbued with ethics when they work at any firm. It’s that simple, you work at an airline manufacturer, your duty is to not cause harm, test properly, it’s no different with AI. The engineer understands this more than any board member getting paid to do nothing.

0xDEAFBEAD 3y ago
>the engineer is usually imbued with ethics when they work at any firm. It’s that simple, you work at an airline manufacturer, your duty is to not cause harm, test properly, it’s no different with AI. One point here is that there's no licensing requirement for AI engineers like you see in some other fields of engineering. As someone recently quipped on Twitter: > did you know that there are currently no licensing requirements in the US for training potentially world-threatening AI models, but there are in the state of Louisiana for selling flower arrangements https://nitter.net/littIeramblings/status/1730925237326565425#m https://nitter.net/littIeramblings/status/173092523732656542... In any case, what's been happening in AI engineering is significantly different than in other fields of engineering. There is no consensus as to the magnitude of AI risk, although extinction probability estimates are generally high: https://nitter.net/AISafetyMemes/status/1729892336782524676#m https://nitter.net/AISafetyMemes/status/1729892336782524676#... In practice, AI engineers self-select into different places of employment depending on how worried they are. If you're not very worried, you work at Meta. If you're significantly worried, you work at Anthropic or even someplace like MIRI. Most AI labs have a safety culture that's somewhere in between those extremes. [Obviously, from the point of view of preventing extinction, this situation seems unsatisfactory.] With regard to the point about the board -- their job was to supervise the CEO, and they say Sam was making that difficult. See https://nitter.net/hlntnr/status/1730034020140912901#m https://nitter.net/hlntnr/status/1730034020140912901#m

kj4211cash 3y ago

It seems to me as if Toner was just in the right place at the right time in order to get a seat on the board of Open AI. She has an undergrad degree in chemical engineering and a master's in "security studies." Her work on AI Safety opened this door for her but seems, at least to me, to be ... superficial. I'm no expert but I have worked in tech for awhile and at a public policy think tank. So I guess my question is there a real scientific field of AI Safety? Are there any real experts? Are there any real insights? I dislike the idea of trusting giant tech companies with breakthrough technologies with minimal oversight / regulation. But it just seems like there is no real science regarding AI x public policy. Like the policy experts have no clue what they are talking about. And after this debacle they probably won't be lucky enough to find themselves on the boards of organizations like Open AI.

0xDEAFBEAD 3y ago
>So I guess my question is there a real scientific field of AI Safety? Are there any real experts? Are there any real insights? I'd say so, and in fact it's older than you might think. Here's a lit review from 2018 for example: https://arxiv.org/pdf/1805.01109.pdf https://arxiv.org/pdf/1805.01109.pdf This post is more recent, but understanding it might require some context: https://www.alignmentforum.org/posts/zaaGsFBeDTpCsYHef/shallow-review-of-live-agendas-in-alignment-and-safety https://www.alignmentforum.org/posts/zaaGsFBeDTpCsYHef/shall... A group from Cambridge has been running an online course you can take if you're interested in getting up to speed: https://aisafetyfundamentals.com/ https://aisafetyfundamentals.com/ Application deadline is early February. I took the course a couple years ago; happy to answer any questions.
sytelus 3y ago
There are real experts on AI safety. the way to distinguish between them and the others is simply ask where is the code for your research?

trace_id 3y ago

She is a front for you know who.

benreesman 3y ago

AI Safety or lack thereof, to the extent that it emerges as a real threat, will be a matter for law enforcement. Just like “terrorism” and “drugs” and all the other vaguely menacing things that ended up being worth a pile of money to some incumbent interest, the real story will be boring: some crooks used a fancy-ass LLM to take a bank apart or something. The bank calls the cops, this happens about twice before the cops have a division for this sort of thing, and sooner or later someone is in a courtroom. It would be nauseatingly boring to get into how many times this has happened. Also predictably depressing, the real story, which is yet another brick in the wall of “we’re handing the reigns to people who make movie Zuck look flat fucking normal”, goes largely unreported. We used to complain about dual-class share structures. Sam just eviscerated a board of an ostensible charity with the full backing of MSFT and got himself installed as Ungovernable God Emperor of Arrakis for life. Still doing the eyeball scanner thing. Still has never shown the public he can handle numpy to the tune of MNIST. This is all in broad daylight and the Gemini launch gets clobbered because the 4-series comparable one comes out in January. Sam’s a smart guy, I’m quite sure he’d pay attention if the braintrust (this site) complained loudly. But the message is: “more, not less”.

gardenhedge 3y ago

Helen Toner has proven herself to be a poor communicator. This is what will happen if you give inexperienced people, that have not earned it, board seats.

siliconc0w 3y ago

Missing the quarter, sure that's grounds. Lying and backstabbing is SOP.

patcon 3y ago

> Some of Altman’s backers, including OpenAI investor Vinod Khosla, publicly expressed derision specifically toward Toner and Tasha McCauley, another former OpenAI board member who voted to fire Altman and is connected to organizations that promote effective altruism. > “Fancy titles like ‘Director of Strategy at Georgetown’s Center for Security and Emerging Technology’ can lead to a false sense of understanding of the complex process of entrepreneurial innovation,” Khosla wrote in an essay in tech-news publication the Information, referring to Toner and her current position. Strong words from the infamous beach villain and resident asshole, Vinod Khosla... Every Generation Gets the Beach Villain It Deserves (NYT) https://archive.is/6mss9 https://archive.is/6mss9