5 ms·

> how did she get a board seat on OpenAI? She's the leading academic on the issue of AI safety. It's really ridiculous people don't even know her name and say

> how did she get a board seat on OpenAI? She's the leading academic on the issue of AI safety. It's really ridiculous people don't even know her name and say random things about her, not realizing she's a rock star in her field. Her leaving the board is a tremendous loss for OpenAI, about as terrible as Ilya leaving the board. These are two giants of AI currently.

edgyquant 3y ago
> the leading academic on the issue of AI safety. Sounds like a made up position to me. Just another bureaucrat with a cool sounding name
- reducesuffering 3y ago
  Why don't you reserve your ignorant judgement until you do due diligence? https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en
  - PeterisP 3y ago
    As an academic from a different field, this is not a publication record that would be anywhere close to someone being "the leading academic on the issue of AI safety". It indicates that she has had been one of very many participants at two impactful papers (with zero indication of what her individual contribution was, with so many authors it might as well be being part of the funding organization), has done a few collaborations with strong researchers, and that her personal first-author or sole-author research is barely relevant. Like, it's not a bad publication record, especially for a junior researcher (heck, her first participation in other people's papers is just in 2018 and her own work starts in 2021-2022, so she's effectively just getting started and may have many opportunities to prove herself in the future), it does indicate doing a couple years of research and might qualify for an okayish faculty job when (if?) she gets her PhD, but it's not also something indicating a senior researcher that's the pillar of anything and has had some impact - a grad student who's lucky to study in a strong leading researcher's lab and does some work on two of the advisor's papers might have a similar publication record already at their graduation before their career has even seriously started. Many of her co-authors have significantly larger research impact and personal research (https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ or https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ for example or https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ) - which also raises a question of how much each co-author did in a two-author paper where the leading author is a strong researcher and the second author is a "Director of Strategy and Foundational Research Grants" i.e. getting money for that research, so labeling her as a leading researcher (much less the leading researcher) of some field seems misleading as the publication record would indicate that she's not even on the same level as they are.
  - MisterBastahrd 3y ago
    And what happens when they read the articles and come to realize that you could give an average college freshman any of those subjects and they could produce a comparable paper in a few weeks' time with a full course load and a part time job? Seriously, I've never seen a more underwhelming set of articles on google scholar from a supposed "leading" expert in a field... and what's worse is that she wasn't even the lead writer on most of them.
  - WillPostForFood 3y ago
    She was one real paper of note, and was one of a dozen authors on. Those authors include ppl with significantly greater reputation. https://scholar.google.com/citations?hl=en&user=6-e-ZBEAAAAJ https://scholar.google.com/citations?hl=en&user=6-e-ZBEAAAAJ
saberience 3y ago
What impact has she made? What has she built? Did she actually make AI safer? If so, how did she do it? Are current AI models safer because of her? It sounds like someone who talked a lot but didn’t actually create anything, change anything, or make anyone’s lives better. Happy to revise my perspective if someone can provide any concrete data on her impact.
- magpi3 3y ago
  > What impact has she made? What has she built? Did she actually make AI safer? She was on the board that fired Sam Altman, that had the courage to fire Sam Altman, presumably (according to the article at least) because he is a liar and she and three other board members did not trust him to be loyal to the non-profit's mission of working to keep AI safe.
  - WJW 3y ago
    That particular impact was undone in a single weekend. She may be an absolute rockstar otherwise but "firing Altman" is not impact when he is still in control of OpenAI.
  - PeterisP 3y ago
    The parent post question presumably was about the impact she has had as an academic or researcher (as "the leading academic on the issue of AI safety"), where the political actions on some board are not relevant.
- storf45 3y ago
  from the article, the paragraph right before it explained how she got on the board: "In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, where she continues to work."
  - PeterisP 3y ago
    It's interesting that on her LinkedIn profile she instead states that in Beijing she spent those 9 months in "intensive Mandarin Chinese language training" at a Chinese language program in Tsinghua University, not studying its AI ecosystem.
AlbertCory 3y ago
> she's a rock star in her field I don't think "rock star" is quite the right term. Unless you think David Geffen or Lester Bangs are rock stars. Normally rock stars sing or play an instrument. Someone who got a nice title in a nonprofit by writing about rock is not a rock star.
FuriouslyAdrift 3y ago
She's a lobbyist or policy wonk, not a researcher. Yann LeCun is a leading academic...
- jonstewart 3y ago
  Lobbyists and policy wonks are dramatically different vocations.
alvah 3y ago
She comes across as a childish idealist, typical of many uni graduates who've yet to encounter the real world, also typical of the EA cult. Giving people like her real power before they've had the uni nonsense knocked out of their heads is a ludicrous idea. I'm embarrassed she comes from my country, but not surprised about the particular uni she attended within my country.
- 0xDEAFBEAD 3y ago
  I have my issues with EA, but I don't think the "cult" description is accurate. For example, in 2022 they handed out $120K in prizes to the best criticisms of EA: https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/winners-of-the-ea-criticism-and-red-teaming-contest https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/... I'm doubtful that the "childish idealist" description is accurate. But even if it was, given the choice between having existentially dangerous AI developed by a childish idealist vs a slippery machiavellian like Sam, I would choose the childish idealist. The big risk with a childish idealist is that they become the pawn of a slippery machiavellian. Choosing the slippery machiavellian just seems like speedrunning failure.
  - alvah 3y ago
    You’re welcome to disagree but it’s not just me who thinks EA has cult-like attributes: https://www.google.com/search?q=is+ea+a+cult https://www.google.com/search?q=is+ea+a+cult I’m quite happy with the childish idealist description though. Putting childish idealists in charge of $billions of other people’s money might sound like a great idea to you, but would you put them in charge of your money?
    - 0xDEAFBEAD 3y ago
      "Cult" makes it sound like Scientology. Scientology doesn't run criticism contests of itself. As another datapoint, many of the most-upvoted posts on the EA Forum are EA critiques of one sort or another: https://forum.effectivealtruism.org/allPosts?timeframe=allTime&sortedBy=top&hideCommunity=false https://forum.effectivealtruism.org/allPosts?timeframe=allTi... Wikipedia says you have to apply to attend some EA events. That's true for lots of things. TED isn't open to just anyone either. If you want to go to an EA event without applying, most group events are pretty casual and don't require any application: https://forum.effectivealtruism.org/groups https://forum.effectivealtruism.org/groups (Note: I heard a rumor that EA events in Germany are less likely to be open to the public) >Putting childish idealists in charge of $billions of other people’s money might sound like a great idea to you, but would you put them in charge of your money? Well, I have in fact donated significant sums to some EA organizations, and I have a recurring donation to the Good Food Institute (working on meat alternatives). I don't agree with the "childish idealist" description of EA though. From my perspective, it's remarkable how easily Sam Altman seems to manipulate people, and he seems more successful at manipulating people outside of the EA community.
kj4211cash 3y ago
What are you talking about? She has no qualifications and her publications are juvenile. Her h-index is 8. She has no background in CS and has a few publication in Foreign Affairs. If she is the leading academic, then AI safety is not a serious academic field. She is in no way comparable to Ilya. This is the worst take I've seen on HN in awhile.
bogomipz 3y ago
>"She's the leading academic on the issue of AI safety." From article, this is what I know about her: >"Toner graduated from the University of Melbourne, Australia, in 2014 with a degree in chemical engineering and subsequently worked as a research analyst at a series of firms, including Open Philanthropy, a foundation that makes grants based on the effective-altruism philosophy." >"In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, where she continues to work." She's been working in the field of AI safety for what 3 years at most? I find it very hard to believe that she's "thee" leading academic on the issue of safety. Stuart Russell's group, the Center for Human Compatible AI at UC Berkeley, MIT's The Algorithmic Alignment Group and David Kruegger's group at Cambridge are all academic leaders and groups with great expertise in this subdomain. Further it's not even clear what "studying Beijing's AI ecosystem" actually means. Does this refer to the city? The university? The CCPs' surveillance state(not likely)? It's very hand wavy. I don't have any problem with what she did or her views. I also think it's a loss for the company. I do question her bona fides for being called the leading academic on the issue though.

edgyquant 3y ago

> the leading academic on the issue of AI safety. Sounds like a made up position to me. Just another bureaucrat with a cool sounding name

reducesuffering 3y ago
Why don't you reserve your ignorant judgement until you do due diligence? https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en https://scholar.google.com/citations?user=NNnQg0MAAAAJ&hl=en
- PeterisP 3y ago
  As an academic from a different field, this is not a publication record that would be anywhere close to someone being "the leading academic on the issue of AI safety". It indicates that she has had been one of very many participants at two impactful papers (with zero indication of what her individual contribution was, with so many authors it might as well be being part of the funding organization), has done a few collaborations with strong researchers, and that her personal first-author or sole-author research is barely relevant. Like, it's not a bad publication record, especially for a junior researcher (heck, her first participation in other people's papers is just in 2018 and her own work starts in 2021-2022, so she's effectively just getting started and may have many opportunities to prove herself in the future), it does indicate doing a couple years of research and might qualify for an okayish faculty job when (if?) she gets her PhD, but it's not also something indicating a senior researcher that's the pillar of anything and has had some impact - a grad student who's lucky to study in a strong leading researcher's lab and does some work on two of the advisor's papers might have a similar publication record already at their graduation before their career has even seriously started. Many of her co-authors have significantly larger research impact and personal research (https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ https://scholar.google.com/citations?hl=en&user=VclFrJ8AAAAJ or https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ https://scholar.google.com/citations?hl=en&user=MbBntPgAAAAJ for example or https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ https://scholar.google.com/citations?hl=en&user=0-G2eiEAAAAJ) - which also raises a question of how much each co-author did in a two-author paper where the leading author is a strong researcher and the second author is a "Director of Strategy and Foundational Research Grants" i.e. getting money for that research, so labeling her as a leading researcher (much less the leading researcher) of some field seems misleading as the publication record would indicate that she's not even on the same level as they are.
- MisterBastahrd 3y ago
  And what happens when they read the articles and come to realize that you could give an average college freshman any of those subjects and they could produce a comparable paper in a few weeks' time with a full course load and a part time job? Seriously, I've never seen a more underwhelming set of articles on google scholar from a supposed "leading" expert in a field... and what's worse is that she wasn't even the lead writer on most of them.
- WillPostForFood 3y ago
  She was one real paper of note, and was one of a dozen authors on. Those authors include ppl with significantly greater reputation. https://scholar.google.com/citations?hl=en&user=6-e-ZBEAAAAJ https://scholar.google.com/citations?hl=en&user=6-e-ZBEAAAAJ

saberience 3y ago

What impact has she made? What has she built? Did she actually make AI safer? If so, how did she do it? Are current AI models safer because of her? It sounds like someone who talked a lot but didn’t actually create anything, change anything, or make anyone’s lives better. Happy to revise my perspective if someone can provide any concrete data on her impact.

magpi3 3y ago
> What impact has she made? What has she built? Did she actually make AI safer? She was on the board that fired Sam Altman, that had the courage to fire Sam Altman, presumably (according to the article at least) because he is a liar and she and three other board members did not trust him to be loyal to the non-profit's mission of working to keep AI safe.
- WJW 3y ago
  That particular impact was undone in a single weekend. She may be an absolute rockstar otherwise but "firing Altman" is not impact when he is still in control of OpenAI.
- PeterisP 3y ago
  The parent post question presumably was about the impact she has had as an academic or researcher (as "the leading academic on the issue of AI safety"), where the political actions on some board are not relevant.
storf45 3y ago
from the article, the paragraph right before it explained how she got on the board: "In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, where she continues to work."
- PeterisP 3y ago
  It's interesting that on her LinkedIn profile she instead states that in Beijing she spent those 9 months in "intensive Mandarin Chinese language training" at a Chinese language program in Tsinghua University, not studying its AI ecosystem.

AlbertCory 3y ago

> she's a rock star in her field I don't think "rock star" is quite the right term. Unless you think David Geffen or Lester Bangs are rock stars. Normally rock stars sing or play an instrument. Someone who got a nice title in a nonprofit by writing about rock is not a rock star.

FuriouslyAdrift 3y ago

She's a lobbyist or policy wonk, not a researcher. Yann LeCun is a leading academic...

jonstewart 3y ago
Lobbyists and policy wonks are dramatically different vocations.

alvah 3y ago

She comes across as a childish idealist, typical of many uni graduates who've yet to encounter the real world, also typical of the EA cult. Giving people like her real power before they've had the uni nonsense knocked out of their heads is a ludicrous idea. I'm embarrassed she comes from my country, but not surprised about the particular uni she attended within my country.

0xDEAFBEAD 3y ago
I have my issues with EA, but I don't think the "cult" description is accurate. For example, in 2022 they handed out $120K in prizes to the best criticisms of EA: https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/winners-of-the-ea-criticism-and-red-teaming-contest https://forum.effectivealtruism.org/posts/YgbpxJmEdFhFGpqci/... I'm doubtful that the "childish idealist" description is accurate. But even if it was, given the choice between having existentially dangerous AI developed by a childish idealist vs a slippery machiavellian like Sam, I would choose the childish idealist. The big risk with a childish idealist is that they become the pawn of a slippery machiavellian. Choosing the slippery machiavellian just seems like speedrunning failure.
- alvah 3y ago
  You’re welcome to disagree but it’s not just me who thinks EA has cult-like attributes: https://www.google.com/search?q=is+ea+a+cult https://www.google.com/search?q=is+ea+a+cult I’m quite happy with the childish idealist description though. Putting childish idealists in charge of $billions of other people’s money might sound like a great idea to you, but would you put them in charge of your money?
  - 0xDEAFBEAD 3y ago
    "Cult" makes it sound like Scientology. Scientology doesn't run criticism contests of itself. As another datapoint, many of the most-upvoted posts on the EA Forum are EA critiques of one sort or another: https://forum.effectivealtruism.org/allPosts?timeframe=allTime&sortedBy=top&hideCommunity=false https://forum.effectivealtruism.org/allPosts?timeframe=allTi... Wikipedia says you have to apply to attend some EA events. That's true for lots of things. TED isn't open to just anyone either. If you want to go to an EA event without applying, most group events are pretty casual and don't require any application: https://forum.effectivealtruism.org/groups https://forum.effectivealtruism.org/groups (Note: I heard a rumor that EA events in Germany are less likely to be open to the public) >Putting childish idealists in charge of $billions of other people’s money might sound like a great idea to you, but would you put them in charge of your money? Well, I have in fact donated significant sums to some EA organizations, and I have a recurring donation to the Good Food Institute (working on meat alternatives). I don't agree with the "childish idealist" description of EA though. From my perspective, it's remarkable how easily Sam Altman seems to manipulate people, and he seems more successful at manipulating people outside of the EA community.

kj4211cash 3y ago

What are you talking about? She has no qualifications and her publications are juvenile. Her h-index is 8. She has no background in CS and has a few publication in Foreign Affairs. If she is the leading academic, then AI safety is not a serious academic field. She is in no way comparable to Ilya. This is the worst take I've seen on HN in awhile.

bogomipz 3y ago

>"She's the leading academic on the issue of AI safety." From article, this is what I know about her: >"Toner graduated from the University of Melbourne, Australia, in 2014 with a degree in chemical engineering and subsequently worked as a research analyst at a series of firms, including Open Philanthropy, a foundation that makes grants based on the effective-altruism philosophy." >"In 2019, she spent nine months in Beijing studying its AI ecosystem. When she returned, Toner helped establish a research organization at Georgetown University, called the Center for Security and Emerging Technology, where she continues to work." She's been working in the field of AI safety for what 3 years at most? I find it very hard to believe that she's "thee" leading academic on the issue of safety. Stuart Russell's group, the Center for Human Compatible AI at UC Berkeley, MIT's The Algorithmic Alignment Group and David Kruegger's group at Cambridge are all academic leaders and groups with great expertise in this subdomain. Further it's not even clear what "studying Beijing's AI ecosystem" actually means. Does this refer to the city? The university? The CCPs' surveillance state(not likely)? It's very hand wavy. I don't have any problem with what she did or her views. I also think it's a loss for the company. I do question her bona fides for being called the leading academic on the issue though.