25 ms·

Future of Humanity Institute shuts down

207 points by rdl 2y ago

mikece 2y ago
Possibly because the future of humanity itself is in question? "We're not going to make it, are we? Humans, I mean." --John Connor
- JohnKemeny 2y ago
  Or because Nick Bostrom is a hack. Together with Max Tegmark. Not to mention Ray Kurzweil. Initially wrote wrong surname.
  - patrulo 2y ago
    elaborate?
    - busterarm 2y ago
      academia is a vicious nest of vipers.
  - FabHK 2y ago
    You mean Max Tegmark? I mean, some of their ideas are way out there, and obviously if not tempered can lead to bad consequences (e.g. SBF/FTX). But a hack?
    - deleted 2y ago
      [deleted]
    - observationist 2y ago
      Laying SBF at the feet of effective altruism is more than a little silly. It'd be similar to associating the Democratic National Party with Bernie Madoff, or Epstein with any of the universities and foundations he donated to. One of the things criminals do to hide their activity, assuage their guilt, or put on a good show is associate with legitimate organizations. Social organizations typically don't and probably shouldn't do intrusive investigations into the lives and activities of their members sufficient to uncover their crimes. When such organizations learn of any benefits accrued through the illegal activities of members, there are obviously moral and sometimes legal requirements to disassociate, disavow, and return ill-gotten funds. Tegmark and Bostrom should be no means be tainted by association with EA. Their ideas and work on AI alignment and safety are excellent, and they ask the questions that should be answered as AI starts to reach human levels of competence and beyond.
      - johngossman 2y ago
        https://forum.effectivealtruism.org/posts/qFEwQbetaaSpvHm9e/my-takes-on-the-ftx-situation-will-mostly-be-cold-not-hot https://forum.effectivealtruism.org/posts/qFEwQbetaaSpvHm9e/... Not so silly that the EA community didn’t talk about what went wrong.
        observationist 2y ago
        EA tend to be a bunch of intellectual windbags, by and large, and overestimate their impact on the world at every level it's possible to do so. Just because they collectively gasped and claimed responsibility doesn't mean their interpretation has anything to do with reality. The reality is SBF was a con man, and however complex his motivations and personality and psychological issues, whatever his ultimate intent, he willfully scammed a lot of money from a lot of people. EA might have been an influence, but SBF is a human with complex agency whose actions can't and shouldn't be reduced to membership or association with a community. Going from "here's a set of good ideas about how to effectively give to charity, since we see a lot of corruption and inefficiency in charities" to "we have a moral duty to ensure that we only associate with good people" is how you go from a good idea to a pompous internet cult.
      - arduanika 2y ago
        You're correct that reputation laundering is a thing, but in the FTX case, the facts do not match that pattern. These were true believers.
  - helboi4 2y ago
    I really did not like Max Tegmark's AI book. It felt like bad science fiction.
  - esafak 2y ago
    I have not read any paper by Tegmark that seems hacky: https://arxiv.org/search/cs?searchtype=author&query=Tegmark,+M https://arxiv.org/search/cs?searchtype=author&query=Tegmark,... Could you point to some?
    - nybsjytm 2y ago
      Before turning to AI, Tegmark's claim to public fame was his "mathematical universe" hypothesis, which is complete drivel
      - frozenseven 2y ago
        >"mathematical universe" hypothesis, which is complete drivel How so? I would like to hear your argument.
  - goatlover 2y ago
    Would you also say Sean Carroll is a hack for his strong support of the MWI?
patrulo 2y ago
Curious as to why they didn't explore independence, moving from Oxford to a separate non-profit org. Getting funded doesn't seem impossible given the amount of high-profile e/acc advocates.
- rdl 2y ago
  I think they're the enemies of e/acc, and their EA background (due to FTX, SBF, etc.) taints them a bit when it comes to fundraising.
  - CorruptedArc 2y ago
    Not to mention the "big picture" EAs did damage when put in charge of the money FTX was doling out. They had people who'd been funding organizations that would feed, cloth, and medicate people move their funding to "existential problems" like AI and climate science. While I won't argue the value of those, vague theoreticals are arguably more commonly tacticaled than those on the ground trying to help with active issues.
  - cad31 2y ago
    By "enemies" you mean the sinking boat that e/acc people jumped out of, giving way for the next scam.
    - drngdds 2y ago
      No, these people and their ideology were around way way before e/acc was a thing
      - cad31 2y ago
        [dead]
- constantcrying 2y ago
  I think you are confusing EA and e/acc. EA was about billionaires defrauding the population who entrusted them with their money to give away to whatever cause was deemed best by the current "scientific" consensus in EA. Causes can include things like the suffering of fish or AI ending the known universe. e/acc is about doing capitalism so hard so that AI will solve every problem.
sinuhe69 2y ago
What an irony! The Bostrom Institute has to close in the midst of the spectacular rise of AI, and the all-important questions of AI risks and human value alignment are more hotly debated than ever. I hope that Bostrom and his colleagues will continue and deepen their research, because humanity needs such insights now more than ever.
- EarthAmbassador 2y ago
  How does one research and generate insights for what does not exist yet? Is there a framework? At SXSW this year, there were a couple of talks about forecasting, as a tool for futurism, but where ethics are concerned, I’ve not seen a good template. Maybe all if this is obvious, yet I’m curious so I’m asking.
  - skeeter2020 2y ago
    I'm not expert and struggled to read Bostrom's Super Intelligence, but my interpretation was it started with a lot of broad, hand-wavy factors & historical interpetations, layered on some pretty tenious projections as inevitable and wanked off with deep thought experiments. I get this sort of open-ended exploration has value, but I'm not sold on it's "prioritized value" when compared with other areas and directions.
    - christkv 2y ago
      It was literally under the department of intellectual wankery. Them shutting it down is the pot calling the kettle….
    - karma_pharmer 2y ago
      I just finished reading it. It's extremely repetitive; he could have written a book one-third of its length without leaving anything out. There's some good stuff in there, but I was sort of annoyed that the author forces you to read everything three times. Kinda disrespectful to his readers' available free time.
  - deleted 2y ago
ctxc 2y ago
On an unrelated note - my HN client UI breaks since the domain name is long xD
Maro 2y ago
This 2023 incident around founder Nick Bostrom may have something to do with it: https://en.wikipedia.org/wiki/Nick_Bostrom#1996_email_controversy https://en.wikipedia.org/wiki/Nick_Bostrom#1996_email_contro... From the wikipedia article: > In January 2023, Bostrom issued an apology for a 1996 email where he had stated that he thought "Blacks are more stupid than whites", and where he also used the word "niggers" in a description of how he thought this statement might be perceived by others. The apology, posted on his website, stated that "the invocation of a racial slur was repulsive" and that he "completely repudiate[d] this disgusting email". In his apology, he wrote “I think it is deeply unfair that unequal access to education, nutrients and basic healthcare leads to inequality in social outcomes, including sometimes disparities in skills and cognitive capacity.” Edit: also adding the second paragraph from Wikipedia to avoid accusations of smearing: > In January 2023, Oxford University told The Daily Beast, "The University and Faculty of Philosophy is currently investigating the matter but condemns in the strongest terms possible the views this particular academic expressed in his communications." In August 2023, the investigation concluded (according to a letter Bostrom posted on his website) that "we do not consider [Bostrom] to be a racist or that [he holds] racist views, and we consider that the apology [he] posted in January 2023 was sincere." Another member of this institute was effective altruist William MacAskill, who has appeared in media in 2023 in this context: "Sam Bankman-Fried and Will MacAskill weren't just philosophical allies. They were old friends." https://time.com/6262810/sam-bankman-fried-effective-altruism-alameda-ftx/ https://time.com/6262810/sam-bankman-fried-effective-altruis... From the OP: > In late 2023, the Faculty of Philosophy decided that the contracts of the remaining FHI staff would not be renewed. On 16 April 2024, the Institute was closed down. I'm just speculating, but it stands to reason that the Institute became a PR net negative for Oxford (or this specific Faculty).
- busterarm 2y ago
  FHI/Bostrom's relationship with the university was damaged for years before the apology email. They'd already frozen funding and hiring years earlier which means their issues go back even further than that.
- malfist 2y ago
  Wow, that's disgusting
optimalsolver 2y ago
EDIT: Got my “future of” institutes mixed up.
- rmbyrro 2y ago
  Or maybe some god scientist made an observation and the probability function collapsed in this non-funded state
- complianceowl 2y ago
  I'm literally laughing out loud at my desk right now XD
- dotsam 2y ago
  Tegmark’s institution is the Future of Life Institute, this is the Oxford Future of Humanity Institute. Tegmark’s institute is well-funded, apparently largely due to a big crypto donation from Vitalik Buterin. https://www.politico.com/news/2024/03/25/a-665m-crypto-war-chest-roils-ai-safety-fight-00148621 https://www.politico.com/news/2024/03/25/a-665m-crypto-war-c...
  - swyx 2y ago
    why do people not like Max Tegmark? he was kind of a rockstar at NeurIPS
    - gojomo 2y ago
      My read is that some former fans strongly disagree with, and are thus disappointed by, Tegmark's recent enthusiasm for "AI will kill us all" arguments, & advocacy of strong/intrusive policies against AI progress.
    - zehaeva 2y ago
      I like the guy, but I do think he's nutty over the level of multiverse he believes in.
    - VirusNewbie 2y ago
exo-pla-net 2y ago
Seems the incompetent and curiously hostile Philosophy Faculty at Oxford killed FHI. > While FHI had achieved significant academic and policy impact, the final years were affected by a gradual suffocation by Faculty bureaucracy. The flexible, fast-moving approach of the institute did not function well with the rigid rules and slow decision-making of the surrounding organization. (One of our administrators developed a joke measurement unit, “the Oxford”. 1 Oxford is the amount of work it takes to read and write 308 emails. This is the actual administrative effort it took for FHI to have a small grant disbursed into its account within the Philosophy Faculty so that we could start using it - after both the funder and the University had already approved the grant.) Starting in 2020, the Faculty imposed a freeze on fundraising and hiring. Unfortunately, this led to the eventual loss of lead researchers and especially the promising and diverse cohort of junior researchers, who have gone on to great things in the years since. While building an impressive alumni network and ecosystem of new nonprofits, these departures severely reduced the Institute. In late 2023, the Faculty of Philosophy announced that the contracts of the remaining FHI staff would not be renewed. On 16 April 2024, the Institute was closed down.
- goodcanadian 2y ago
  Curious use of the word faculty . . . I would say the problem is the administration and bureaucracy and not the faculty (which usually refers to the academic staff). I guess they mean the word in the sense of the administrative unit of the university. Regardless, this is not a problem confined to Oxford; it seems to have proliferated throughout academia in the last couple of decades. The sheer amount of utter bullshit is mind boggling; I figure about 1 in 10 people actually do useful work while the other 9 conspire to make that person's life more difficult. Of course, industry is hardly any better . . .
  - hollerith 2y ago
    Why not assume that the OP knows the definition of the word "faculty" and that when the OP writes, "suffocation by Faculty bureaucracy", he meant suffocation by academic staff?
johngossman 2y ago
Bear with a digression. CS Lewis was a professor at Oxford. His novel “That Hideous Strength” is about an organization that wants to use science to save humanity. “It’s a little fantastic to base one’s actions on a supposed concern for what’s going to happen millions of years hence; and you must remember that the other side would claim to be preserving humanity, too.” — That Hideous Strength: (Space Trilogy, Book Three) (The Space Trilogy 3) by C. S. Lewis Of course, it all goes very, very bad. The whole book can be read as a warning against what we now call transhumanism, obviously from a Christian perspective. Given CS Lewis’s Oxford connection, I have always wondered if some of the faculty had their doubts about the FHI
- n4r9 2y ago
  I doubt that the philosophy faculty would take CS Lewis' opinion seriously into consideration. He was a decent storyteller, but not much of a philosopher.
  - busterarm 2y ago
    His work is often quietly cited by a handful of moral philosophers. In particular, The Discarded Image, anytime someone is writing about medieval philosophy. Anyway, being a philosopher simply isn't what the guy did and academics are fairly dismissive without consideration anytime religion (and specifically Christian apologetics) gets in the mix.
    - n4r9 2y ago
      I'd interested to take a look if you have any further info about which moral philosophers and which works cite The Discarded Image? I admit I haven't read that one myself; my own take on him as a philosopher stems from reading Mere Christianity and summaries of Surprised by Joy, and not being aware of any references to his work by more recent philosophers that I've been interested in such as Chomsky, Zizek, or Dennett. I got a strong feeling that Lewis' arguments and exposition were guided by something other than logic, though they pretended to be following it. The trilemma is an example of this.
stainablesteel 2y ago
i'm honestly glad, there seems to be a lot of places that may as well be called "the institute of impending doom for all of humanity" and i just don't care for this fear-based grant entrenchment, it gets us nowhere. people who are pioneering the front lines of any technology understand risk better than people who write about it
- mitthrowaway2 2y ago
  > people who are pioneering the front lines of any technology understand risk better than people who write about it That's far from guaranteed, and we have a long history of lessons written in blood that says otherwise. The people pioneering a technology are going to be self-selected to be the most optimistic about its potential and the most dismissive of its negative impacts.
  - stainablesteel 2y ago
    this isn't chemistry in the 1800s this is computer science
    - mitthrowaway2 2y ago
      What does that matter?
neilv 2y ago
The HN headline, "Future of Humanity Institute shuts down", sounds cynically funny, in context of current problems/challenges. The grandiose name Oxford chose now sounds like they've determined the situation is hopeless.
- jessriedel 2y ago
  "Oxford" did not choose the name
  - kikokikokiko 2y ago
    By your definition no name ever was "chosen" by any corporation in history. Organizations are made of people, and when a new org inside the main org is named, it obviously was named by someone (or a committee). In the end, the "organization" choose the name, potato po-ta-to.
    - jessriedel 2y ago
      You misunderstand. There are many cases where there is a decision making process (either single person or collective) that reasonably represents an organization's decision. But no such organization representing Oxford picked the name of FHI. It was picked by the people, most likely Bostrom, who started FHI, and did not at all represent a decision by greater Oxford.
- colechristensen 2y ago
  > in context of current problems/challenges People have been saying nonsense like this for as long as we have recorded history of people saying anything.
  - stronglikedan 2y ago
    Nothing new under the sun --Abraham Lincoln
    - 2y ago
PeterStuer 2y ago
Seems at first glance to be the same type slippery eel org as the 'Future of Life Institute'. They seem to have far more in common under the covers than just similar sounding names.
Aromasin 2y ago
The full final report is well worth a read for those with time on their hands: https://www.futureofhumanityinstitute.org/s/FHI-Final-Report.pdf https://www.futureofhumanityinstitute.org/s/FHI-Final-Report...
- lifeinthevoid 2y ago
  [flagged]
  - deleted 2y ago
    [deleted]
  - swyx 2y ago
    Presentation of FHI: - Founded in 2005 by Prof Nick Bostrom as a multidisciplinary research group at Oxford to study big-picture questions for human civilization - Aimed to shield researchers from ordinary academic pressures and foster creativity and intellectual progress History: Early days (2005-2010): - Initial funding from James Martin and the Oxford Martin School - Focus on human enhancement ethics, global catastrophic and existential risks, methodology for thinking about uncertain futures - Hosted influential conferences and workshops that helped build academic communities around key research areas Maturation (2010-2020): - Artificial intelligence, especially AI safety and governance, became a major focus - Expanded into other areas like biosafety, priorities research, population ethics - Received major grants allowing expansion of research - Increasing policy impact and advising governments Final Years (2020-2024): - Continued work through the COVID-19 pandemic after moving to a new building - Some new research directions like digital minds and grand futures - But also increasing bureaucratic obstacles from the Philosophy Faculty - FHI was closed down in April 2024 when the University declined to renew staff contracts Research Topics and Findings: - Existential risk - pioneered the study of risks that threaten humanity's long-term potential - Biological risk - modeled risks from emerging biotechnologies and pandemics - Macrostrategy - studying how long-term outcomes for humanity connect to present-day actions - Longtermism - the idea that positively influencing the long-term future is a key moral priority - Grand futures - exploring the limits of what spacefaring civilizations could accomplish - SETI/Fermi paradox - dissolving the paradox; grabby aliens hypothesis - Effective altruism - identifying the highest-impact ways to improve the world - Technology: - AI safety, alignment and governance - Whole brain emulation - Digital minds and AI consciousness - Human enhancement ethics - Epistemology and rationality - anthropic reasoning, information hazards, moral uncertainty - Ethics - challenges in infinite ethics, Parliamentary model for normative uncertainty Concepts originated at FHI that are now influential: existential risk, astronomical waste, information hazards, differential technological development, crucial considerations, exploratory engineering, whole brain emulation Learnings and Advice: - Take the long-term view; build up new fields even if not currently fashionable - Have a diverse team from many disciplines - Invest in the right organizational relationships to maintain stability - As an organization scales, its structure needs to evolve - The key to replicating FHI is having the right people and intellectual culture focused on the most important questions
arduanika 2y ago
Good riddance to these ideologues. It's a shame that the announcement shows no trace of remorse for the crimes they inspired (e.g. at FTX) or the lives that have been ruined by the cults to which they lent academic legitimacy. Instead, their demise is chalked up to some bland moaning about "administrative headwinds". Realistically, we're probably only looking at a brief respite before they regroup under some other benevolent-sounding name. Be on the lookout for their next incarnation, and let's hope that Oxford won't repeat the mistake of allowing them to evangelize under the seal of a great university. (Edit in response to causal's question below): They legitimized longtermism and pseudo-rational AI panic, which transformed much of EA into an apocalyptic sect, with all the high-demand group dynamics that come with it. Their research created an air of urgency and expediency which, among other bad outcomes, inspired the devoted EAs at FTX to justify their crimes to themselves. These crimes resulted in privation and a few suicides for their innocent depositors. To this day, there are EAs who rationalize these crimes against their outgroup as not that bad, considering that the "Future of Humanity" is at stake. You can look at the movement and find plenty of other negative effects of this urgency and expediency, but for me personally, the FTX crimes are what woke me up to the true nature of these hazardous ideas.
- causal 2y ago
  That's a lot of vague vitriol, what exactly did FHI do? Genuinely curious.
  - deleted 2y ago
    [deleted]
- rurp 2y ago
  It's quite a leap to assume that SBF only turned to a life of crime because the EA movement lured him into it. There are legitimate criticisms of EA and I'm not personally a follower of the movement but your post comes off as way too generally dismissive. Many of the people involved seem to have good intentions and a lot of money has been donated to objectively good causes. The longtermism stuff is more squishy, but our society is pretty bad at dealing with certain types of existential risks and we could use more people thinking about solutions rather than less.
ConanRus 2y ago
[dead]
kvee 2y ago
Future of Humanity Final Report from Anders Sandberg here: https://static1.squarespace.com/static/660e95991cf0293c2463bcc8/t/661a3fc3cecceb2b8ffce80d/1712996303164/FHI+Final+Report.pdf https://static1.squarespace.com/static/660e95991cf0293c2463b... and Google Doc version: https://docs.google.com/document/d/1jgl2KqtiJ6lLkpoZ1I_VeniPwWZ-8b1fXYJCspXOddc/edit https://docs.google.com/document/d/1jgl2KqtiJ6lLkpoZ1I_VeniP...
- deleted 2y ago
  [deleted]
- zheninghuang 2y ago
  I might be very biased: - This report looks like it was put together in a week's time. - As an academic, I am sure the quality of the presentation is lower than that of 90% of the academic papers out there.
gizajob 2y ago
Seems like they didn’t see it coming…
- bakchodi 2y ago
  [dead]
- Vecr 2y ago
  They did, I read the report. Probably since 2020 or 2021 at the latest.
karma_pharmer 2y ago
Starting in 2020, the Faculty imposed a freeze on fundraising and hiring. In late 2023, the Faculty of Philosophy decided that the contracts of the remaining FHI staff would not be renewed. Can anybody offer insight into the reasons here? Obviously there is not going to be an objective answer to this question. And that is probably why the linked page does not try to give an answer. I'm assuming lack of funding wasn't the problem, since they froze fundraising (if the inability to raise funds was the problem it is unlikely that they would have done this). Edit: looks like he got smeared in January of 2023, but that doesn't explain the freeze starting three years earlier: https://news.ycombinator.com/item?id=40066352 https://news.ycombinator.com/item?id=40066352
- williamtrask 2y ago
  "Any organization embedded in a larger organization or community needs to invest to a certain degree in establishing the right kind of social relationships to maintain this embeddedness. Incentives must be aligned, and both parties must also recognize this alignment. We did not invest enough in university politics and sociality to form a long-term stable relationship with our faculty. There also needs to be an understanding of how to communicate across organizational communities. When epistemic and communicative practices diverge too much, misunderstandings proliferate. Several times we made serious missteps in our communications with other parts of the university because we misunderstood how the message would be received. Finding friendly local translators and bridgebuilders is important." https://static1.squarespace.com/static/660e95991cf0293c2463bcc8/t/661a3fc3cecceb2b8ffce80d/1712996303164/FHI+Final+Report.pdf https://static1.squarespace.com/static/660e95991cf0293c2463b...
carlottaludwig 2y ago
[dead]
javajosh 2y ago
The real question is whether Mr. Beast can make compelling content about saving the Future of Humanity Institute. If so, they're good.
- klyrs 2y ago
  I saw some Mr Beast branded candy bars at the supermarket this week. Pretty sure that's not a future-saving venture, that's just taking advantage of the sugar-addicted kids that form his fanbase.
andrelaszlo 2y ago
The future is here?
Lockal 2y ago
Good riddance. A group of scammers without specialized education, theorizing about immortality (from people without medical education), saving humanity (thank you, but humanity thrives without your help), about ethics in artificial intelligence (from people who do not know any programming language), about drug promotion (from professional drug abusers), about pumping money into cryptocurrencies (which they personally purchased) and even crypto exchanges they own (hello SBF). I hope to see all these people in jail (as I see they have already removed their names from the site).
- delton137 2y ago
  I don't think FHI received any funding from SBF's Future Fund. (maybe a few of the visiting staff did, though)
- arduanika 2y ago
  I hate to break it to you, but these scholars are not going to jail for their bad ideas. That's not how a free society works. We'll have to settle for general disrepute.
deleted 2y ago
[deleted]
nineteen999 2y ago
> attempting to shield them from ordinary academic pressures oh the poor darlings
ronnieadkins08 2y ago
[dead]

207 points by rdl 2y ago

Possibly because the future of humanity itself is in question? "We're not going to make it, are we? Humans, I mean." --John Connor

JohnKemeny 2y ago
Or because Nick Bostrom is a hack. Together with Max Tegmark. Not to mention Ray Kurzweil. Initially wrote wrong surname.
- patrulo 2y ago
  elaborate?
  - busterarm 2y ago
    academia is a vicious nest of vipers.
- FabHK 2y ago
  You mean Max Tegmark? I mean, some of their ideas are way out there, and obviously if not tempered can lead to bad consequences (e.g. SBF/FTX). But a hack?
  - deleted 2y ago
    [deleted]
  - observationist 2y ago
    Laying SBF at the feet of effective altruism is more than a little silly. It'd be similar to associating the Democratic National Party with Bernie Madoff, or Epstein with any of the universities and foundations he donated to. One of the things criminals do to hide their activity, assuage their guilt, or put on a good show is associate with legitimate organizations. Social organizations typically don't and probably shouldn't do intrusive investigations into the lives and activities of their members sufficient to uncover their crimes. When such organizations learn of any benefits accrued through the illegal activities of members, there are obviously moral and sometimes legal requirements to disassociate, disavow, and return ill-gotten funds. Tegmark and Bostrom should be no means be tainted by association with EA. Their ideas and work on AI alignment and safety are excellent, and they ask the questions that should be answered as AI starts to reach human levels of competence and beyond.
    - johngossman 2y ago
      https://forum.effectivealtruism.org/posts/qFEwQbetaaSpvHm9e/my-takes-on-the-ftx-situation-will-mostly-be-cold-not-hot https://forum.effectivealtruism.org/posts/qFEwQbetaaSpvHm9e/... Not so silly that the EA community didn’t talk about what went wrong.
      - observationist 2y ago
        EA tend to be a bunch of intellectual windbags, by and large, and overestimate their impact on the world at every level it's possible to do so. Just because they collectively gasped and claimed responsibility doesn't mean their interpretation has anything to do with reality. The reality is SBF was a con man, and however complex his motivations and personality and psychological issues, whatever his ultimate intent, he willfully scammed a lot of money from a lot of people. EA might have been an influence, but SBF is a human with complex agency whose actions can't and shouldn't be reduced to membership or association with a community. Going from "here's a set of good ideas about how to effectively give to charity, since we see a lot of corruption and inefficiency in charities" to "we have a moral duty to ensure that we only associate with good people" is how you go from a good idea to a pompous internet cult.
    - arduanika 2y ago
      You're correct that reputation laundering is a thing, but in the FTX case, the facts do not match that pattern. These were true believers.
- helboi4 2y ago
  I really did not like Max Tegmark's AI book. It felt like bad science fiction.
- esafak 2y ago
  I have not read any paper by Tegmark that seems hacky: https://arxiv.org/search/cs?searchtype=author&query=Tegmark,+M https://arxiv.org/search/cs?searchtype=author&query=Tegmark,... Could you point to some?
  - nybsjytm 2y ago
    Before turning to AI, Tegmark's claim to public fame was his "mathematical universe" hypothesis, which is complete drivel
    - frozenseven 2y ago
      >"mathematical universe" hypothesis, which is complete drivel How so? I would like to hear your argument.
- goatlover 2y ago
  Would you also say Sean Carroll is a hack for his strong support of the MWI?

patrulo 2y ago

Curious as to why they didn't explore independence, moving from Oxford to a separate non-profit org. Getting funded doesn't seem impossible given the amount of high-profile e/acc advocates.

rdl 2y ago
I think they're the enemies of e/acc, and their EA background (due to FTX, SBF, etc.) taints them a bit when it comes to fundraising.
- CorruptedArc 2y ago
  Not to mention the "big picture" EAs did damage when put in charge of the money FTX was doling out. They had people who'd been funding organizations that would feed, cloth, and medicate people move their funding to "existential problems" like AI and climate science. While I won't argue the value of those, vague theoreticals are arguably more commonly tacticaled than those on the ground trying to help with active issues.
- cad31 2y ago
  By "enemies" you mean the sinking boat that e/acc people jumped out of, giving way for the next scam.
  - drngdds 2y ago
    No, these people and their ideology were around way way before e/acc was a thing
    - cad31 2y ago
      [dead]
constantcrying 2y ago
I think you are confusing EA and e/acc. EA was about billionaires defrauding the population who entrusted them with their money to give away to whatever cause was deemed best by the current "scientific" consensus in EA. Causes can include things like the suffering of fish or AI ending the known universe. e/acc is about doing capitalism so hard so that AI will solve every problem.

sinuhe69 2y ago

What an irony! The Bostrom Institute has to close in the midst of the spectacular rise of AI, and the all-important questions of AI risks and human value alignment are more hotly debated than ever. I hope that Bostrom and his colleagues will continue and deepen their research, because humanity needs such insights now more than ever.

EarthAmbassador 2y ago
How does one research and generate insights for what does not exist yet? Is there a framework? At SXSW this year, there were a couple of talks about forecasting, as a tool for futurism, but where ethics are concerned, I’ve not seen a good template. Maybe all if this is obvious, yet I’m curious so I’m asking.
- skeeter2020 2y ago
  I'm not expert and struggled to read Bostrom's Super Intelligence, but my interpretation was it started with a lot of broad, hand-wavy factors & historical interpetations, layered on some pretty tenious projections as inevitable and wanked off with deep thought experiments. I get this sort of open-ended exploration has value, but I'm not sold on it's "prioritized value" when compared with other areas and directions.
  - christkv 2y ago
    It was literally under the department of intellectual wankery. Them shutting it down is the pot calling the kettle….
  - karma_pharmer 2y ago
    I just finished reading it. It's extremely repetitive; he could have written a book one-third of its length without leaving anything out. There's some good stuff in there, but I was sort of annoyed that the author forces you to read everything three times. Kinda disrespectful to his readers' available free time.
- deleted 2y ago

ctxc 2y ago

On an unrelated note - my HN client UI breaks since the domain name is long xD

Maro 2y ago

This 2023 incident around founder Nick Bostrom may have something to do with it: https://en.wikipedia.org/wiki/Nick_Bostrom#1996_email_controversy https://en.wikipedia.org/wiki/Nick_Bostrom#1996_email_contro... From the wikipedia article: > In January 2023, Bostrom issued an apology for a 1996 email where he had stated that he thought "Blacks are more stupid than whites", and where he also used the word "niggers" in a description of how he thought this statement might be perceived by others. The apology, posted on his website, stated that "the invocation of a racial slur was repulsive" and that he "completely repudiate[d] this disgusting email". In his apology, he wrote “I think it is deeply unfair that unequal access to education, nutrients and basic healthcare leads to inequality in social outcomes, including sometimes disparities in skills and cognitive capacity.” Edit: also adding the second paragraph from Wikipedia to avoid accusations of smearing: > In January 2023, Oxford University told The Daily Beast, "The University and Faculty of Philosophy is currently investigating the matter but condemns in the strongest terms possible the views this particular academic expressed in his communications." In August 2023, the investigation concluded (according to a letter Bostrom posted on his website) that "we do not consider [Bostrom] to be a racist or that [he holds] racist views, and we consider that the apology [he] posted in January 2023 was sincere." Another member of this institute was effective altruist William MacAskill, who has appeared in media in 2023 in this context: "Sam Bankman-Fried and Will MacAskill weren't just philosophical allies. They were old friends." https://time.com/6262810/sam-bankman-fried-effective-altruism-alameda-ftx/ https://time.com/6262810/sam-bankman-fried-effective-altruis... From the OP: > In late 2023, the Faculty of Philosophy decided that the contracts of the remaining FHI staff would not be renewed. On 16 April 2024, the Institute was closed down. I'm just speculating, but it stands to reason that the Institute became a PR net negative for Oxford (or this specific Faculty).

busterarm 2y ago
FHI/Bostrom's relationship with the university was damaged for years before the apology email. They'd already frozen funding and hiring years earlier which means their issues go back even further than that.
malfist 2y ago
Wow, that's disgusting

optimalsolver 2y ago

EDIT: Got my “future of” institutes mixed up.

rmbyrro 2y ago
Or maybe some god scientist made an observation and the probability function collapsed in this non-funded state
complianceowl 2y ago
I'm literally laughing out loud at my desk right now XD
dotsam 2y ago
Tegmark’s institution is the Future of Life Institute, this is the Oxford Future of Humanity Institute. Tegmark’s institute is well-funded, apparently largely due to a big crypto donation from Vitalik Buterin. https://www.politico.com/news/2024/03/25/a-665m-crypto-war-chest-roils-ai-safety-fight-00148621 https://www.politico.com/news/2024/03/25/a-665m-crypto-war-c...
- swyx 2y ago
  why do people not like Max Tegmark? he was kind of a rockstar at NeurIPS
  - gojomo 2y ago
    My read is that some former fans strongly disagree with, and are thus disappointed by, Tegmark's recent enthusiasm for "AI will kill us all" arguments, & advocacy of strong/intrusive policies against AI progress.
  - zehaeva 2y ago
    I like the guy, but I do think he's nutty over the level of multiverse he believes in.
  - VirusNewbie 2y ago

exo-pla-net 2y ago

Seems the incompetent and curiously hostile Philosophy Faculty at Oxford killed FHI. > While FHI had achieved significant academic and policy impact, the final years were affected by a gradual suffocation by Faculty bureaucracy. The flexible, fast-moving approach of the institute did not function well with the rigid rules and slow decision-making of the surrounding organization. (One of our administrators developed a joke measurement unit, “the Oxford”. 1 Oxford is the amount of work it takes to read and write 308 emails. This is the actual administrative effort it took for FHI to have a small grant disbursed into its account within the Philosophy Faculty so that we could start using it - after both the funder and the University had already approved the grant.) Starting in 2020, the Faculty imposed a freeze on fundraising and hiring. Unfortunately, this led to the eventual loss of lead researchers and especially the promising and diverse cohort of junior researchers, who have gone on to great things in the years since. While building an impressive alumni network and ecosystem of new nonprofits, these departures severely reduced the Institute. In late 2023, the Faculty of Philosophy announced that the contracts of the remaining FHI staff would not be renewed. On 16 April 2024, the Institute was closed down.

goodcanadian 2y ago
Curious use of the word faculty . . . I would say the problem is the administration and bureaucracy and not the faculty (which usually refers to the academic staff). I guess they mean the word in the sense of the administrative unit of the university. Regardless, this is not a problem confined to Oxford; it seems to have proliferated throughout academia in the last couple of decades. The sheer amount of utter bullshit is mind boggling; I figure about 1 in 10 people actually do useful work while the other 9 conspire to make that person's life more difficult. Of course, industry is hardly any better . . .
- hollerith 2y ago
  Why not assume that the OP knows the definition of the word "faculty" and that when the OP writes, "suffocation by Faculty bureaucracy", he meant suffocation by academic staff?

johngossman 2y ago

Bear with a digression. CS Lewis was a professor at Oxford. His novel “That Hideous Strength” is about an organization that wants to use science to save humanity. “It’s a little fantastic to base one’s actions on a supposed concern for what’s going to happen millions of years hence; and you must remember that the other side would claim to be preserving humanity, too.” — That Hideous Strength: (Space Trilogy, Book Three) (The Space Trilogy 3) by C. S. Lewis Of course, it all goes very, very bad. The whole book can be read as a warning against what we now call transhumanism, obviously from a Christian perspective. Given CS Lewis’s Oxford connection, I have always wondered if some of the faculty had their doubts about the FHI

n4r9 2y ago
I doubt that the philosophy faculty would take CS Lewis' opinion seriously into consideration. He was a decent storyteller, but not much of a philosopher.
- busterarm 2y ago
  His work is often quietly cited by a handful of moral philosophers. In particular, The Discarded Image, anytime someone is writing about medieval philosophy. Anyway, being a philosopher simply isn't what the guy did and academics are fairly dismissive without consideration anytime religion (and specifically Christian apologetics) gets in the mix.
  - n4r9 2y ago
    I'd interested to take a look if you have any further info about which moral philosophers and which works cite The Discarded Image? I admit I haven't read that one myself; my own take on him as a philosopher stems from reading Mere Christianity and summaries of Surprised by Joy, and not being aware of any references to his work by more recent philosophers that I've been interested in such as Chomsky, Zizek, or Dennett. I got a strong feeling that Lewis' arguments and exposition were guided by something other than logic, though they pretended to be following it. The trilemma is an example of this.

stainablesteel 2y ago

i'm honestly glad, there seems to be a lot of places that may as well be called "the institute of impending doom for all of humanity" and i just don't care for this fear-based grant entrenchment, it gets us nowhere. people who are pioneering the front lines of any technology understand risk better than people who write about it

mitthrowaway2 2y ago
> people who are pioneering the front lines of any technology understand risk better than people who write about it That's far from guaranteed, and we have a long history of lessons written in blood that says otherwise. The people pioneering a technology are going to be self-selected to be the most optimistic about its potential and the most dismissive of its negative impacts.
- stainablesteel 2y ago
  this isn't chemistry in the 1800s this is computer science
  - mitthrowaway2 2y ago
    What does that matter?

neilv 2y ago

The HN headline, "Future of Humanity Institute shuts down", sounds cynically funny, in context of current problems/challenges. The grandiose name Oxford chose now sounds like they've determined the situation is hopeless.

jessriedel 2y ago
"Oxford" did not choose the name
- kikokikokiko 2y ago
  By your definition no name ever was "chosen" by any corporation in history. Organizations are made of people, and when a new org inside the main org is named, it obviously was named by someone (or a committee). In the end, the "organization" choose the name, potato po-ta-to.
  - jessriedel 2y ago
    You misunderstand. There are many cases where there is a decision making process (either single person or collective) that reasonably represents an organization's decision. But no such organization representing Oxford picked the name of FHI. It was picked by the people, most likely Bostrom, who started FHI, and did not at all represent a decision by greater Oxford.
colechristensen 2y ago
> in context of current problems/challenges People have been saying nonsense like this for as long as we have recorded history of people saying anything.
- stronglikedan 2y ago
  Nothing new under the sun --Abraham Lincoln
  - 2y ago

PeterStuer 2y ago

Seems at first glance to be the same type slippery eel org as the 'Future of Life Institute'. They seem to have far more in common under the covers than just similar sounding names.

Aromasin 2y ago

The full final report is well worth a read for those with time on their hands: https://www.futureofhumanityinstitute.org/s/FHI-Final-Report.pdf https://www.futureofhumanityinstitute.org/s/FHI-Final-Report...

lifeinthevoid 2y ago
[flagged]
- deleted 2y ago
  [deleted]
- swyx 2y ago
  Presentation of FHI: - Founded in 2005 by Prof Nick Bostrom as a multidisciplinary research group at Oxford to study big-picture questions for human civilization - Aimed to shield researchers from ordinary academic pressures and foster creativity and intellectual progress History: Early days (2005-2010): - Initial funding from James Martin and the Oxford Martin School - Focus on human enhancement ethics, global catastrophic and existential risks, methodology for thinking about uncertain futures - Hosted influential conferences and workshops that helped build academic communities around key research areas Maturation (2010-2020): - Artificial intelligence, especially AI safety and governance, became a major focus - Expanded into other areas like biosafety, priorities research, population ethics - Received major grants allowing expansion of research - Increasing policy impact and advising governments Final Years (2020-2024): - Continued work through the COVID-19 pandemic after moving to a new building - Some new research directions like digital minds and grand futures - But also increasing bureaucratic obstacles from the Philosophy Faculty - FHI was closed down in April 2024 when the University declined to renew staff contracts Research Topics and Findings: - Existential risk - pioneered the study of risks that threaten humanity's long-term potential - Biological risk - modeled risks from emerging biotechnologies and pandemics - Macrostrategy - studying how long-term outcomes for humanity connect to present-day actions - Longtermism - the idea that positively influencing the long-term future is a key moral priority - Grand futures - exploring the limits of what spacefaring civilizations could accomplish - SETI/Fermi paradox - dissolving the paradox; grabby aliens hypothesis - Effective altruism - identifying the highest-impact ways to improve the world - Technology: - AI safety, alignment and governance - Whole brain emulation - Digital minds and AI consciousness - Human enhancement ethics - Epistemology and rationality - anthropic reasoning, information hazards, moral uncertainty - Ethics - challenges in infinite ethics, Parliamentary model for normative uncertainty Concepts originated at FHI that are now influential: existential risk, astronomical waste, information hazards, differential technological development, crucial considerations, exploratory engineering, whole brain emulation Learnings and Advice: - Take the long-term view; build up new fields even if not currently fashionable - Have a diverse team from many disciplines - Invest in the right organizational relationships to maintain stability - As an organization scales, its structure needs to evolve - The key to replicating FHI is having the right people and intellectual culture focused on the most important questions

arduanika 2y ago

Good riddance to these ideologues. It's a shame that the announcement shows no trace of remorse for the crimes they inspired (e.g. at FTX) or the lives that have been ruined by the cults to which they lent academic legitimacy. Instead, their demise is chalked up to some bland moaning about "administrative headwinds". Realistically, we're probably only looking at a brief respite before they regroup under some other benevolent-sounding name. Be on the lookout for their next incarnation, and let's hope that Oxford won't repeat the mistake of allowing them to evangelize under the seal of a great university. (Edit in response to causal's question below): They legitimized longtermism and pseudo-rational AI panic, which transformed much of EA into an apocalyptic sect, with all the high-demand group dynamics that come with it. Their research created an air of urgency and expediency which, among other bad outcomes, inspired the devoted EAs at FTX to justify their crimes to themselves. These crimes resulted in privation and a few suicides for their innocent depositors. To this day, there are EAs who rationalize these crimes against their outgroup as not that bad, considering that the "Future of Humanity" is at stake. You can look at the movement and find plenty of other negative effects of this urgency and expediency, but for me personally, the FTX crimes are what woke me up to the true nature of these hazardous ideas.

causal 2y ago
That's a lot of vague vitriol, what exactly did FHI do? Genuinely curious.
- deleted 2y ago
  [deleted]
rurp 2y ago
It's quite a leap to assume that SBF only turned to a life of crime because the EA movement lured him into it. There are legitimate criticisms of EA and I'm not personally a follower of the movement but your post comes off as way too generally dismissive. Many of the people involved seem to have good intentions and a lot of money has been donated to objectively good causes. The longtermism stuff is more squishy, but our society is pretty bad at dealing with certain types of existential risks and we could use more people thinking about solutions rather than less.

ConanRus 2y ago

[dead]

kvee 2y ago

Future of Humanity Final Report from Anders Sandberg here: https://static1.squarespace.com/static/660e95991cf0293c2463bcc8/t/661a3fc3cecceb2b8ffce80d/1712996303164/FHI+Final+Report.pdf https://static1.squarespace.com/static/660e95991cf0293c2463b... and Google Doc version: https://docs.google.com/document/d/1jgl2KqtiJ6lLkpoZ1I_VeniPwWZ-8b1fXYJCspXOddc/edit https://docs.google.com/document/d/1jgl2KqtiJ6lLkpoZ1I_VeniP...

deleted 2y ago
[deleted]
zheninghuang 2y ago
I might be very biased: - This report looks like it was put together in a week's time. - As an academic, I am sure the quality of the presentation is lower than that of 90% of the academic papers out there.

gizajob 2y ago

Seems like they didn’t see it coming…

bakchodi 2y ago
[dead]
Vecr 2y ago
They did, I read the report. Probably since 2020 or 2021 at the latest.

karma_pharmer 2y ago

Starting in 2020, the Faculty imposed a freeze on fundraising and hiring. In late 2023, the Faculty of Philosophy decided that the contracts of the remaining FHI staff would not be renewed. Can anybody offer insight into the reasons here? Obviously there is not going to be an objective answer to this question. And that is probably why the linked page does not try to give an answer. I'm assuming lack of funding wasn't the problem, since they froze fundraising (if the inability to raise funds was the problem it is unlikely that they would have done this). Edit: looks like he got smeared in January of 2023, but that doesn't explain the freeze starting three years earlier: https://news.ycombinator.com/item?id=40066352 https://news.ycombinator.com/item?id=40066352

williamtrask 2y ago
"Any organization embedded in a larger organization or community needs to invest to a certain degree in establishing the right kind of social relationships to maintain this embeddedness. Incentives must be aligned, and both parties must also recognize this alignment. We did not invest enough in university politics and sociality to form a long-term stable relationship with our faculty. There also needs to be an understanding of how to communicate across organizational communities. When epistemic and communicative practices diverge too much, misunderstandings proliferate. Several times we made serious missteps in our communications with other parts of the university because we misunderstood how the message would be received. Finding friendly local translators and bridgebuilders is important." https://static1.squarespace.com/static/660e95991cf0293c2463bcc8/t/661a3fc3cecceb2b8ffce80d/1712996303164/FHI+Final+Report.pdf https://static1.squarespace.com/static/660e95991cf0293c2463b...

carlottaludwig 2y ago

[dead]

javajosh 2y ago

The real question is whether Mr. Beast can make compelling content about saving the Future of Humanity Institute. If so, they're good.

klyrs 2y ago
I saw some Mr Beast branded candy bars at the supermarket this week. Pretty sure that's not a future-saving venture, that's just taking advantage of the sugar-addicted kids that form his fanbase.

andrelaszlo 2y ago

The future is here?

Lockal 2y ago

Good riddance. A group of scammers without specialized education, theorizing about immortality (from people without medical education), saving humanity (thank you, but humanity thrives without your help), about ethics in artificial intelligence (from people who do not know any programming language), about drug promotion (from professional drug abusers), about pumping money into cryptocurrencies (which they personally purchased) and even crypto exchanges they own (hello SBF). I hope to see all these people in jail (as I see they have already removed their names from the site).

delton137 2y ago
I don't think FHI received any funding from SBF's Future Fund. (maybe a few of the visiting staff did, though)
arduanika 2y ago
I hate to break it to you, but these scholars are not going to jail for their bad ideas. That's not how a free society works. We'll have to settle for general disrepute.

deleted 2y ago

[deleted]

nineteen999 2y ago

> attempting to shield them from ordinary academic pressures oh the poor darlings

ronnieadkins08 2y ago

[dead]