4 ms·
Scary stuff from one of the winners: "A Taxonomy of Omnicidal Futures Involving Artificial Intelligence" (Jacob Tsimerman, Andrew Critch) https://arxiv.org/p
by hackandthink 2mo ago
Scary stuff from one of the winners:
"A Taxonomy of Omnicidal Futures Involving Artificial Intelligence"
(Jacob Tsimerman, Andrew Critch)
https://arxiv.org/pdf/2507.09369 https://arxiv.org/pdf/2507.09369
- sirbutters 2mo agoJust by reading the abstract it's terrifying.
- gizajob 2mo agoThis is a piece of sci-fi formatted with LaTeX so it looks like philosophy.
- yewenjie 2mo agoDoes anyone know if Tsimerman talked about AI extinction risk at other places? Critch has worked long-term in the field (MIRI, CHAI) but I was surprised to see this colab.
- lhd1 2mo agothey are friends according to https://www.ams.org/journals/notices/202607/noti3372/noti3372.html https://www.ams.org/journals/notices/202607/noti3372/noti337...
- energy123 2mo agoJacob Tsimerman believes that AI will be better than human mathematicians within 2 years. We should not be surprised if AI solves the Millenium problems very soon and advances to a level that is barely comprehensible, or even incomprehensible, to the best humans. We can expect this not by curve fitting to recent progress but by reasoning from first principles about where the progress has come from (synthetic data, non-human corpus) that has no obvious upper bound on capabilities.
- WarmWash 2mo agoThe obvious and primary application will not be on showy millennium problems or long sought conjectures, but on optimizations and breakthroughs surrounding transformers.
- subhajeet2107 2mo agoWe are doing fine so far with just Transformers, we have just crossed Trillion parameter models, we don't know if 100s of Trillion param model wont be better.
- WarmWash 2mo agoThe breakthroughs would be mostly in optimizations for computing tokens, not as much in expanding parameter count (if anything we want to shrink that).
- warumdarum 2mo ago[dead]
- ares623 2mo agoEd Zitron should write his newsletter with LaTeX and publish them as PDFs in arxiv. Seems to make the techbros automatically take it seriously. Maybe that's what it takes to pop the bubble.
- fragmede 2mo agoWhether or not Ed is right, he doesn't take any criticism, but peer review is a fundamental tenet of actual science. Vs being a blowhard on the Internet and getting paid for it.
- zipy124 2mo agoPeer review is actually a surprisingly modern concept. It is not actually a fundamental tenant of science. The scientific method notably doesn't actually include peer review in it's core.
- walrus01 2mo agoI think this says more that being very good at theoretical math does not at all translate into intelligence or subject matter experience about how humans will realistically handle potential armed conflict and potential risks of mass casualties, at the political/nation-state level.
- nextaccountic 2mo agoThis would work better as a lesswrong post
- Jach 2mo agoMay all world-class mathematicians take a look at the AI alignment problem such that humanity can have a better chance of passing through, and may some of them decide to not focus just on sexy parts like coming up with ways to kill us all, and may those who do so anyway at least come up with more interesting or plausible stories than the ones presented in this paper.