3 ms·
This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity. This was
by x312 25d ago
This is increasingly the consensus I see also on the academic side of AI/safety research. Specifically that AI poses an existential risk to humanity.
This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human comprehension.
There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint. And it's hard to imagine a world where current limitations like poor sample efficiency or lack of continual learning won't eventually be solved.
Total AI compute is estimated to grow somewhere in the 1-10 million-fold range in the next decade. Please don't underestimate the phase change that's still coming.
Sure, maybe there's some plateau due to RL being fundamentally limited in some surprising way, but this is nothing but a hope.
- platinumrad 25d ago> There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint. Yes there is: write an English paragraph that doesn't make me want to claw my eyes out. LLMs are not better than human mathematicians (or security researchers) in all respects, just some specific ways (e.g. not having to take a lunch break) that make them good at exhaustively searching for an answer, given the right constraints.
- aesthesia 25d agoWhat's the fundamental constraint that will ensure this continues to be the case in a year, or five years?
- deaux 25d ago> write an English paragraph that doesn't make me want to claw my eyes out. LLMs are very much capable of that. Your belief in the opposite has two causes. Firstly the toupee fallacy. You don't notice LLM written text that doesn't make you claw your eyes out. Second is defaults. The huge majority of people who writes text with LLMs just uses default Claude/GPT models, and put near zero effort in making it sound human. Those models are indeed bad at it by default so they need a lot of effort to overcome it. In cases like Opus 5 it's near impossible to overcome. That doesn't generalize to "LLMs".
- platinumrad 24d agoSure, it might take more than a single paragraph for that grating feeling to sink in, but none of the entries from the Un-Slop Fiction Prize[1] impressed me. [1] https://www.hyperstitionai.com/unslop https://www.hyperstitionai.com/unslop
- sph 25d agoI really hate how people who have always thought AI research to be an existential risk for humanity, now are apparently bundled to be on the same side of the Sam Altmans and Dario Amodeis that are using the existential risk as a sneaky form of marketing for their products. You cannot discuss existential risks of AI without being seen as a booster, and that is very unhealthy for the discourse around this tech. I hate how AI ‘doomer’ is now used to indicate pro-AI sentiment. The “moderate” person now is the one that just shrugs and scoffs at the deep societal changes this tech will bring, head deep in the sand.
- Mr_Minderbinder 23d agoI have been wary of the risks surrounding AI since watching Robert Miles on Computerphile more than a decade ago. Since then I have paid at least some attention to developments coming out of AI safety research. Even back then the discussion was fraught with doubt which, given the nature of this topic, was to be expected. It is unfortunate that it is worse now, when the stakes are much higher.