3 ms·
> Do we still think we'll have soft take off? There's still no evidence we'll have any take off. At least in the "Foom!" sense of LLMs independently improving
by mrandish 9mo ago
> Do we still think we'll have soft take off?
There's still no evidence we'll have any take off. At least in the "Foom!" sense of LLMs independently improving themselves iteratively to substantial new levels being reliably sustained over many generations.
To be clear, I think LLMs are valuable and will continue to significantly improve. But self-sustaining runaway positive feedback loops delivering exponential improvements resulting in leaps of tangible, real-world utility is a substantially different hypothesis. All the impressive and rapid achievements in LLMs to date can still be true while major elements required for Foom-ish exponential take-off are still missing.
- rahulyc 9mo agoYes, but also you'll never have any early evidence of the Foom until the Foom itself happens.
- janalsncm 9mo agoIf only General Relativity had such an ironclad defense of being as unfalsifiable as Foom Hypothesis is. We could’ve avoided all of the quantum physics nonsense.
- Davidzheng 9mo agoit doesn't mean it's unfalsifiable - it's a prediction about the future so you can falsify it when there's a bound on when it is going to happen. it just means there's little to no warning. I think it's a significant risk to AI progress that it can reach some sort of improvement speed > speed of warning or any threats from AI improvement
- phdlalala 9mo agohttps://zenodo.org/records/18498514 https://zenodo.org/records/18498514 GR is nonsense!
- Davidzheng 9mo agoTo me FOOM means like the hardest of hard takeoffs and improving at a sustained rate which is higher than without humans is not a takeoff at all.