4 ms·
AI is already improving itself. Most / all of the coding harnesses are AI-written. And yet… there are still people telling AI how to improve itself. IMO there
by brookst 3mo ago
AI is already improving itself. Most / all of the coding harnesses are AI-written.
And yet… there are still people telling AI how to improve itself.
IMO there will always be a level of abstraction at which AI needs guidance. Perhaps ASI means it decides everything on its own, but I don’t think so. Genius humans often excel at the how but not the why, or even the what. So far there’s no indication that AI is different.
- dinfinity 3mo ago> AI is already improving itself. Most / all of the coding harnesses are AI-written. AI has not improved the network topology much yet. The next (and possibly 'last') big thing is enabling AI to come up with something as impactful as the transformer architecture.
- simonw 3mo agoIt's not exactly discovering the transformer, but GPT-5.6 Sol apparently just found optimizations that reduced OpenAI's cost of serving it by 20%: https://twitter.com/reach_vb/status/2082581596608376980 https://twitter.com/reach_vb/status/2082581596608376980 I'm guessing that's hundreds of millions and maybe even billions of dollars per month in savings.
- brookst 3mo agoIt’s a fair point, but I’d argue “AI” is the whole stack, not just model weights. And AI is absolutely improving the whole stack in ways that make it smarter (from a consumer’s perspective).
- dinfinity 3mo agoTrue, but that is clearly not where the biggest potential gains lie. The number of different topologies we've tried for ANNs is miniscule compared to the number of biological neural networks evolution has tried. The latter are also far more intricately organized. Honestly, when it comes to fundamental ANN topology improvement we've only just gotten started.