3 ms·
AI Improves at Improving Itself Using an Evolutionary Trick
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- datameta 1y agoI think this is interesting enough for a post in and of itself: https://arxiv.org/abs/2505.22954 https://arxiv.org/abs/2505.22954
- wiz21c 1y agoFrom the article abstract: "All experiments were done with safety precautions (e.g., sandboxing, human oversight)." Do the authors really believe "safety" is necessary, that is, there is a risk that somethign goes wrong ? What kind of risk ?
- datameta 1y agoFrom what I understand, alignment and interpretability were rewarded as part of the optimization function. I think it is prudent that we bake in these "guardrails" early on.
- achrono 1y agoI wish an org like IEEE would be way more rigorous than what's revealed with the first paragraph: >In April, Microsoft’s CEO said that artificial intelligence now wrote close to a third of the company’s code. Last October, Google’s CEO put their number at around a quarter. Other tech companies can’t be far off. Take a moment to reflect -- a third of the company's code? Generative AI capable enough to write reasonable code has arguably not been around longer than 5 years. In the 50 years of Microsoft, have the last 5 years contributed to a third of the total code base? This itself would require that not a single engineer write a single line of code in these 5 years. Okay, maybe Microsoft meant to say new/incremental code? No, because Satya is reported to have said, "I’d say maybe 20%, 30% of the code that is inside of our repos today [...] written by software".
- throwawayoldie 1y agoIt's not his job to accurately report numbers, or really to do anything that involves technical acumen. His job is more akin to being a cheerleader, or a carnival barker.
- exe34 1y agoMaybe they've had LLMs for a very long time, given the quality of their code...
- cimi_ 1y agoThey probably mean new code not the entire codebase, but even so I think those numbers are ridiculous given my experience. Is there any evidence of this (anywhere, not just MS or Google)?
- paulluuk 1y agoI'm not sure if it's ridiculous if you factor in something like copilot. Heck, even just your IDE's built-in autocomplete (which only finishes the current variable name) can get close to being responsible for 20% of your code, with tools like copilot I think you can even more easily hit that target.
- davidmurdoch 1y agoThey clearly mean "new" code. Meaning on any recent day, that amount of code is authored by AI.
- achrono 1y agoNo, because Satya's claim is about "30% of the code that is inside of our repos today". Source: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a...
- davidmurdoch 1y agoVery clearly not what he meant.
- gavinray 1y agoThere's an interesting parallel to be drawn here from prior RL research: "Some evolutionary algorithms keep only the best performers in the population, on the assumption that progress moves endlessly forward. DGMs, however, keep them all, in case an innovation that initially fails actually holds the key to a later breakthrough when further tweaked. It’s a form of “open-ended exploration,” not closing any paths to progress. (DGMs do prioritize higher scorers when selecting progenitors.)" Kenneth Stanley[0], the creator of the NEAT[1]/HyperNEAT (Picbreeder) algorithms wrote an entire book about open-ended exploration, "Why Greatness Cannot Be Planned: The Myth of the Objective". [0]: https://en.wikipedia.org/wiki/Kenneth_Stanley https://en.wikipedia.org/wiki/Kenneth_Stanley [1]: https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...
- paulluuk 1y agoIt's really a choice: do you want to waste compute or do you want to waste potential? While prioritizing higher scorers for selecting progenitors will initially mitigate some of the problems, you will eventually end up with hundreds of thousands of agents that only learned to repeat the letter "a" a million times in a row, which is a huge waste of processing.
- tmaly 1y agoI followed his work on NEAT at the time. It was really cool. But I never imagined we would get to where we are today with AI.
- spwa4 1y agoThe same could be said of transformers, that only started to perform when scaled up to an absolutely ridiculous degree. I would argue most researchers are of the opinion that any learning system, scaled up enough, would work. I think the limits of machine learning are related to the fact that all ML "knowledge" is secondhand, except talking to humans and already to a much smaller extent programming. Getting AIs to interact with, say, cars, during training is the way forward.
- catoc 1y agoNumber of lines of code… airplane weight… etc
- pvg 1y agoDiscussion of the paper https://news.ycombinator.com/item?id=44135369 https://news.ycombinator.com/item?id=44135369
- deleted 1y ago[deleted]
- mucha 1y agoWhat Satya says: “I’d say maybe 20%, 30% of the code that is inside of our repos today and some of our projects are probably all written by software,” First line from the article: In April, Microsoft’s CEO said that artificial intelligence now wrote close to a third of the company’s code. Software != AI Source: https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-as-30percent-of-microsoft-code-is-written-by-ai.html https://www.cnbc.com/2025/04/29/satya-nadella-says-as-much-a... CNBC misquotes Satya in the same article with his actual quote.