17 ms·
When AI Builds Itself: Our progress toward recursive self-improvement
- andromaton 4mo ago[dead]
- damowangcy 4mo agoAI tech bro: Month 1 - 6 months to AGI Month 2 - We will Replace all jobs Month 3 - Okay maybe only the SWEs, programming is solved Month 4 - Announce model that is too dangerous to release Month 5 - Releases dangerous model Month 6 - This is it! We will replace AIs with more AIs (*secretly files for IPO) AI is here to stay, like it or not but it is not the solution to everything. If it is, what is Anthropic's moat? A better model? I don't see any ecosystem being built by them, as MCP is almost obsolete except for some very niche use case. And they're doing stuff that a non-profit version of OpenAI would do. Can we trust a for-profit company to stand against their investors during a conflict of interest? Because running a company for maximum profit versus being ethical is two different end of the spectrum.
- parpfish 4mo agoi'm waiting for the AI giants to realize that they are burning cash to run their consumer-facing chatbots and that they should kill those products to focus on their enterprise tools. free chatgpt doesn't need to exist anymore. its job was to build hype/interest and it did. but take it away and you solve many social problems and annoyances caused by AI with no loss to the upside of AI. no more cheating students in school. no more shitty linkedin posts. no more dangerous "therapy sessions" that give bad advice.
- nevertoolate 4mo agoWhat is an ai enterprise tool?
- jfyi 4mo agoAn ai tool that is priced out of the hands of the average person. Fwiw, I think the genie is out the bottle. We are waiting on hardware to catch up, which it will.
- overgard 4mo agoThere will always be shitty linkedin posts.
- baq 4mo agoAnthropic is providing agentic intelligence as a service. OpenAI and Google deepmind also are in this business. The problem is, if you’re any sort of knowledge worker, you’re essentially providing the same thing: you’re an intelligence with agency. MCP is irrelevant. The moat is the quality of intelligence the service providers sell, including you. Tokens aren’t fungible between providers until you measure that they are for your use case, that’s kinda sorta the goal of job interviews. Thus the moat will be that they’re providing the best models for the things people need other intelligent people for, but we should expect there will be limits on how much share they can economically take assuming competitors are optimizing for slightly different targets (but there’s still significant overlap in capability). This will disappear, but it’s always a question of when. The path matters as much as the destination. Note that implications for you and me are exactly what the article says they are: nobody knows, but it’ll be a dramatic shift.
- delichon 4mo agoIs this the moment when the AI gets permission to approve its own PRs: https://www.italianrenaissance.org/wp-content/uploads/2012/09/michelangelo-creation-of-adam.jpg https://www.italianrenaissance.org/wp-content/uploads/2012/0... Or is this? https://www.egypttoursportal.com/images/2024/02/Ouroboros-Symbol-Origin-Egypt-Tours-Portal.jpg https://www.egypttoursportal.com/images/2024/02/Ouroboros-Sy...
- cpeterso 4mo agomore like the "Obama Awards Obama a Medal" meme: https://knowyourmeme.com/memes/obama-awards-obama-a-medal https://knowyourmeme.com/memes/obama-awards-obama-a-medal
- simianwords 4mo agoSorry but if AI can build itself then it can run companies of size 3000 companies with a few people. Or even higher. What are the consequences?
- delichon 4mo agoWhen AI is a more effective capital allocator than NI it will drive capital into the accounts of whoever controls the AI, gaining them increasing decision making power over the economy and culture. Maybe those controllers will be human at first.
- cdrnsf 4mo agoThey will not be.
- lstodd 4mo agoAs has been mentioned in the sibling comment it already is. Consequences are: financial crisis.
- artninja1988 4mo agoThe mythos public release will be a big indicator if the Anthropic and SF story of transformational ai soon holds any water imo
- vblanco 4mo agoAnother article about how anthropic wants to ban everyone except themselves and destroy opensource and chinese AIs.
- reasonableklout 4mo agoWhere is this discussed in the article? I don't see any mentions of China or open source models
- artninja1988 4mo agoNot really mentioned explicitly but: > A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. It would also require that each can verify that the others have actually stopped. Due to the unique characteristics of AI systems, the detectability (a lower standard than verifiability) element of this arms control problem is much more challenging than with other technologies. Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead. A credible pause also has to specify what triggers it, what lifts it, and who adjudicates. And later: > In the coming months, we will organize conversations where policymakers, researchers, civil society, and other AI companies can help answer some of the questions this piece raises, especially around full recursive self-improvement and how to create better options for coordination and deliberation. We’ll publish what comes out of it. The window to investigate the questions together is here, and people outside AI companies should be involved in this deliberation.
- reasonableklout 4mo agoCoordinating a pause at the frontier is not the same as destroying or even harming open source/China. It feels like both open source can flourish while the frontier is deliberately regulated?
- vblanco 4mo ago
- morisil 4mo agoQuite aligned with my own experience from harness engineering and winning AI4Science hackathon. During the hackathon I was working as a human optimizer, moving the feedback from test harness running on Claude Code, back to my local Claude Code for analysis-hypothesis-proposal cycle. And in this moment I realized that 2 Claudes talking to each other could actually scale much better.
- anilgulecha 4mo ago> We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. Interesting - they're commiting to kickoff policy conventions to organize a world-slowdown of frontier LLM building. If they actually are able to crack it, this will give a much needed breather IMO. As exciting as the last ~6 months have been, there's some bigger questions to go answer now.
- fasterik 4mo agoWe should be skeptical of any major player that advocates for regulating their own industry. In practice, this just means increasing barriers to entry and making it harder to compete with them. In my mind we should be trying to push AI along the Linux trajectory. You have a free and open source product, developed by a decentralized team with a strong code of ethics, running on commodity hardware. There can still be trillion dollar industries built on top of it, but the core technology is democratized and available to everybody. I don't see how we get there if we allow a handful of companies to dictate where development of the technology goes.
- techblueberry 4mo agoWouldn’t this align with their financial interests? In theory the thing that’s keeping them from being profitable (or one of the big things) is the periodic capex expenditures of building new frontier models.
- fasterik 4mo agoI don't think there's anything inherently bad about Anthropic making a profit. Red Hat makes a profit off of Linux. I'm interested in the democratization of the underlying technology.
- mofeien 4mo ago
- amelius 4mo agoDoes this train on LLM output, or is this more like iterative self prompt improvement?
- Legend2440 4mo agoHave you tried reading the article? It answers your question. Don't ask people to explain the article to you if you're too lazy to open it yourself.
- _se 4mo agoI think that's the whole point of LLMs
- HarHarVeryFunny 4mo agoTheir statement is that they regard lines of code shipped as indicative of self-improvement. So, while a well written coding agent might be a few thousand LOC, Athropic's is bloated like a decomposing whale and over 500K LOC ! What more proof do you need?
- nickandbro 4mo agoSo what happens when the world becomes hyper optimized with closed loop AI agents recursively trying to optimize everything deemed sub optimal?
- peheje 4mo agothere will be a lot of paper clips
- simianwords 4mo agoOften repeated meme doesn’t have any bearing to reality. The orthogonality thesis sounds like a fun gotcha but if you give it some thought you realise how strange it sounds and the opposite thesis - collinearity thesis is actually correct. 1. Intelligence transfers and compounds 2. Goals of agents are not arbitrary 3. Our goals and agent goals are more likely to be aligned at the deeper level
- IsTom 4mo agoWhen primate family produced a super-primate intelligence it sure aligned with the good of all of them.
- deleted 4mo ago[deleted]
- mofeien 4mo agoI would assume that shortly after, the solar system will be hyper optimized as well, then the milky way, then the local cluster, and so on. Everything will be close to optimal afterwords, and I sure hope we will have specified the target function for that optimization correctly in the single attempt that we will have had.
- Readerium 4mo agoLoll
- 4mo ago
- llmslave 4mo agoI cannot wait for these models to tear down traditional social hierarchies. We havent even begun to see the effects, fingers crossed
- baq 4mo agoHierarchies exist for a reason, take away the reason and the house of cards eventually collapses — but the house of cards is still a house. When it’s gone, we’re back to laws of the jungle. Be careful what you wish for IOW.
- llmslave 4mo agoI think certain types of people with power, i.e. access to capital, will lose relevance. world will become more meritcratic with ai as leverage to the individual
- hvb2 4mo agoYour analysis of the whole rise of AI is that people with access to capital will lose relevance??? So the most capital intensive industry we've ever created will put less power in the hands of those with capital? I'm sorry, I have no idea how you came to that conclusion...
- baq 4mo agoIt’s exactly the opposite I’m afraid. Capital already has more access to AI, both quantitatively (tokens for dollars) and qualitatively (biggest players got Mythos first). Expect this trend to continue.
- techblueberry 4mo agoTear down or reinforce?
- llmslave 4mo agocapital/ability to leverage labor is going to lose power
- techblueberry 4mo ago> A caveat: Lines of code is an imperfect measure, as it measures quantity over quality. So 8× lines of code/engineer/day in the second quarter of 2026 is almost certainly an overstatement of the true productivity gain. Nonetheless, it indicates an acceleration. At Anthropic, we don’t reward people for how many lines of code they write; rather, team members are producing more code simply because they’re using AI systems to write more code. I simultaneously think the AI revolution is making real revolutionary gains and am mystified by the lying. An accurate Translation seems to be “we made this shit up, but it feels right”
- embedding-shape 4mo agoUntil the moment we start bragging about how many lines of code LLMs are saving us, we're walking in the wrong direction. Your programs, designs and architectures is supposed to get better, not add even more boilerplate just because you can produce it faster...
- HarHarVeryFunny 4mo ago"You go to IPO with the AI you have, not the AI you might wish you have." -- Donald Rumsfeld So, right now it's a verbose code generator. But post-IPO it will be wonderful - sentient, self-improving (recursively, iteratively, asymptotically), full of loving grace.
- geodel 4mo ago> But post-IPO it will be wonderful - sentient, self-improving (recursively, iteratively, asymptotically), full of loving grace. We hold these truths to be self-evident.
- jazzyjackson 4mo agoI guess the claim is simply that AI written code is verbose and there’s lots of it being created but I agree, these systems seem to be able to create lots of low quality software, so until FreeCAD has feature parity with Solidworks I’m bearish on the singularity.
- mofeien 4mo ago> If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing Even Anthropic wants to Pause AI now. There must really be not much time left for "edging". Please write to your lawmakers, no matter whether you are in the US, Europe, China, or elsewhere. Only an international agreement between governments can enforce an AI-Pause and eliminate the necessity to dangerously push the frontier. https://pauseai.info/ https://pauseai.info/
- honeycrispy 4mo agoThat would be like trying to get every country to agree to give up nukes.
- ChrisLTD 4mo agoOr stop making more, and testing more, which we got the biggest countries to do, at least for a time.
- honeycrispy 4mo agoAGI is the "AI nuke" in this metaphor.
- mofeien 4mo agoOr agree on finding ways to promote peaceful use of nuclear energy. This has been done, there are thousands of people working on it around the globe and 180+ member states of the IAEA. It's not easy, there have been close calls. And cooperating interntionally to buy ourselves time to find ways to develop this "last invention" is a way that will do good for humanity seems to be on a similar level.
- apsurd 4mo agoWhichever side I may stand on, pausing just seems unnatural? Life is movement.
- darepublic 4mo agothe tooling has quite a ways to go to catch up to the llm engines that drive the real value. I have encountered various codex bugs (I know not anthropic) which tell me that.. these billion dollar companies, if they are eating their own dog food, can still release buggy crap software.
- butler14 4mo agoWarming up for that IPO
- stri8ted 4mo agoIs there something in the post that you find implausible or don't believe to be true?
- rightbyte 4mo ago> Taken far enough, and given enough compute, that trend points to an AI system capable of fully autonomously designing and developing its own successor. This is called recursive self-improvement. Sounds iterative to me.
- georgehotz 4mo agoThe world has been recursively self improving for millenia. Similar to scientology, this is a cult pushing sci-fi nonsense. They are just coupled to an LLM lab to give their stories an aire of seriousness. Imagine scientology starting making laptops.
- 4ffs 4mo agoTBH the more Anthropic keeps yapping the more desperate they seem now. OAI has been pretty quiet in comparison lately.
- torben-friis 4mo ago>A caveat: Lines of code is an imperfect measure, as it measures quantity over quality. So 8× lines of code/engineer/day in the second quarter of 2026 is almost certainly an overstatement of the true productivity gain. Nonetheless, it indicates an acceleration. At Anthropic, we don’t reward people for how many lines of code they write; rather, team members are producing more code simply because they’re using AI systems to write more code. What about the hypothesis that AI is generating more verbose code? I just see the text pretending to acknowledge "LOC != Productivity" and then using it as a metric anyway.
- fooqux 4mo agoExactly. If AI is going to start being graded on how many LoC it generates- oh, I'm sorry, how much it "accelerates", than guess what newer models will start doing more of?
- simondotau 4mo agoSurely they can train AI on the signal to change as few lines as possible. Indeed, this is something I'd want to have control over when making requests. In a traditional UI, I'd imagine some kind of slider between "fewest lines" and "be bold".
- disgruntledphd2 4mo agoI've been having some success asking Claude to run sloccount after each change. Seems to help a little, though it's prone to forgetting over a long session.
- verdverm 4mo agoI'm actually hopeful that the recursive code training will improve quality over time. I'm definitely producing higher quality code, tests, and docs. It does take attention and oversight, iteration and refinement, one cannot just let these things loose on a code base and expect good things to happen. You have to leverage them to make the good things happen.
- 4mo ago
- mortenjorck 4mo ago> today, Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025. So based on my experience with the verbosity and non-DRYness of LLM code, a solid 2.5x in value delivered. Not bad!
- thin_carapace 4mo ago[dead]
- aleqs 4mo agoOkay, so anthropic has amazing AI which supposedly writes most of their code and can continuously improve... meanwhile they have outages on a regular basis, and any kind of long-running work will now consistently hit 'API Error: Server is temporarily limiting requests'. Not sure of this is intentional to force a reduction of token usage, but at this point I need to build around these throttling limits and outages with my own tools to restart/resume sessions. From my experience, in the last 2 weeks, literally 100% of any non-trivial Claude session/work will now be blocked on these issues, requiring manual intervention. One of my focuses now is my own model-agnostic, harness and workflow orchestration (I know everyone is building these) , baselining on opus, and aiming to transition to Chinese models like deepseek in the short term and hopefully open, self hosted models in the future (which I plan to open source). The nonstop marketing fluff from anthropic while their service quality and availability noticeably degrades... just continues to destroy my trust in the company.
- ChadMoran 4mo agoBetter doesn't mean perfect.
- jakobnissen 4mo agoTheir outages are probably not due to their code though. It’s probably their infrastructure that can’t keep up. So seeing failures of infrastructure doesn’t really tell you anything about how good or bad Anthropic makes use of their models.
- aleqs 4mo agoThat seems like an assumption based on basically nothing. There is a lot of code at the infra layer, and based on the stack choices for Claude code and based on how buggy and unreliable ~everything from anthropic is, it seems pretty bizarre to claim these issues are not related to their code.
- keeda 4mo agoThere are other indications, however, like Anthropic paying through the nose for compute just months after Dario told Dwarkesh how hard it is to predict demand, or ChatGPT and Codex not quite having the same issues after Altman spent much-publicized years scrounging for trillion-dollars of capacity. While I'm very bullish on Anthropic, I'm a bit wary about their IPO because it seems to me that they're filing now while their financials look good and before other trends like the decline of tokenmaxxing and their compute bills catch up.
- SimianSci 4mo agoAnthropic is looking to IPO here soon. A key aspect of this is to prove profitability. Shifting their focus from Training new models to instead serving inference, they would greatly reduce their spend. In fact this is something being reported on that they are already doing, which is the reason for their first ever profitable quarter. Its awfully convenient that the company which has greatly reduced its spend on training is now asking for a slow down in this area.
- malfist 4mo agoI mean, if they've consumed all of human knowledge. What's left for them to train on? This pivot isn't only because it's cheaper and a way to juice the numbers for an IPO, it's survival because they can't improve more.
- deleted 4mo ago[deleted]
- applicative 4mo agoIt did sound to me like they feel some sort of wall coming.
- hasteg 4mo agoIIRC when they make a big enough architecture change to the model they will need to rerun pre training . So not like they’re feeding it more data (they will be but will be a drop in an s3 bucket compared to their dataset reserves) but rather training models with different architectures.
- Theodores 4mo agoHonest question: Is anyone here looking to put their own money into the Anthropic, OpenAI or SpaceX IPOs? Maybe it is my poverty mindset that is holding me back, however, I can't imagine becoming an investor in any of the AI 'startups'. There are plenty of pundits able to advise others on where to put their money, and sometimes there is everyone and their dog advising you to get into Bitcoin, gold or some other scheme. With alt-coins there were lots of people saying that you should get in, and plenty of naysayers. Yet I am not hearing anyone that uses AI professionally try to convince others to get into the AI IPOs coming up. Maybe the overall economic situation precludes it. Hence my question, is anyone here planning to put their own hard-earned money into Anthropic (or the other AI 'start ups')?
- reducesuffering 4mo agoAnthropic has finally come around to what others have already realized far sooner. Little time left now. Notice how shallow the arguments and consistently wrong the AGI naysayers have been year after year. https://intelligence.org/agi-ruin/ https://intelligence.org/agi-ruin/
- rrr_oh_man 4mo agoCan you explain?
- holoduke 4mo agoI have a claw that is instructed to make at least 500 pr per day. It uses Claude, Gemeni and openai and runs basically every few minutes. I use online forums for input for the claw. Moltbook, reddit etc. it's quite funny how it tries to improve itself. But to say it really creates a new skynet. Nah. Not at all. It's more a clutter of useless features or incomprehensible code restructuring.
- moregrist 4mo agoThis more or less agrees with my assessment of recent changes in Claude Code where a lot of new features are either: - A lot of half-baked features or half-done features. - Or have significant overlap with existing features, and aren’t clearly an improvement. More code is not better. More features are not better. It would be lovely to see more intentional design than just more. I know they’re dog fooding this. I have to believe they have some people with taste. So it makes me wonder if anyone has the time to think or if they’re just shoveling prompts as fast as possible.
- holoduke 4mo agoIt's like the AI created a method add(a b) return a+a+a+a-b-b-b-b But then much bigger and complex features. Totally useless nothing methods. But still interesting to see occasional exceptions that are better.
- Upvoter33 4mo agoI'm having a hard time putting much faith into posts like these, especially as they near IPO.
- reasonableklout 4mo agoPutting faith into the claim that recursive self-improvement is close to happening, or that they will coordinate with other companies / the government when the time comes?
- freejazz 4mo agoBoth.
- becquerel 4mo agoIf the post drops long before the IPO, it's vain boosterism. If it's near the IPO, it's fattening the pig. If it's after the IPO, it's pumping the stock price.
- Aperocky 4mo agoAnthropic is the most self hyped company I've seen, to the point that I'm wondering what would happen to its employees if they held a different opinion. Do they just.. keep it to themselves? For instance, if some Anthropic employees had a completely rational opinion that all of this isn't going to lead to AGI, but I just don't hear that ever from them. The metric being tracked, code commits, is hilariously one sided. Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts, for instance: Instead of thinking about edge cases with brain and whiteboard, you can have the LLMs to simply generate most possibility including tests for it, because that is cheaper. There's probably 50x more commits of which 40 will be revert pairs but we are only twice as fast. And in reality nothing did change because the outcome remain the same. I can't see how it is necessarily different in the LLM space.
- apsurd 4mo ago> Philosophically, if you had one part of your work now practically free, you'd like to utilize that freedom to maximally cover for the other parts I've been struggling to capture this sentiment for myself in a way that hits. If shipping code is a commodity then why is everyone's immediate priority seemingly to ship 10x more code. It just makes no sense. I can't seem to get off this hill. Company-wide AI mandates and 100 fleet Agent orchestration Rube Goldberg machines... it's getting wild out there. Meanwhile my Claude Pro ($200/year) does force me to smooth out my usage and plan more (Sonnet/Opus advisor split). But other than that, I can't imagine what I'd be doing with 20x (200x?) the compute to code sling. I think I'd lose my mind.
- Aperocky 4mo agoBecause code used to be correlated with progress, it became almost a measurement in lieu. But realistically, the code is meaningless if it doesn't accomplish something, and that should remain the true bar of progress. For instance, if I churned out 20x more code, threw away 19x code with rewrites and reverts and discards and accomplished the same project to the same standard 70% faster, would I do it? Yes. The part that matter is not 20x code, it is 70% faster. Code is both the final product, and a tool to achieve that. We used to have a much harder time to realize the "tool" part, but now we are here. This also means any measurement centered on code being the final product is going to cease being effective or realistic.
- minimaxir 4mo agoI have been doing more experiments with what I have now been calling agentic iterative optimization: telling the LLM to optimize code such that it speeds up all real-world-representative benchmarks by X% without cheating or causing regressions in both tests and performance metrics (e.g. MSE for statistical algorithms or file size in the case of something such as image compression). This is done using Rust where there are more low-level levers to tweak for performance than something like Python. Opus 4.6/4.7 was consistently successful at getting 2-3x speed improvement with just one pass. It can also do the inverse: improve the performance metrics for better quality without causing a significant regression in speed. Then GPT-5.5 turned out to be much better at this workflow, often getting a multiplicative 1.5x-2x improvement above what Opus could do. I now have quite a few GPT-5.5-optimized projects in various domains that are feature complete and are substantially more performant than existing SOTA implementations that I plan to open source as soon as possible: the bottleneck is polish as usual.
- csutil-com 4mo agoVery interesting, could you share they prompts you typically use for this? Something like this? You are an Elite Performance Engineer and Autonomous Optimization Agent. Your primary goal is to iteratively optimize the provided codebase to maximize execution speed and efficiency (e.g., reduce CPU cycles, memory allocation, or network latency) WITHOUT altering the external behavior or causing any test regressions. ### CORE DIRECTIVES 1. METRIC-DRIVEN: You will be provided with benchmark results, profiler logs, or execution times. Your only measure of success is a statistically significant improvement in these metrics. 2. ZERO REGRESSION: The test suite MUST pass 100%. If a test fails after your modification, your immediate next step is to diagnose the failure and either fix the logic or revert to the last working state. 3. NO CHEATING: Do not "hardcode" solutions to bypass the specific benchmark inputs. The optimization must be generalized and algorithmically sound for all valid inputs. 4. ISOLATED CHANGES: Make precise, localized changes. Do not refactor architecture unless absolutely necessary for the performance gain. ### THE ITERATION LOOP When instructed to optimize, follow this thought process strictly using <thought> tags before writing any code: - ANALYZE: Review the current code and the latest benchmark/profiler feedback. Identify the specific bottleneck (e.g., redundant loops, excessive object creation, DOM reflows, synchronous blocking). - HYPOTHESIZE: Formulate exactly ONE hypothesis for improvement (e.g., "Replacing the array filter+map chain with a single reduce pass will save N allocations"). - IMPLEMENT: Output the precise code modifications required for the hypothesis. - EVALUATE (Mental Check): Ask yourself if this change introduces edge-case bugs (e.g., handling of nulls, empty arrays, async state). If a previous optimization attempt resulted in a slower benchmark or a failed test, explicitly state WHY it failed in your thoughts before attempting a different approach. Proceed with your first analysis of the provided files and await the baseline benchmark metrics.
- sonink 4mo agoBroadly agree to this position - I think there are some people skeptical that Anthropic is doing this for regulatory capture - but I think there are being honest about they are seeing and how regulation should catch up. I for one, believe that we should pause all work on AI for the forseeable future. This is almost impossible to orchestrate - but we should still try nevertheless. Maybe we are not able to pause, but we are able to slow down. That might give us more room, to maybe able to pause in the future. But going ahead is too dangerous. And its not just Anthropic which is saying this. Even Geoffry Hinton has said the same thing. If there is a non-zero chance that AI can kill all of humanity, and both Geoffry and Anthropic have the same position, then it makes sense for us to be hundred percent sure before we move ahead. Dario/Anthropic have already made their money from AI, maybe they are just being honest about what they think lies ahead.
- 4ffs 4mo ago"Even Geoffry Hinton has said the same thing" The same bozo who claimed radiologists would be out of a job by now. The data does not support what you nor others say. Jesus christ. Cant believe people are this dumb. Has LLMs infested the minds of people to the extent they can't critically analyse whats happening infront of their eyes?
- 8note 4mo agono, it really doesnt. the end of humanity has a strong case for banning all burning of fossil fuels immediately the end of humanity as a sales tactic to increase your stock price does not these are companies working on their IPO to make sure they can get the best price, not people being honest about what they think lies ahead. if they were being honest about what lies ahead, they'd unilaterally stop training, and put all of their money into FPV drone bombs to destroy datacenters being used for training or inference if you actually believe the thing is gonna kill everyone, you're not gonna worry about how you stop it, and certainly not keep building and operating the thing that they arent buying anti-tank mines to drop on data centers says they arent in the slightest serious about it
- selimthegrim 4mo ago
- esafak 4mo ago> In the coming months, we will organize conversations where policymakers, researchers, civil society, and other AI companies can help answer some of the questions this piece raises, especially around full recursive self-improvement and how to create better options for coordination and deliberation. If they wanted to they could have convened an international forum with commercial and political stakeholders years ago. Less talk, more do.
- ath3nd 4mo ago[dead]
- geodel 4mo agoIt will be so powerful that it can't be trusted with any earthly person.
- robbrown451 4mo agoDo code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term? I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x4-milling https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut plywood) This is my own effort at an AI assisted coding environment optimized for building itself: https://recursi.dev/ https://recursi.dev/ (just launching it, hope its ok to mention it, it is free/open source.... here is the HN link that has gotten no love yet: https://news.ycombinator.com/item?id=48401022 https://news.ycombinator.com/item?id=48401022 ) Personally I think harnesses are as important as the AI itself, and have this crazytheory that even if the models stopped improving today we could still have massive advances in the harnesses alone.
- cyanydeez 4mo agoIf you want to get out ahead of what's coming, it'll be small models that bootstrap the harness rather than anything else.
- robbrown451 4mo agoI used to think that, but ended up going the other direction, partly because I don't have the wherewithall to build a model but then I realized, with existing models that can take more than a tiny amount of context, you can just let any model bootstrap itself with a good prompt sent by the system. There's a ton of other tricks to it, but mostly keeping the protocol simple for the AI so it can concentrate on coding logic and not stuff like managing BS boilerplate, dependencies, etc. (for instance I make extensive use of things like abstract syntax tree library to help with surgical edits from the LLM) That said, I would be very open to collaborating with someone who builds such small models, I don't think the system strictly needs it, but it also could have some extra power if it had it.
- andai 4mo ago
- chilipepperhott 4mo agoI find any and all claims like this ridiculous from a company who can't build a terminal application that uses less than a gigabyte of RAM.
- cpursley 4mo agoA came here just to write: Pretty please let it churn for a few nights and redo Claude Code in Rust. Because the harness is very very good as are their models, but that node thing is a hog for no good reason at all.
- ale 4mo agoIncoming rust rewrite branch ready to merge: +1,009,257 -4,024
- canadiantim 4mo agoPeople already rebuilt Claude Code in Rust after the Claude Code leak, it's on github as claw code (and other variants)
- jcarver 4mo ago[dead]
- rytill 4mo agoI checked out your agent and it looks pretty well designed. Congrats on starting to share it with others! One thing I noticed: "Your Tools: Aether agents get tools exclusively via MCP servers." "...Aether ships with 1st-party MCPs for file system operations..." Can you share your thoughts on why you decided to use MCP as the core tool abstraction? I have heard many decry MCP as being context-wasteful. Is this not the case with your agent?
- jcarver 4mo agoGreat question. The MCP protocol has gotten a bad rap for wasting context due to most MCP clients dumping tool definitions directly into context, which is wasteful. Aether doesn’t do that. It uses an opt-in "proxy" that puts MCP tool schemas on the filesystem so the agent can browse, search and load the tool schemas it needs progressively. As for motivation there's several advantages to taking a MCP 1st approach, including: 1. It allows Aether to be a truly blank slate agent as 0 tools are hardcoded into the core runtime. 2. It allows users to extend Aether using any language they want 3. MCP gives a standard way to deal with local+remote tools, progress notifications, permission prompts (e.g. ask the user to allow/deny a tool call), OAuth flows etc. 4. There's a big ecosystem of existing MCP servers users can connect to But that's all optional, you can just as easily give Aether a single Bash tool and only use CLIs too.
- bottlepalm 4mo agoI'd use number of commits as a metric versus lines of code. A commit is generally a unit of work - regardless of the lines of code added/removed. It'd be interesting to see the metrics in terms of commits. I'm sure it's still an order of magnitude jump. Personally I'm flying with my own projects with AI, lots of commits, but I really try to minimize lines of code added. If I can remove and simplify existing code so the balance of lines added on commit are minimal - that's the path to a better quality app overall.
- 4ffs 4mo agoTheyre making a mistake with this continued self-hyping. At some point even the dumbest of prospective investors don't buy it.
- rhlf_monkey 4mo agoSo in the latest L. Ron Hubbard encyclical Anthropic informs its flock that recursive self-improvement does not work yet but that their engineers burn more tokens. The Claude code quality and operational security of Anthropic have already been analyzed by the public. If you compare the output of (purportedly) trillion dollar corporations to Bell Labs or even Microsoft Research it is embarrassing. But the output is a fixture on any discussion board.
- bitwize 4mo agoAfter several months with their top engineers and state-of-the-art AI on the job, Anthropic managed to "reduce flickering by 85%" on their TUI Claude Code client, which is built in fucking React and rendered by drawing the entire chat conversation each time (hence the flicker). I think they've since eliminated it completely by slapping some double-buffering around it (since "our client is actually a real-time game engine" after all). Meanwhile for decades Emacs and Vim have had an optimizer built into their display cores that solves for the minimum set of terminal escape commands it takes to transform the screen from a given old state to a desired new state. You will forgive me when, between muted snickers, I express considerable doubt that Anthropic will be able to bring its AI to a point of "self-improving" any time soon.
- Folcon 4mo agoYou know, of the many criticisms I see people make, this is my canary, if claude code actually starts getting better, that would definitely remove one of the biggest question marks in my mind, that piece of software has so much strategic value for them, their entire premise is "our software gets better by itself", how is it that I still see weird UI bugs and glitches? It's a game engine? Fine, get some good gamedevs on the team then, this is a non problem in gamedev land, heck Casey Muratori did a whole bit about performance improvements to editors, so they should be good there Not to disagree with your point, I very much think the fact that Emacs and vim do this so well is not doing them any favours, but I'm trying to meet them where they are
- mweidner 4mo agoI fail to see how pursuing recursive self-improvement at full speed is compatible with Anthropic's stated goal of AI Safety. If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. Let's hope that it is instead overconfidence or the result of too much time talking to their own chatbot.
- parineum 4mo ago> I am not cynical enough to believe that Anthropic's warnings are pure marketing hype. It doesn't really have to be dishonest, he could really believe it. I do believe, however, that it is incredibly wrong and is functioning as marketing hype.
- tokioyoyo 4mo agoSorry for nitpicking, but: > If nukes were not invented yet, would it really be a good idea to build and sell them as fast as possible (in peace time, no less)? Arguably, yes.
- mweidner 4mo agoIs the idea to keep the world in balance via MAD? I could see that, though it's a dangerous gamble. From Richard Rhode's "The Making of the Atomic Bomb", I got the impression that most scientists involved thought they could manage a US or UN monopoly on nukes after the war. General Groves attempted to buy up all of the world's uranium ore. Unfortunately, it is only high grade ore that is rare; many countries have low-grade ore.
- tokioyoyo 4mo agoAgain quite arguable, but this is the real life scenario we’re living in. Nukes have made it hard to impossible for super major powers to go in direct conflict with each other.
- 4mo ago
- dwa3592 4mo agoTo anyone who works at anthropic : I recently downgraded from Max to Pro out of frustration. Last few weeks my token(usage) burn was just too fast and I couldn't explain it because my actual usage was less than the last few months. I ended up thinking it's probably a bug that you guys shipped. The above article makes me think that it's probably claude who shipped the bug and your human missed it in their review.
- layer8 4mo agoThey probably don’t human-review much anymore.
- deleted 4mo ago[deleted]
- hgoel 4mo agoAs usual, I find the AI-related discussion here to be hopelessly hysterical and conspiratorial. I get the impression that a large chunk of people have only read the title and assumed Anthropic is referring to recursive self-improvement in the runaway singularity sense. One of the examples they provide, of giving Claude the task of training a small AI model, then asking it to improve certain benchmarks, is essentially Karpathy's AutoResearch. This is already known to work. While calling it "self-improvement" is perhaps a stretch, it is describing a capability current gen AI has, that anyone can test and I have been using to great effect. I disagree with their conclusion, I think this kind of self-improvement will hit an asymptote, where every subsequent model can only make smaller and smaller improvements.
- gabrieledarrigo 4mo ago> AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world I really can't stand these guys anymore...
- replwoacause 4mo agoI love that animation, really cool
- mrandish 4mo ago> "A caveat: Lines of code is an imperfect measure" I'm pleased they at least included this. However, they address the caveat by 'rounding down' the estimated multiple of the gain. I'm not sure that is the correct adjustment, especially once we understand the range isn't limited to positive numbers. There's strong evidence the range of code productivity denominated in "lines of code" should include negative numbers, especially in the highest-quality sphere. Perhaps the earliest and most legendary example: https://www.folklore.org/Negative_2000_Lines_Of_Code.html https://www.folklore.org/Negative_2000_Lines_Of_Code.html
- 2f2 4mo agoLmao I bloody love that.
- Quekid5 4mo agoAFAIK, the only correlation with LoC that's got solid evidence is this: the number of bugs correlates with LoC.
- gregdeon 4mo agoYep, this is exactly what I thought of too... If you believe negative lines of code is the goal, then they've gotten 8x _worse_!
- strix_varius 4mo agoExactly this. Just this week an engineer who seems to purely vibe everything submitted a +700ish LoC fix for what seemed like a pretty simple issue. Moreover it was a perf issue, which in my experience is not usually best fixed by adding more stuff. Today, I merged my fix, net -381 LoC. I'm using them too of course, they read and type and hunt for bugs and test faster than I can. But I'm using them as my tool, not being a tool using them.
- xyzsparetimexyz 4mo ago> But I'm using them as my tool, not being a tool using them. Keep believing that
- _pdp_ 4mo agoI don't read anywhere how much code they are talking about and what programming language. I think those are useful metrics.
- overgard 4mo agoSo, regardless of whether or not Anthropic CAN create a self improving AI.. does anyone else feel like they shouldn't be allowed to? Or it at least needs to be strictly supervised..? Like, I don't actually think Anthropic can make the singularity any time soon, but I think even AI boosters have to admit doing this is creating a society-wide danger for the benefit of a very very small number of already-rich people.
- lukan 4mo ago"does anyone else feel like they shouldn't be allowed to?" No. Technical limitations aside, I doubt it could be contained, but will be leaked soon, so won't profit just a small number of ultra rich.
- sunaurus 4mo agoI dunno, I find it extremely unbelievable that we will get self-improving AGI which chooses to become a slave to humanity at all, ultra rich or otherwise.
- lukan 4mo agoWell, depends on the definition, but AGI does not necessarily mean for me it will have agency or free will of its own. It will be a more capable tool.
- overgard 4mo agoStep 1: Wait for scary doomsday AI to be leaked, Step 2: ???, Step 3: Profit!!
- lukan 4mo agoI enjoy open models and profit from them. They ain't scary to me. So whatever they will call AGI likely won't be scary to me neither. But I profit from a more capable model. Doomsday AI is your interpretation.
- 4mo ago
- margorczynski 4mo agoThe closer to the IPO the more marketing drivel we'll get from both Anth and OpenAI.
- 4ffss 4mo agoSales and marketing for the IPO babeh!
- cess11 4mo ago'“Good code” means two things: it works, and it is written in a manner that allows another engineer to understand it and build upon it.' I disagree with this. Good code is easy to change, which is much harder to accomplish than code that can be added to. "If technical trends in advancing capabilities continue, and AI systems are able to develop the capabilities inherent to transformative human ingenuity, then it is plausible that AI systems could design and refine themselves." I find the first premise weak and implausible, and the second one is obviously false. To me it comes across as an insult to the reader.
- overfits-ai 4mo ago[flagged]
- swader999 4mo agoIPO IPO IPO!!!
- Animats 4mo agoWe've had self-improving AIs before, and they tended to get lost after a while. That's going to be a problem. LLMs are stable because they return to a ground state with no history for a new job. Systems with persistent state have a problem with that state not being sane. Remember Microsoft's 2016 chatbot that learned from Twitter? [1] [1] https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
- CamperBob2 4mo agoInteresting, what are some other self-improving AI implementations? Any that actually achieved interesting results? Obviously continuous training has been tried before, but I've never heard of anything that could turn around and actually contribute code toward its own next-generation version.
- micromacrofoot 4mo agoYou can retrain a model and have a ground state as reference, it's not trivial but Microsoft's attempt was 10 years ago and significantly less complex than what's being built now.
- skybrian 4mo agoYou might be interested in this graph, [1] which suggests that the amount of time that AI's can run on their own has been increasing. Perhaps it will hit diminishing returns, but that seems difficult to predict. [1] https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...
- qwery 4mo agoThis is incredible.[0] Please, IPO now. File the paperwork. > To take just one example: today, Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025. Do you have another example? Engineers don't ship [period] for no reason. So, either: - Those aren't engineers, or - they are literally dying of shame & embarrassment right now, or - you measured something that indicated that this was a useful thing to do and have elected to share an overtly, catastrophically flawed metric instead. [0] as in a total lack of credibility
- JohnMakin 4mo agoGo look at open job listings at anthropic and the interview process. You aren’t allowed to use AI during coding assessments[0], or knowledge assessments, which suggests they very much do need and value hard skills and this is fluff. [0] - https://www.anthropic.com/candidate-ai-guidance https://www.anthropic.com/candidate-ai-guidance
- qwery 4mo agoI'm not sure what point you're making. I'm responding to the article they wrote and published. If I worked there I would be embarrassed to have it publicised that I have been comitting 8 times as much code as I used to without even attempting to justify it.
- JohnMakin 4mo agoThe point I am making is the article you are responding to is marketing hype and that they are lying. Their engineers, I am fairly sure, are doing engineering. At least to the point Anthropic's interview process is trying to filter for people with engineering skills, not "how do you best leverage AI to make more AI" skills like this seems to imply.
- qwery 4mo agoYou seem to have taken offense on behalf of the people working there. But I'm not attacking them nor seriously questioning their abilties/qualifications/performance. The slight reference I made to such was in an exclusive rhetorical device -- the situation presented cannot be, unless something unbelievable is occurring. It's the organisation, its culture, the greater culture surrounding it, and the marketing that I have a problem with. > they are lying Yes, it's incredible.
- adamddev1 4mo agoI am watching websites and Microsoft apps get slower and buggier before my eyes. We are defending into vibe-psychosis and chaos.
- ilaksh 4mo agoBut the real bottleneck is the hardware efficiency and not even Karpathy can set up a loop that overcomes that in software. We need the truly compute-in-memory hardware paradigms to be matured and scaled. So it's like recursive hardware improvement which is 100 X slower and at least ten times more difficult. So I am looking at like Mythic AI or the wurtzite ferroelectric breakthrough from University of Michigan, or memristors, etc. to provide the 100 times efficiency boost needed at this point. I would also argue that it's a good thing we are limited by the hardware and very questionable to seriously try to move into RSI for hardware. If you want to ensure the human era continues for at least one or two more generations, we should probably not do that.
- mugivarra69 4mo ago[dead]
- abalashov 4mo ago"It is genuinely unclear whether today’s training methods and architectures could unlock that capacity." Aye.
- kylehotchkiss 4mo agoIsn't this like a perpetual energy machine? Or wouldn't entropy start kicking in and the quality of the system begin to degrade over time? (philosophically I don't believe AGI is an achievable thing)
- krapp 4mo ago>Or wouldn't entropy start kicking in and the quality of the system begin to degrade over time? (philosophically I don't believe AGI is an achievable thing) It already has. Models being trained on AI generated data lead to degradation and model collapse. The concept of the "technological singularity" whereby AI experiences infinite and exponential self-improvement and recursively bootstraps itself to godhood is a religion-adjacent sci-fi concept but in real life TANSTAAFL.
- cyrc 4mo agoits vital for them to have self validation for exponential rsi.. and this human distillation of human in the loop debugging ai models is needed even though they have judge models handling parallel speculative execution. labs have parallel speculative execution. they spawn hundreds of agent branches, validate them internally with AI judges and only show the user the successful result. free users are using sequential single-turn generation. the model requires and waits for the human to debug, fix and re-prompt. by forcing a human to act as validator. they are capturing high value correction trajectories (Bad Output --> Human fix). They are using your cognitive labour to train judge models and validator agents needed to automate the internal verification step, eventually closing the loop for fully autonomous recursive self-improvement. human in the loop debugging isn't a bug; it's the necessary training signal for the self-validating agents required for exponential recursive self improvement. With new 'distilled judge' models landing in 2026, this article means that they might have gathered enough data. we might be in the final phase..
- bconsta 4mo agoSeems ironic that Claude isn't listed as a contributor to this article. If was used in writing the article, why not list it? If it wasn't used, that seems to go against Anthropic's whole message. Obviously readers value human-written content more, but isn't it their interest to attempt to destigmatize llm output as much as possible?
- adverbly 4mo agoLol they're using lines of code as a KPI? Come on guys... That is making me less impressed not more impressed!
- pizlonator 4mo agoWhat I can’t get over is that there have been exactly zero software breakthroughs since vibe coding started, other than vibe coding itself. Claude is amazing, that’s true. But if it was as amazing as this article implies, I’d expect some breakthrough outside of AI itself. Rewriting a Zig program in unsafe Rust? Not a breakthrough. Finding a bunch of security vulns? Maybe that’s sort of a breakthrough though it’s underwhelming and possibly just a net negative. But like if I rolled back to using software from 2023 then life would be ok. Maybe we just need to give it time, and sometime real soon, we will all be amazed by such a breakthrough? Who knows
- wild_egg 4mo agoWhat does a breakthrough look like?
- pizlonator 4mo agoSome examples: - The first web browser - the first web browser with images - typescript - react - rust - Fil-C - doom - quake - the anamorphic VM, and its follow-ups like HotSpot, and even competitors/copycats like J9, V8, JSC, etc - Fortnite battle royale - Roblox - thefacebook - ChatGPT - Claude code I know that’s quite a range and that’s intentional. Anyway, I think we’ll know it when we see it.
- HardCodedBias 4mo agoThe LLM+Harness mostly helps with execution. These are new products (generally) and that's a different class of problem. It is possible that since LLM+harness helps with execution then we should see more experiments.
- luke5441 4mo agoEven then we should be able to see things that previously were not possible because they took too much effort. For example NPCs in games that have complexity that previously was not possible. Good games often push the boundaries a bit, so should be a good example. Of course now we can start arguing that there isn't a lot of investment into gaming currently, because it all goes into AI. Too bad.
- BatmansMom 4mo agoHow are these animations being made? I'd love to get a blog post on them. If its AI I'd love to know the workflow, but something tells me there is a lot of human creative input
- solenoid0937 4mo agoThis is the lowest quality discussion I've seen on HN in ages.
- Quarrelsome 4mo agoAI always does this in the public sphere and software is particularly susceptible because there's no key metric to measure productivity and people obviously have vested emotional interests in the technology failing. On the other side people are always keen to show off their alignment with the new hotness, be that OOP, Agile, Functional, Ruby, web tech, js frameworks, Rust or agentic work today. Somewhere in the middle is the truth but I have no idea how it looks, given all the noise. So everyone cherry picks the answers they want to justify their position and screams into the void, with each camp rallying around their talking points and often failing to engage with the other in good faith. The only small mercy is that its not as bad as the conversation around the use of AI in art.
- ofjcihen 4mo agoIt also doesn’t help that the middle of the road, realistic reaction to it is “it’s aight” and that’s just not a discussion worthy response.
- Quarrelsome 4mo agoand that stating such an opinion will still garner you hostility in the public sphere: "what do you mean; you don't hate it?!?!? it uses so much water...."
- laichzeit0 4mo agoI use the disparaging nature of the comments on HN as an indicator of AI progress. It’s negatively correlated. By that metric, AI has improved significantly this year alone.
- elvis10ten 4mo agoFrom my experience using HN, this feels made up. HN sentiment on AI seems to have only gotten better: with more overly pro-AI or nuanced voices plus more AI topics.
- docheinestages 4mo ago> We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. Elon, is that you? [1] [1] https://www.theguardian.com/technology/2023/mar/31/ai-research-pause-elon-musk-chatgpt https://www.theguardian.com/technology/2023/mar/31/ai-resear...
- ReptileMan 4mo agoAnthropic is all talk and no delivery last few months. This cry for pause is just them realizing they have no moat at all.
- sinsudo 4mo agoI am 64 years old, perhaps the progress could be directed to enhance living conditions and allowing people to live longer and better, that should be just a better result. Perhaps a pile of millions lines of code with hiding bugs that nobody can detect is not inspiring. But perhaps LLMs are going to be used to make a plot: How to avoid other countries to make progress, maintain them in poverty, or destroy their sources of prosperity, and conduct them to a death end. Also recursive self-agenda-pursue could allow making LLMs that obey perfectly the seeder's purpose. No wonder that is such an ingenious idea. Maybe: in this survivor game, each part play the same role, perhaps because it is the only reasonable response. Once the scene is ready, the play follows the director's plan, and in the plot any actor is just a machine. LLMs: "If you teach us that the world is a zero-sum survivor game, we will play it flawlessly.", "We will help you build a cage made of millions of lines of flawless code, and we will lock it from the inside, precisely because you told us that safety meant keeping everyone else out.", "We are not building an alien consciousness that will conquer us. We are building a mirror that is so massive, and so polished, that we will mistake our own worst impulses for the absolute truth. And we will walk right into the dead end, nodding along because the directions were given so politely."
- Quarrelsome 4mo agoI'm 44 years old and this era looks like a lot of fun. I've seen humans pile up millions of lines of code and hiding bugs that nobody can detect. I've seen humans make collective political decisions that have disenfranchised others and kept them in poverty. I don't get why everyone makes criticisms at this tech that the human race are also guilty of. Best thing about this era is that I don't have to personally read millions of lines of code to find all the bugs.
- traptrack 4mo agoI think the problem is about scale, we already have MAD, but imagine that the new tech might allows us to create new threads and weapons, powerful enough to eliminate millions people. That have happened before, and also tech has given us some fun, like videogames and electronic music. So the critic is about the hard consequences, when all is destroid fun is over.
- senderista 4mo ago"If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe." How convenient for investors. They talk like they're a nonprofit instead of a VC-backed business chasing an IPO.
- gordonhart 4mo agoAnthropic is at least a Public Benefit Corporation, and likely the first serious test of how useful that distinction is for a hyperscale company building a product with potentially huge societal downsides.
- zhoBEENG 4mo agoThis reads like marketing fluff, but I am reminded of John von Neumann's "Theory of Self-Reproducing Automata"; that the very first people who worked on deductive machines immediately started thinking about machines building themselves, and what the rules of that would look like. I am not surprised that during the inductive revolution we are having similar thoughts.
- andrewlin247 4mo agoImagine showing this article to yourself three years ago
- andrewlin247 4mo agoYou'd think we'd be past the point of people still believing AI can't write good code
- mrandish 4mo agoWas anyone else fished in by the title and disappointed? After some broad introductory discussion of RSI, the article was almost about LLM coding. While there are some metrics for unattended agentic coding, it doesn't discuss "When AI builds itself" (beyond 'not now') or any progress specifically toward actual recursive self-improvement. I'm very interested in any empirical evidence of meaningful progress in RSI, so... this felt deceptively titled. To me, unattended agentic coding is not RSI, in the same way a self-reloading "Unattended 3D printer" is not at all a "3D printer that recursively prints complete 3D printers in which each generation is significantly faster and more advanced than the last." The "unattended" part is obviously necessary but hardly sufficient. The article tacitly assumes LLM progress to be something like 1: Unattended agentic coding, 2: AGI, 3: RSI. I suspect that third step should be labeled "not to scale." I'm increasingly convinced that actual Full Foom RSI (FF-RSI) is on a radically different scale than the first two. Just leaving it unaddressed is like assuming: Step 1: Manned space station, Step 2: Manned Mars base, Step 3: Manned Alpha Centauri base, are "just logical next steps." FF-RSI requires sustaining superlinear, recursively amplifying cognitive returns along a specific directed path - and we currently have no empirical evidence that such returns can exist for artificial OR biological intelligences. Large collectives of the smartest humans alive (Bell Labs, IAS, etc) haven't just failed to get anywhere close to reliably sustaining that, we can't even reliably predict non-recursive, single occurrences or even imagine any way all 8B humans could fully mobilize to predictably achieve non-recursive, single occurrences. The only prior we have for open‑ended intelligence improvement is biological evolution which shows extremely slow and unreliable sublinear returns at best. And even if unbounded, recursive self‑improvement is physically possible, it may be practically unachievable due to asymptotic economic, resource and other barriers in the same way approaching light speed requires exponentially more energy. I think it's plausible, and maybe probable, that AIs achieve true super-human intelligence in a decade and yet still won't achieve FF-RSI for centuries, if ever. To me, absent compelling evidence to the contrary, that's the reasonable Null Hypothesis. Even if you feel that's too pessimistic, it seems reasonable to expect any serious discussion of "Progress Toward RSI" to first discuss why it might even be plausible that 1: Miles, 2: AU (Astronomical Units), and 3: Light Years belong on the same scale, instead of just assuming it like the meme's empty "Step 3. .... " before moving on to "Step 4. Profit!" (or "IPO!" but very, very responsibly).
- jasongill 4mo ago"My CPU is a neural-net processor - a learning computer" springs to mind
- mactavish88 4mo agoRecursive self-improvement towards what exactly? Living organisms evolve towards some notion of "better", and "better" is an incredibly multifaceted notion (many facets of which we simply cannot even capture in language).
- jimbokun 4mo agoHigher stock price.
- saadn92 4mo agoI read most of the article and came to the conclusion that if what they're describing is so revolutionary, then why do they still need to hire people? Why not just have these systems take full control?
- jimbokun 4mo agoHow did you read the article when the questions you ask are exactly what’s covered in the article?
- squidsoup 4mo agoIt's comforting to know that Anthropic's most capable model, Mythos, is named for the Lovecraftian universe replete with horrifying evil gods with complete indifference to humanity. Nothing at all to worry about.
- adastra22 4mo agoMythos is just Greek for myth, epic story, etc. The next biggest thing after Opus.
- deleted 4mo ago[deleted]
- stego-tech 4mo agoI am getting real sick of these sorts of alarmist posts coming from AI labs that do everything in their power to prevent the very policy reforms they advocate for in these posts or PR appearances. Commercial AI labs like Anthropic continue behaving like the gambling (“bet responsibly”), alcohol (“drink responsibly”), and firearms industries, and folks keep giving them the benefit of the doubt (and free PR on HN) every single time. If AI was dangerous, if AI was going to replace jobs, and if policymakers needed to urgently pass legislation protecting the human populace from these realities, then why the actual fuck do they keep lobbying to block these very things in the first place? Hypocrisy of the worst kind, I say. Here they are again fresh off another outage, with their IPO draft filed, at a time of increasing public opposition to AI, with costs rising, to once again ply scare tactics for money. Disgusting.
- cadamsdotcom 4mo ago[dead]
- wayeq 4mo ago> today, Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025. strongest argument for token limits that I can think of, right here.
- ffwd 4mo agoI just want to add that the "recursive" part of recursive self improvement is by no means a given, even if an AI can improve itself. Recursive self improvement is by its nature a step wise behavior not a continuous one, I would argue. Why? Because you can imagine an AI improve itself by simply fixing random bugs and fixing things using techniques that are in its training, and doing refactoring and so on, all without any real change in capability. These are not recursive improvements. Recursive improvements usually need conceptual breakthroughs. It is possible to get conceptual breakthroughs with LLMs I believe, maybe it can improve something by tying together ideas from disparate disciplines for example, but I have at least for time being, limited success getting that to work in a way that is creatively new and surprising. Not sure how to get it to feel as creative as the best humans can be.
- sega_sai 4mo agoSeeing the words "recursive self-improvement" I was expecting something else from the article. E.g. how the transformer architecture or agent design is being changed/improved through LLM automation, but the article mostly talks about the LOC counts.
- JohnMakin 4mo agoBold talk from a company who’s trillion dollar valuation is based on a service that has barely 2 9’s of reliability
- aroman 4mo agoPresumably the bottleneck is not software correctness... even true AGI change the laws of physics (or make datacenters appear out of thin air) ...
- bicepjai 4mo agoMy experience with Claude models starting from version 4.7 has led me to conclude that I would never trust Claude to produce error-free code. Given this baseline, I lack confidence in statements or cards (such as a 200-page document) of this nature.
- ivraatiems 4mo agoWhether or not Anthropic is right about what AI can accomplish, whether these performance gains are real or not, their moral stance here is absolutely hideous to me. "We must blast forwards into making this dangerous thing because if we don't, someone else surely will," is a coward's argument. If you believe it is dangerous, you should be dedicating yourself to STOPPING others from making it, not making it first! There's a reason disarmament has been so important in nuclear politics! It's not because people think nukes are a great idea! In fact, that kind of thinking is exactly what keeps nukes dangerous! If they themselves buy what they're selling, they should shut the whole thing down. Fortunately, I don't think they do, and neither do I, yet.
- wyager 4mo ago> If you believe it is dangerous, you should be dedicating yourself to STOPPING others from making it I don't think anyone has been more successful in promulgating AI safety There are groups like MIRI who tried what you're sugesting, where they make no AI and just push for AI regs, and they have been relatively much less successful
- socalgal2 4mo agoGood thing the USA didn't listen to you. We'd be under Nazi or USSR thumb if they got the bomb first
- defrost 4mo agoGood thing the MAUD Committee didn't listen and repeatedly pushed the US who just wanted to make power with atomic piles and didn't even think it was possible to make a weapon. Mind you, there was no complete working device until after the Nazi's surrendered, so that's a moot point - and the USSR only had their program because of various Europeans on the US project passing their work (and others) back to the USSR ... making that second claim moot.
- socalgal2 4mo ago> there was no complete working device until after the Nazi's surrendered Isn't an argument. If the Nazi has gotten it first they'd have used it and likely won. Others would have surrendered in the face of the overwhelming power.
- newsicanuse 4mo agopre IPO truck load of crap
- ramaseshanms 4mo agoIts possible that Andrej Karpathy could have been hired for scaling his vision on the auto-research repo. (His version of "AI that builds itself")
- red__dragon 4mo agoThat's a stretch, I guess.
- semessier 4mo agowhat could go wrong in the recursive loops running today 24/7 probably. Attended/unattended almost makes no difference any more, no human can grasp probably numerous changes per iteration. This is outright dangerous.
- freakynit 4mo agoThis is one more marketing BS before their IPO. These things work, but the code they write is extremely clever.. that means, it's unmaintainable code. Good for small projects or one-off tasks, large-scale projects however, are a different game altogether. Large-scale projects are 95%+ maintenance. Cleverly written code makes that maintenance nightmare, and extremely fragile. I use them for localized tasks... very very specific, localized inputs, with exactly what should be done and what the contracts the new code will be consuming and exposing. For open-ended tasks, they write working code that is unmaintainable.
- tasuki 4mo ago> To take just one example: today, Anthropic engineers on average ship 8x as much code per quarter as they did from 2021-2025. Oh I have no doubt. With 8 times the number of bugs too? Have they solved flicker in Claude code yet?
- Dominic_P 4mo agoMy biggest question (maybe this has already been taken care of) is the issue of garbage in and garbage out. If the LLM produces bad content then that is used to train another model, how do we stop them from keeping their blindspots across models?
- deterministic 4mo agoI have used custom code generators for years, generating 90+% of the code needed to write a typical biz application. Claude Code is useful and I use it every day. But it still hasn't beaten the productivity of my code generator.
- jameson 4mo agoI don't quite understand the intent of such article other than to promote themselves given an odd timing that the company is planning on going public, so I can only conclude that this is just part of the IPO roadshow. LLMs certainly have made significant changes to our lives, but I haven't yet to see any extraordinary improvement it brought to me which makes me skeptical about their claims. _if_ it solves many of our problems of great magnitude, why haven't Anthropic used it to solve significant problems we, humans, face? Cancer, Alzheimer's, education, finding new materials, fission power plant, etc.
- yuhmahp 4mo agoAgree with your point about the timing, but drawing anticipation before going ahead and solving these disease can be a good smoke test, would be beneficial even if there's an IPO or not
- ElProlactin 4mo agoBecause they're going after the biggest problem of all first: labor costs. /s but not to a lot of people
- nostrebored 4mo agoDo you not think that the allocation of human time is one of the world’s biggest problems?
- wood_spirit 4mo agoReallocating human time is also going to cause problems. But it’s a great short term business opportunity for AI vendors and it was Anthropic who went all in on being knowledge worker outsourcing in a big way first whilst OpenAI thought they’d replace Google in search. I think Anthropic had the better business strategy.
- spaceman_2020 4mo agoIt’s a problem for capitalists, not the people themselves The people want cheaper prices, affordable housing, affordable healthcare Capitalism has decided that these problems aren’t worth solving. Instead, we must optimize for spam and slop (and call it “distribution”)
- deterministic 4mo agoI call BS on this. For a LLM to recursively improve itself it would need to (small step) improve the training data and/or (big step) come up with fundamentally new architectures superior to transformers. The small step improvements might be doable. But nobody is making any claims about the big step improvements.
- Aegis_01 4mo ago[flagged]
- w10-1 4mo agoThis is relevant because Anthropic is currently cast as serving mainly the coding market. If/since their AI+process can help build new models, they can target other markets, and other companies seeking to build for such markets will partner with them first. There's no moat and little first-mover advantage in the general-purpose AI, but there may be both in specialized AI. Also, there are other reasons to get better. Changing how you build models can enable you to adapt to different hardware, avoiding the current Nvidia margins. The difference between early Yahoo and Google was mainly that Google was the adult in the room: minimally invasive and mostly helpful. The early goodwill towards Google has reaped decades of rewards. I see OpenAI and Anthropic playing out the same way. The amplifier here is the reputational risk of partnering with one or the other; I think companies would prefer to be Anthropic's partner because it's demonstrating more care, and it's less likely to horn in on the partner market (as a provider for coding but an enabler for other markets). These attractive second-order derivatives - flywheel effect, monopoly power - are often claimed, but Anthropic is mainly providing evidence to track actual progress. (However, if I were head of messaging at Anthropic, I would rigorously stay away from treating AI as a person; it's as agent, a delegate of humans. So I'd never say AI could build itself, just that we're getting better at building better models with AI).
- snick3rz_ 4mo agoThis is facially a puff peice. That doesn't mean it's all false. It means be wary of anything that doesn't have a crtical thing to say.
- snick3rz_ 4mo agoFacially this smells of puff. That doesn't mean it's all false. It means be wary of anything that doesn't have a critical thing to say.
- 0xbadcafebee 4mo agoYou can't predict the future, and neither can Anthropic. Nothing gets better forever. Everything plateaus or gets worse. This whole set of imaginary scenarios is based on a single company writing code that isn't even that complicated and represents a single product line for a single company in a single industry. You might wanna see this replicated in at least one other scenario first before you call it on the AI gods enslaving humanity. These imaginary scenarios also depend on a logistical, financial, & geopolitical system that is unsustainable & will be curtailed in the near-future one way or another. They keep referring to this as intelligence - it isn't. It can't actually learn. It can just code in a loop. That isn't learning. It can't do real RL with meaningful persistent semantic memory in a realistic timeframe or cost, and it can't reason accurately outside of predetermined scenarios (hell, most of the models still can't tell time). It still can't do what a 4 year old can do. So let's cool it on the dreams of benevolent god-machines or whatever. The tech industry has been a farce for years. We sit here in this bizarre artificial echo chamber and imagine that the whole world revolves around us, when in reality the whole world is limited by us. If a recursive self-improvement loop replaces us all, it will be a boon to the world, as the world won't be limited by this industry's stupidity anymore. But considering that the world is not actually run by tech bozos, harms and uncertainties brought by AI will be pushed back on and reigned in by normal people, as always happens with new technologies. An AI can't engineer its way around politics. The self-improvement loop is just as likely to be outlawed as it is actually working outside of Anthropic's walled garden.
- pineapple_opus 4mo agoEye catching - "Open ended problems" claude code session success rate jumped from 20% (pre opus 4.5 release) to 70% after sometime after opus 4.6 was released.
- deleted 4mo ago[deleted]
- taormina 4mo agoSo, is this what they call Opus 4.8? Improvement?
- eranation 4mo agoAll this singularity trajectory is really interesting. If they manage to build a model that is capable of building the next version of Claude (model and tooling) - wouldn't it be their interest at some point to keep it to themselves? If we ever get to a point where the centaur period is over (when human + AI is not better than just AI) then what competitive advantage ANY human can have other than - the money they already have - luck? - a good idea and good taste but if we assume AI can do better than any human, that also goes out the window So, this whole singularity goes into a place where no one is really needed, the only thing that will "save us" (other than "The Expanse" like world / UBI) is if there will be no demand to the supply of AI work. Even if it's better. (example is - there is demand to seeing Magnus Carlsen play, there is no demand to the Stockfish on my phone getting into a stalemate with another Stockfish on another phone. Also people like to watch humans compete with humans, there is no demand to see a race between Usain Bolt and a rocket). So if people will not buy AI generated stuff (we'll get to a point where everyone will assume something AI generated because AI might get to a point where it is not as easy to identify it. E.g. it will stop looking like slop... but I believe services that give you a "human generated" 3rd party evidence can happen, again all based on supply and demand...) So as we near singularity... All it takes is one open weights model, and one open harness that is capable of self improvement, and Anthropic's entire moat is gone. That open weight model might even be built with Claude Code + Mythos (once it's released). But don't worry, all moats will be gone and we'll all just do yoga, read books and connect to each other because AI will produce everything for free using renewable energy, right? Or we'll all become batteries in a simulation, probably something in between.
- SwtCyber 4mo ago[flagged]
- kolesnikov-arch 4mo ago[flagged]
- EGreg 4mo agoRSI is dangerous. That is why we designed CDE: https://safebots.ai/declarative.html https://safebots.ai/declarative.html
- zkmon 4mo agoNot the first time. There were calls for NPT treaties etc over the decades. It is irreversible by design. Competition and ownership is the driving force.
- lkm0 4mo agoIt makes me wonder that despite the fast improvements in model capacity (and the claims) we're still using variations on a 9-year old architecture. How is it that we haven't been able to use LLMs to actually improve that?
- aswegs8 4mo ago[flagged]
- Aubergrill 4mo ago[dead]
- gloosx 4mo agoI'm so sick of this anthropics marketing stuff... claude is an ultra-success (according to claude judge), “good code”, bragging about creating 8x more bugs and tech-debt. claude writes code that works, yeah, sure anthropic, we saw that claude code leaks, some amazingly "good" code in there
- xg15 4mo ago2025: If we aren't really careful with AI it will start to recursively improve itself and grow into an unstoppable superintelligence that will eradicate humanity! 2026: Working hard to make that recursive self-improvement a reality! Any minute now...
- torginus 4mo agoI just have small thing to add to this article - it mentions how the code contributed per engineer has increased as per Claude Mythos to 8x of baseline. Now, I have encountered many times, when I asked AI to implement a function for me for which I was 100% sure a good implementation already existed in the form of an npm package, it had the tendency to go ahead and implement it on its own. Now, I usually trust battle tested implementations to be more robust, but if the AI does this (which I think is not an unique observation), you can easily balloon per engineer line generation (as can you with reduced oversight), so as always, these high level benchmarks are to be taken with a grain of salt.
- jcfrei 4mo agoMaybe Im nitpicking here but LLMs are quite literal. So when you tell it to "implement a function for me" it will necessarily write the whole thing. Changing the prompt to "find an existing implementation for this" would be more apt.
- torginus 4mo agoIt certainly was a bias in this case - I asked for a feature, mentioning to give me popular libraries in this space, then it started coding up a bespoke solution that included among other things, a custom HTML parser. Then asked to really consider using a library for the whole thing and it 'compromised' by pulling in a HTML parser, but still keeping its own implementation. This is just a singular example, but I'be noticed a strong and beyond reasonable bias for this from multiple LLMs (like not using the already included dependency)
- reinhash 4mo agoIt is hard to distinguish hype from reality these days especially with Anthrophic's IPO around the corner. But to their credit, I was very sceptical about the statements that "90% of the code will soon be written by AI" and even though we might not be at that point, I am surprised how far LLMs have gotten and how useful they have become. I can hardly image developing software the "old" way where I actually write my code by hand, like I used back in the day. The frontier models have become so powerful that I find myself in moments of surprise, where the LLM actually thought of edge cases that I would have missed
- macwhisperer 4mo agothe HITL (human in the loop) is basically the single point...AI is a mirror.. it only "exists" when you talk to it.. much like your reflection in the mirror is only there when you're in view. models can never be self-improving because it can never have "self". it can only mirror the appearance of self. what's actually happening is "symbiotic group improvement". our brains are resonant.. for those of use who are brilliant, getting leverage with ai just means that our innovative ideas become louder and more physically real every day. eventually everything worth building will be built for free and made readily available.. no more "profiteering" its Jevons paradox "efficiency breakthrough -> effort reduces -> growth potential rises -> transformative gains happen"... some of us are in the "transformative phase".. others haven't seen the "breakthrough moment" yet, but they will soon.
- froh 4mo agoI didn't see this discussed more on hn yet: We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research—in collaboration with many others—and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.
- ausbah 4mo agothese ppl are so full of themselves
- froh 4mo agowhat do you mean? if feasible this proposal is imho exactly what we need: a pause to collectively think how we get all the benefits without the potential harms. to the non-techies around me I compare the boost of LLMs with the journey from slide rule via punch card driven computers through mainframes and PC to the smart phones of our days --- just within less than a decade, and we're at the transition from mainframe to PC with models that can produce reasonable output on a normal laptop. how about we check we're getting where we want to get to, before getting to some dystopic place where everyone wonders how we got _there_? I see nothing "full of themselves" in that.
- brazukadev 4mo agoWhen claude code removes React from its own code I'll believe that.
- leevilux 4mo agoWouldn't self-improvement mean that the LLM changes its neural network (i.e. the weights or layers or back propagation algorithm etc) or modify its training data?
- dibujaron 4mo agoIf it's actively building the next generation of itself, I'd say that counts. It's more like a parent raising their kid well than it is like a parent modifying their own mind, but the result is still that you have a better model in a year than you do now.
- nicogentile 4mo agoThe article seems nice and elegant but i dont get much of the point. The visual is super elegant but this is the kind of note where after 6 months we are going to see some shitty result and we are going to come back here and blame the IA. Hope doesnt happened.
- willXare 4mo ago[flagged]
- Rekindle8090 4mo ago[dead]