4 ms·
> Not my experience. It excels in existing codebases too. Why don't you prove it? 1. Find an old large codebase in codeberg (avoiding the octopus for obvious
by netdevphoenix 8mo ago
> Not my experience. It excels in existing codebases too.
Why don't you prove it?
1. Find an old large codebase in codeberg (avoiding the octopus for obvious reasons)
2. Video stream the session and make the LLM convo public
3. Ask your LLM to remove jQuery from the db and submit regular commits to a public remote branch
Then we will be able to judge if the evidence stands
- aurareturn 8mo agoI don't have to prove it. I do it every single day at work in a real production codebase that my business relies on. And I don't remove jQuery every day. Maybe the OP is right that Opus 4.6 sucks at removing jQuery. I don't know. I've never asked an AI to do it. The moment you point it at a real, existing codebase - even a small one - everything falls apart. This statement is absolutely not true based on my experience. Codex has been amazing for me at existing code bases.
- netdevphoenix 8mo agoExtraordinary claims require extraordinary evidence. "Works on my machine" ain't it.
- aurareturn 8mo agoIs it an extraordinary claim that Opus 4.6 or GPT 5.3 works amazing on existing code bases in my experience? That's funny. I feel like it's the opposite. Claiming that Opus 4.6 or GPT 5.3 fails as soon as you point them to an existing code base, big or small, is a much more extraordinary claim.
- simonw 8mo agoWhat are the obvious reasons?
- netdevphoenix 8mo agoI thought it would be obvious: OpenAI has used repos on GitHub as training data. Would be like testing someone using a past paper publicly available. Are you planning on carrying out the experiment? Regardless of the outcome, it would be of value to developers.
- simonw 8mo agoWhy wouldn't they train on Codeberg too? It's pretty hard to block automated uses of "git clone".
- netdevphoenix 8mo agoWhy would they? Github has 28 million public repos, Codeberg only hit 300k last year. Anyway, Codeberg was just a placeholder for 'repo source _less_ likely to be in their training data'. Codeberg was quick candidate for a place to find a big old codebase with non-sensitive data. It is indeed hard but the guys at Codeberg are certainly an order of magnitude better than Github as they opted out of the main AI crawlers, regularly block IPs known to belong to AI startups and they allow you to make your repos only be accessible to logged in users. You seem be going on a tangent, here. Main point was about performing a well documented test anyway.
- simonw 8mo agoMy question about the "obvious" thing was genuine - it wasn't obvious to me.