3 ms·
Yet it still fumbles even when limiting context. Asked it to spot check a simple rate limiter I wrote in TS. Super basic algorithm: let one action through ever
by mattgreenrocks 8mo ago
Yet it still fumbles even when limiting context.
Asked it to spot check a simple rate limiter I wrote in TS. Super basic algorithm: let one action through every 250ms at least, sleeping if necessary. It found bogus errors in my code 3 times because it failed to see that I was using a mutex to prevent reentrancy. This was about 12 lines of code in total.
My rubber duck debugging session was insightful only because I had to reason through the lack of understanding on its part and argue with it.
- EnPissant 8mo agoTry again with gpt-5.3-codex xhigh.
- Capricorn2481 8mo agoTry again with Opus 4.5 Try again with Sonnet 4 Try again with GPT-4.1 Here I thought these things were supposed to be able to handle twelve lines of code, but they just get worse.
- xigoi 8mo agoThe goalposts have been moved so many times that they’re not even on the playing field.
- EnPissant 8mo agoNahh, just trying to make it concrete. I could instead just ask which model they used instead.
- Kiro 8mo agoI would love to see that code.
- seunosewa 8mo agoOnce you've gone through that, you might want to ask it to codify what it learned from you so you don't have to repeat it next time.