3 ms·
You have to keep an eye on them, but they don't just make tests pass.
by cinntaile 4mo ago
You have to keep an eye on them, but they don't just make tests pass.
- kdjkskdndn 4mo agoClaude sonnet 4 (this time last year) did do this. It once made simulation if a test script passing. Literally a script that just echoed test names and then said pass.
- cinntaile 4mo agoChange happens fast, a year old model is pretty outdated. I'm sure it can happen, hence why I said to keep an eye out. Its main mode of operation is not to cook the tests however.
- yw3410 4mo agoHappened to me, 3 days ago - deleted some tests and flipped assertions after outlining that it wasn't to change any assertions. Our team was doing a similar task to move between test frameworks, and I had to do a git diff of hundreds of thousands of lines to try and work out where a test had disappeared to.
- LtWorf 4mo ago> 3 days ago Your fault. You should have used a model from 0.000005 seconds ago!
- cinntaile 4mo agoReading is difficult.
- LtWorf 4mo ago[flagged]
- cinntaile 4mo agoTry to put in a little more effort next time. It makes for better discussions instead of "jokes" that aren't even relevant.
- LtWorf 4mo agoI discuss with civilised people who are up for civilised discussions.
- cinntaile 4mo agoYou are the one ruining this discussion, it's worrying that you don't even realize it. I pointed out that models change quite a bit over time (I said more than that) and you ridicule my reply. "Your fault. You should have used a model from 0.000005 seconds ago!" Civilised. Please.
- customguy 4mo ago> Change happens fast, a year old model is pretty outdated. What change? That you should not fake the results of a test because that defeats the whole purpose of a test has been known before there were computers.
- cinntaile 4mo agoI don't know, the weather?