3 ms·
patterns that may help increase subjective perception of reliability from non-deterministic text generators trained on the theft of millions of developer's work
by Madmallard 7mo ago
patterns that may help increase subjective perception of reliability from non-deterministic text generators trained on the theft of millions of developer's work for the past 25 years.
- logicprog 7mo agoI think it's nonsensical to insist that it would only be a subjective improvement. The tests either exist and ensure that there aren't bugs in certain areas, or they don't. The agent is either in a feedback loop with those tests and continues to work until it has satisfied them or it doesn't.
- deleted 7mo ago[deleted]
- Madmallard 7mo agoThat sounds like a very specific implementation strategy related to TDD
- logicprog 7mo agoRed-Green TDD is one of the main "agent patterns" Simon proposes, so it seemed relevant. Also, the same thing applies to feedback loops with compilers and linters as well: they provide objective feedback that then the AI goes and fixes, verifiably resolving the feedback. Even with less verifiable things like using specifications, the fact that it relies on less objective grounding metrics doesn't mean there's no change in the model's behavior. I'm sure if you looked at the code that a model produced and the amount of intervention necessary to get there for a model that was asked to produce something without a specification versus with one, you would definitely see an objective difference on average. We're already getting objective studies regarding AGENTS.MD