3 ms·
I built a tiny GitHub Action that fails PRs when LLM output breaks contracts. No live model calls in CI... it runs on recorded fixtures. What it does: Determi
by geminimir 1y ago
I built a tiny GitHub Action that fails PRs when LLM output breaks contracts.
No live model calls in CI... it runs on recorded fixtures.
What it does:
Deterministic checks: JSON schema, regex, list/set equality, numeric bounds, file diff
Snapshots + regression compare
Cost budget gate
PR comment + HTML report
Try it in a minute: copy the Quick Start, open a PR, see red → fix → green.
Links:
Marketplace: https://github.com/marketplace/actions/promptproof-eval https://github.com/marketplace/actions/promptproof-eval
Demo repo: https://github.com/geminimir/promptproof-demo-project https://github.com/geminimir/promptproof-demo-project
Sample report: https://geminimir.github.io/promptproof-action/reports/before.html https://geminimir.github.io/promptproof-action/reports/befor...
Looking for blunt feedback... Was onboarding smooth? Anything missing you expected? Is the report clear enough to make this a required check?