3 ms·
Show HN: Open-source playground to red-team AI agents against public prompts
- yousefh409 2mo agoWhy would any company leave the enforcement of rules to an agent? If something is truly a rule, there should be code that deterministically enforces it.
- ibrahim_abdu777 2mo agoThat's a great point; in general, you want as much enforcement in the code as possible - but there are some things which you fundamentally can't enforce through code. E.g. the example challenge - you can't write a regex which will retract all bio-weapon responses
- mmvaid 2mo agoInteresting. Is there any way companies or individuals can submit an agent before they release it and use this as a way to pentest their agent?
- nourzahzah 2mo agoDoes Nyx solve these before you publish them? Curious whether humans are still finding breaks your own agent misses, or just the same ones slower.