3 ms·
Comments like this don't fill me with confidence: https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554ae28f8876f3f976862f8a/internal/judge/llm_judge.go#L
by simonw 7mo ago
Comments like this don't fill me with confidence: https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554ae28f8876f3f976862f8a/internal/judge/llm_judge.go#L106-L110 https://github.com/brexhq/CrabTrap/blob/4fbbda9ca00055c1554a...
// The policy is embedded as a JSON-escaped value inside a structured JSON object.
// This prevents prompt injection via policy content — any special characters,
// delimiters, or instruction-like text in the policy are safely escaped by
// json.Marshal rather than concatenated as raw text.
- Jerem-6ix 7mo ago[dead]
- samcollins 7mo agoWhy do you say that? I thought this pattern was well established, or are you aware of known issues with it?
- simonw 7mo agoIt doesn't work. You can't trust LLMs to 100% reliably obey delimiters or structure in content. That's why prompt injection is a problem in the first place.
- okwhateverdude 7mo agoRobots struggle with syntax-in-syntax. Really easy to confuse them when asking it to write a SQL query that targets a JSON column but it must respond with a JSON envelope so the harness can parse the result. Lots of escaping that needs to happen. Deeply nested structures in JSON also end up with foibles like missing a ] or } in a string of }}]}]}. Aside from the prompt injection possibility, just the result being straight up broken and requiring another LLM call is tokens flushed.
- frumplestlatz 7mo agoWell-established where and amongst who, exactly? Is it seriously a common belief that this prevents prompt injection? That would be more than a little alarming.