5 ms·
Yes! We'll see how it goes, but the product in question (Conveyor) is a downloadable tool that's got deliberately unobfuscated bytecode in it, with lots of det
by mike_hearn 1mo ago
Yes!
We'll see how it goes, but the product in question (Conveyor) is a downloadable tool that's got deliberately unobfuscated bytecode in it, with lots of detailed logging. AI is perfectly capable of reverse engineering it and in fact this bug report contained such a reversing. So even if someone tricks it into revealing source code or similar, they won't get anything that isn't already obtainable via other methods. This isn't a SaaS where security through obscurity might conceivably help, or where the codebase might contain credentials by mistake.
It's a developer tool and this level of trust helps customers debug their own problems quickly. If someone wants to break the law, they'll get a legal answer, but it's never been a problem.
The bot in question cannot write to master though, only open up pull requests from its own isolated repository.
It's a bet on modern models being more resistant to confusion attacks than they were before. The harness setup also makes it very clear to the model where input comes from. This might be a bad bet, but if it's not, then it's helpful for customers to get help right away.