That's why I'll never accept automated systems for determining things that need a human with context. "We used our black box and it says you're guilty." - what

That's why I'll never accept automated systems for determining things that need a human with context. "We used our black box and it says you're guilty." - what do you do against that? There's nothing to argue and inevitably you'll end up facing some tribunal where a group of people are going to decide your ultimate guilt. What if they use their own black box?