I think the unbiasedness is completely red herring here, but do I agree with the point on focusing on the tests separately and implementations separately. Idea

I think the unbiasedness is completely red herring here, but do I agree with the point on focusing on the tests separately and implementations separately. Ideally you'd want two completely different LLMs work on both. But I think the question is, how trustworthy are the LLM tests ? Will the human review of these take more time than writing of the how code ? I think for non-critical applications, it probably does not matter, but in the end I think people will be looking for some guarantees or confidence that the errors happen with frequency less than X%. And I don't think those exist now. And given the models change so frequently it's also hard to be sure if something was working fine yesterday whether it'll be today.