This reminds me of the recent discussion about Baidu's expulsion from an image recognition competition. They submitted too frequently, causing their machine le

This reminds me of the recent discussion about Baidu's expulsion from an image recognition competition. They submitted too frequently, causing their machine learning algorithm to solve specifically for the test, rather than for general purpose image recognition. In your proposed scenario, knowing the CA driver's test constraints, the engineers could easily (and perhaps even inadvertently) be able to design a driver AI capable of passing the test with flying colors, yet fail horribly once faced with real-world driving. That's not to say that's actually what would happen, only to say that the test would actually be rather useless to certify a self-driving car.