3 ms·

Thanks for your answer. With mocking, what happens if someone writes their if condition as: (if randrange(1000) < 1000*p) or worse, (if randrange(1000) > (1000-

Thanks for your answer. With mocking, what happens if someone writes their if condition as: (if randrange(1000) < 1000*p) or worse, (if randrange(1000) > (1000-1000p). I am trying to write tests to autograde some HW assignments, and so I really have to think adversarially and imagine my students writing the weirdest yet correctly working functions. I really haven't come up with a better answer than checking the statistics of the returned value by running the function a whole bunch of times, like you suggest.

mlthoughts2018 6y ago
“How do I test all possible equivalent definitions of this function” is probably not practically answerable for this. What if they wrote this? def biasedcoin(n, p): for answer in answer_key[n][p]: yield answer Or this: def biasedcoin(n, p): while True: mine_bitcoin_on_teachers_computer()
madsbuch 6y ago
Sure, this is simple! You provide them with a definition written in Coq and make them implement a proof that their implementation satisfies that definition. Use probabilistic couplings to make sure the output of the biased coin function adheres to specified distribution. You can easily implement a grader that checks the the proof.