This is not just a problem for evaluating programmers but for ALL job candidates. My father-in-law made a wise suggestion, as a company, to always be in hiring

This is not just a problem for evaluating programmers but for ALL job candidates. My father-in-law made a wise suggestion, as a company, to always be in hiring mode. You don't want to hire someone because you need them at that very moment but always be hiring so you have redundancy.