The current hiring process already does this over 2 days of 1-on-1 interviews with over half the faculty. Those faculty can ask whatever questions they'd like (

The current hiring process already does this over 2 days of 1-on-1 interviews with over half the faculty. Those faculty can ask whatever questions they'd like (talking about specific papers and methods are very common). But there are a lot of smooth talkers who aren't very productive, but can market themselves well in an interview. So looking at actual published work is also important. Given the choice between pure interview, versus counting the number of peer-reviewed publications in top conferences/journals, I'd most certainly pick the latter. In the worst case, you'd get someone who can get a lot of work done and published, versus someone who can talk about "big" problems and sound confident.