I think the "fake product" interview is a great format, but it can be done poorly if the setup is unrealistic. I've been on the receiving end of a couple of ver

I think the "fake product" interview is a great format, but it can be done poorly if the setup is unrealistic. I've been on the receiving end of a couple of very bad attempts. In one, which I have described here before, an interviewer asked me to design a system for selling concert tickets. After half an hour of confused back and forth, it emerged that they wanted a network service that replied to requests with unpredictable random integers, which did not repeat, unless the service rebooted and then it was okay if there was an accidental repeat. Needless to say, I started with a lot of assumptions based on the idea of selling concert tickets, and I don't think it reflected poorly on me that it took so long to figure out that the real problem they were judging me by was so radically different from the one they asked me to solve. I had another interview recently which had a similar wrinkle. The interviewer asked for a system to solve a very particular problem, which because of the particulars of the problem would involve receiving very small amounts of data from systems in the field. The interviewer asked multiple times about the cost of data ingress and data storage, and each time I deflected them, saying that it wasn't significant compared to other costs in the system. I was so focused on solving the problem as stated that it didn't dawn on me until later that the interviewer likely had a different problem in mind in which the data was much bigger, and they were evaluating whether I could be sensitive to those costs and design an economical system. Interviews like these are very frustrating, and I don't think they will lead to great hires. You will end up hiring people who solve the problem you have in mind even though you told them to solve a different problem. These programmers will have the same biases and preoccupations as you, which might make them easy to incorporate into your team, but they will probably have the same blind spots as well, which means your team will be less prepared to solve new problems in the future.