On a related topic, I've been thinking about the interviewing problem and had an insight that is probably blindingly obvious to everybody else. In engineering

On a related topic, I've been thinking about the interviewing problem and had an insight that is probably blindingly obvious to everybody else. In engineering terms, interviewing is benchmarking. Making an effective benchmark is somewhere between very hard and intractable. A benchmark has to be... * Difficult enough to clearly differentiate between good and bad yet be simple enough to "run" in 5-30 minutes. * Repeatable, yet not beatable. Remember the compilers that recognized drystones and compiled them to the final answer? That is what google does to interview "trick" questions. * Applicable: drystones are totally worthless if your algorithm is dominated by floating point calculations. Having said that, "comes up to speed quickly" is the most applicable metric for a candidate, but is awfully hard to benchmark in a limited interview. Anecdote: I had lunch today with the engineer that hired me on my first "real engineering" job (I was a freshman in college at the time). He commented that I was "the smartest guy he knew, until he met my younger brother." My brother is several years younger... Bob was working for a different company and my brother applied for a summer job there. Bob's take, and the other folks at the company's take, on my brother's resume' was "meh". Bob told them "never mind his resume, hire him. Just do it! Do. It." Moral of the anecdote: benchmarks are poor stand-ins for real life. Unfortunately, I don't have a better way. :-/