how productive/useful a programmer is is, frankly, a slippery concept. Especially since things like attractiveness, gender, race and so on can be shown to affe

how productive/useful a programmer is is, frankly, a slippery concept. Especially since things like attractiveness, gender, race and so on can be shown to affect such assessments unconsciously. Seriously, we live in a world where height is correlated with salary. http://www.apa.org/monitor/julaug04/standing.aspx http://www.apa.org/monitor/julaug04/standing.aspx My degree is in psychology, not computer science, so let me see if I can give you a general answer with a specific example of how you might work to figure out this. Short Answer: Research. Long Answer: You'd start with a hypothesis, and you'd test it and then improve your testing based on the results. You try to nail down what you can, and eventually you have an internally consistent system, which is about the best you can do. For example, you might say, "Knowledge of Big O should be positively correlated with job performance." First you come up with an assessment of knowledge of Big O. A test with variable questions that assesses such knowledge. You probably test the assessment by comparing how well a person does if they take the test twice (using Pearson correlation, or maybe Spearman), to see if you have test-retest reliability, and dividing the questions in half randomly, and seeing if it's internally consistent. So you know your test appears to be reliable... you can't necessarily say that it's measuring Big O perfectly, but you can argue that it's measuring Big O knowledge to some degree. You'd start out with a big company, or multiple companies, and assess the programmers there, then use something like relative income (controlling for company, gender, the afore mentioned height, whatever) to see if you are right. Or use another metric, git commits, lines of code. You are dealing with a system where nothing is perfectly correlated with being a great programmer, where how highly you assess a person may be based on characteristics that don't earn your company any money but which you are unconsciously biased towards, but you can still differentiate based on something. Let's say that yes, it seems like better programmers know big O better. Then you might assess big O skills of new hires, and track them to see how successful they are. Maybe you find out, yes, those that knew big O better did better, which is important, because it might be that people that did better then end up needing to know more about big O. Maybe you might give some number of randomly selected new employees a course on big O, and retest all of them afterwards to measure improvement, and then see if the ones who were trained do better then the untrained ones. At that point, it's probably worth while to test people's big O skills before hiring them. So you take your big O test and improve it so that people can't memorize the test, and start testing people. It might also be worth giving new hires a course on big O, if you can show that such training improves their value in the company. Is Big O everything somebody needs to know? Probably not. But it's still more information then you had, and you can maybe get a sense of how it's useful. Conversely you might find that people that know Big O better do worse as employees. Or that it helps certain programmers more then others... for example people making graphically intensive games or developing databases may need to know more about it then people who work with stuff that I/O bound.