4 ms·

"Trivia questions" are not useless. I've put out job ads only to receive hundreds of applications from all over the world, many not even in English or without e

by crdb 10y ago

"Trivia questions" are not useless. I've put out job ads only to receive hundreds of applications from all over the world, many not even in English or without even a word accompanying a CV. Of the "proper" applicants, I've then had to deal with dozens of candidates that claimed a competence they did not have. And guess what - that's a lot of effort and time which is taken away from the business. In my case, that's a big opportunity cost with salaries burning a deeper hole for every minute that they are not offset by client time. I suspect it's much worse for famous organisations. So now I ask a dozen questions by email that should take 5 minutes to answer in order to filter out both these groups, and that cover many basic CS concepts. Works really well, even if it's a bit annoying for candidates and probably turns off many vs just uploading your CV for HR to auto-scan for keywords. For example, if I'm hiring someone to do data modelling work and they tell me it's called the relational model because of the relations between tables, it's unlikely they'll know how to unfold EAV antipatterns. And if you have actual machine learning experience, it's unlikely you'll be stumped by my asking what high leverage means.

marmaduke 10y ago
Have "actual machine learning" experience, no idea what "high leverage" means.
- crdb 10y ago
  I'm referring to leverage in statistics [1], and particularly how high leverage observations may affect your model and how people deal with them. Clearer? For context: one of the candidates I interviewed told me that the best way to pick a model was to pick the model that would have the highest R squared when fitted to the whole dataset. I asked him about overfitting and he didn't know what I was talking about. Same guy whose CV showed 4 years in a research lab doing stats [2]. A lot of people are just going through the motions. [1] https://en.wikipedia.org/wiki/Leverage_(statistics) https://en.wikipedia.org/wiki/Leverage_(statistics) [2] reposting for fun: http://4.bp.blogspot.com/-IJo_Tkw95-o/VANnTdCFgHI/AAAAAAAADVY/3cAV6UEWiPA/s1600/Screen%2BShot%2B2014-08-30%2Bat%2B6.08.23%2BPM.png http://4.bp.blogspot.com/-IJo_Tkw95-o/VANnTdCFgHI/AAAAAAAADV...
  - mlashcorp 10y ago
    Yeah... Have actual machine learning experience as well, never heard the term leverage as applied here. Sometimes the nomenclature can confuse people, and I guess the original point about more in depth interviewing processes was related to discerning whether the fundamental knowledge and team fit are there, not whether the candidate is aware of the particular subset of CS trivia the interviewer is currently interested in. I say this while admiting that I've made this mistake in the past while interviewing candidates for CS positions...
    - crdb 10y ago
      What do you call high leverage observations then?
      - deleted 10y ago
        [deleted]
      - p4wnc6 10y ago
        This is similar to "multicollinearity". I have two graduate degrees in applied statistics and machine learning, and did math as an undergrad, but I did not ever even hear the term "multicollinearity" until I got my first job after grad school in a place that used a lot of regression models. I had studied "spurious correlation" in some machine learning courses and in some research, but almost all of the methods I ever studied or worked on were meant to use data-driven methods to account for spurious correlation, or dimensionality reduction to find subspaces in which the data's natural structure was best preserved without correlation-caused redundancy. Literally none of them ever even mentioned the phrase "multicollinearity" -- which seems to be more popular to people coming from classical statistics or econometrics programs. If I had been asked to describe "what is multicollinearity" during the interviews, I would have been rejected, yet after they told it to me, I learned about it in about 2 hours on Wikipedia, and within two months I had actually done a research project for them where I showed them how you could use ISOMAP or randomized PCA to effectively handle regression multicollinearity better than their ad hoc covariate-averaging techniques. This kind of trivia stuff is useless for hiring. All it does is let the interviewer feel smart and tout their favorite particular buzzwords to see if the candidate is in the same "club" as them. I can imagine some diehard frequentist caring a shit ton about "consistent estimator" or various special tests like Kolmogorov-Smirnoff or F-test or likelihood ratio test. If you say, "I always do Bayesian stats... those test thingies are stupid junk" then it means "This person is not in the Frequentist Club, let's reject their lame ass and tell them it's cause they are 'just not a good fit.'" Same would be true in reverse if some Bayesian diehard interviews a frequentist person. Same for someone who sees deep learning as the hammer for every nail. Etc. etc. People don't want to admit they treat these things kind of like baseball trading cards, and it's more about your cool-kid status than your actual tech skill.
- x0x0 10y ago
  Everyone learns different vocab depending on what discipline you learned about stats/ml. Leverage is a very standard stats term, but I've never seen it used in an ml context.
pigscantfly 10y ago
I've only given a couple of hiring interviews so far in my short career, but I've found specific, factual questions to be vital insofar as they truthfully clarify candidates' knowledge levels in their specific areas of expertise. People will generally say 'yes' when asked if they are familiar with something during an interview, even if their knowledge is tenuous, and as you said, resumes are not always totally accurate. I understand the negative reactions these types of questions get online - it's easy to imagine oneself forgetting something in the heat of the moment or missing out on an opportunity for a silly reason - but a qualified candidate should be able to give a plausible answer or ask a cogent clarifying question. If anyone has a better method for filtering out the noise people employ to pitch themselves, I'd be happy to hear it, but I haven't seen one yet.

by crdb 10y ago

marmaduke 10y ago

Have "actual machine learning" experience, no idea what "high leverage" means.

crdb 10y ago
I'm referring to leverage in statistics [1], and particularly how high leverage observations may affect your model and how people deal with them. Clearer? For context: one of the candidates I interviewed told me that the best way to pick a model was to pick the model that would have the highest R squared when fitted to the whole dataset. I asked him about overfitting and he didn't know what I was talking about. Same guy whose CV showed 4 years in a research lab doing stats [2]. A lot of people are just going through the motions. [1] https://en.wikipedia.org/wiki/Leverage_(statistics) https://en.wikipedia.org/wiki/Leverage_(statistics) [2] reposting for fun: http://4.bp.blogspot.com/-IJo_Tkw95-o/VANnTdCFgHI/AAAAAAAADVY/3cAV6UEWiPA/s1600/Screen%2BShot%2B2014-08-30%2Bat%2B6.08.23%2BPM.png http://4.bp.blogspot.com/-IJo_Tkw95-o/VANnTdCFgHI/AAAAAAAADV...
- mlashcorp 10y ago
  Yeah... Have actual machine learning experience as well, never heard the term leverage as applied here. Sometimes the nomenclature can confuse people, and I guess the original point about more in depth interviewing processes was related to discerning whether the fundamental knowledge and team fit are there, not whether the candidate is aware of the particular subset of CS trivia the interviewer is currently interested in. I say this while admiting that I've made this mistake in the past while interviewing candidates for CS positions...
  - crdb 10y ago
    What do you call high leverage observations then?
    - deleted 10y ago
      [deleted]
    - p4wnc6 10y ago
      This is similar to "multicollinearity". I have two graduate degrees in applied statistics and machine learning, and did math as an undergrad, but I did not ever even hear the term "multicollinearity" until I got my first job after grad school in a place that used a lot of regression models. I had studied "spurious correlation" in some machine learning courses and in some research, but almost all of the methods I ever studied or worked on were meant to use data-driven methods to account for spurious correlation, or dimensionality reduction to find subspaces in which the data's natural structure was best preserved without correlation-caused redundancy. Literally none of them ever even mentioned the phrase "multicollinearity" -- which seems to be more popular to people coming from classical statistics or econometrics programs. If I had been asked to describe "what is multicollinearity" during the interviews, I would have been rejected, yet after they told it to me, I learned about it in about 2 hours on Wikipedia, and within two months I had actually done a research project for them where I showed them how you could use ISOMAP or randomized PCA to effectively handle regression multicollinearity better than their ad hoc covariate-averaging techniques. This kind of trivia stuff is useless for hiring. All it does is let the interviewer feel smart and tout their favorite particular buzzwords to see if the candidate is in the same "club" as them. I can imagine some diehard frequentist caring a shit ton about "consistent estimator" or various special tests like Kolmogorov-Smirnoff or F-test or likelihood ratio test. If you say, "I always do Bayesian stats... those test thingies are stupid junk" then it means "This person is not in the Frequentist Club, let's reject their lame ass and tell them it's cause they are 'just not a good fit.'" Same would be true in reverse if some Bayesian diehard interviews a frequentist person. Same for someone who sees deep learning as the hammer for every nail. Etc. etc. People don't want to admit they treat these things kind of like baseball trading cards, and it's more about your cool-kid status than your actual tech skill.
x0x0 10y ago
Everyone learns different vocab depending on what discipline you learned about stats/ml. Leverage is a very standard stats term, but I've never seen it used in an ml context.

pigscantfly 10y ago

I've only given a couple of hiring interviews so far in my short career, but I've found specific, factual questions to be vital insofar as they truthfully clarify candidates' knowledge levels in their specific areas of expertise. People will generally say 'yes' when asked if they are familiar with something during an interview, even if their knowledge is tenuous, and as you said, resumes are not always totally accurate. I understand the negative reactions these types of questions get online - it's easy to imagine oneself forgetting something in the heat of the moment or missing out on an opportunity for a silly reason - but a qualified candidate should be able to give a plausible answer or ask a cogent clarifying question. If anyone has a better method for filtering out the noise people employ to pitch themselves, I'd be happy to hear it, but I haven't seen one yet.