5 ms·

But it's hard to test creativity. We try to test programming skill because it's one of the few things (we think) we can test. How would you go about testing cr

But it's hard to test creativity. We try to test programming skill because it's one of the few things (we think) we can test. How would you go about testing creativity?

jacquesm 11y ago
> How would you go about testing creativity? Solve actual problems to which you currently do not have a solution and observe the process, not the solution.
- chetanahuja 11y ago
  That road leads to "how many golfballs in a minibus" type of puzzlers and we already have data now that shows that those things don't work. http://www.deathandtaxesmag.com/200732/google-admits-its-famous-job-interview-questions-were-a-complete-waste-of-time/ http://www.deathandtaxesmag.com/200732/google-admits-its-fam...
  - jacquesm 11y ago
    That's not an 'actual' problem, that's a fake problem. I mean actual as in 'problem that you actually have and that needs solving'.
    - tobltobs 11y ago
      It is difficult to find a problem which is solvable but still unsolved.
      - jacquesm 11y ago
        If that's true then you shouldn't be hiring people to begin with. After all, if you already know that your problems are too difficult then there is no hope whatsoever that your hire will be able to contribute. OTOH if you feel that the problems you are dealing with are solvable (and they had better be) then working together with your new prospective hire on some aspect of them will quickly give you insight into how they would go about this and whether or not they have something to contribute.
    - vlasev 11y ago
      I have to humbly disagree with you here. Estimation problems are quite important in science. When dealing with computers and algorithms, a quick back-of-the-envelope calculation can illuminate a lot of things. Maybe your algorithm is running slowly but a quick calculation shows it needs to be orders of magnitude faster. Etc. It's by no means a super necessary skill to have though. And certainly it makes for a rather silly interview question. I would guess it takes no more than half an hour to an hour to train yourself to do estimation problems.
      - jacquesm 11y ago
        Of course they are important in science. But that's not the kind of problem that you're likely to be hiring this particular (presumably a programmer) for. And if it is then estimation problems are exactly what you should be doing but if they are not then it would be a lot better to pick something related to the work at hand.
- cousin_it 11y ago
  I want a peer reviewed paper that answers these questions: What are the exact instructions for "observing the process"? Does it measure some underlying factor that's independent of the examiner? How do we disentangle the influence of youth, good looks, and verbal skills? I suspect the answers won't be flattering to those who think they can measure creativity today.
  - jacquesm 11y ago
    You could apply that list to any interview process. Essentially you are saying that there is no such thing as a fair interview process (and I'd agree with that).
    - cousin_it 11y ago
      Yeah, I dislike "isolated demands for rigor" as much as the next guy, but... no. Just no. There are tons of peer reviewed studies on interviewing. In particular, there are tons of studies on measuring IQ, which is the best known predictor of job performance for a wide variety of jobs. (Yes, better than work sample tests or anything else you might name in five minutes.) See Schmidt & Hunter 1998 for a pretty good meta-analysis. That's the standard I'd like to see when we talk about measuring creativity.

But it's hard to test creativity. We try to test programming skill because it's one of the few things (we think) we can test. How would you go about testing creativity?

jacquesm 11y ago

> How would you go about testing creativity? Solve actual problems to which you currently do not have a solution and observe the process, not the solution.

chetanahuja 11y ago
That road leads to "how many golfballs in a minibus" type of puzzlers and we already have data now that shows that those things don't work. http://www.deathandtaxesmag.com/200732/google-admits-its-famous-job-interview-questions-were-a-complete-waste-of-time/ http://www.deathandtaxesmag.com/200732/google-admits-its-fam...
- jacquesm 11y ago
  That's not an 'actual' problem, that's a fake problem. I mean actual as in 'problem that you actually have and that needs solving'.
  - tobltobs 11y ago
    It is difficult to find a problem which is solvable but still unsolved.
    - jacquesm 11y ago
      If that's true then you shouldn't be hiring people to begin with. After all, if you already know that your problems are too difficult then there is no hope whatsoever that your hire will be able to contribute. OTOH if you feel that the problems you are dealing with are solvable (and they had better be) then working together with your new prospective hire on some aspect of them will quickly give you insight into how they would go about this and whether or not they have something to contribute.
  - vlasev 11y ago
    I have to humbly disagree with you here. Estimation problems are quite important in science. When dealing with computers and algorithms, a quick back-of-the-envelope calculation can illuminate a lot of things. Maybe your algorithm is running slowly but a quick calculation shows it needs to be orders of magnitude faster. Etc. It's by no means a super necessary skill to have though. And certainly it makes for a rather silly interview question. I would guess it takes no more than half an hour to an hour to train yourself to do estimation problems.
    - jacquesm 11y ago
      Of course they are important in science. But that's not the kind of problem that you're likely to be hiring this particular (presumably a programmer) for. And if it is then estimation problems are exactly what you should be doing but if they are not then it would be a lot better to pick something related to the work at hand.
cousin_it 11y ago
I want a peer reviewed paper that answers these questions: What are the exact instructions for "observing the process"? Does it measure some underlying factor that's independent of the examiner? How do we disentangle the influence of youth, good looks, and verbal skills? I suspect the answers won't be flattering to those who think they can measure creativity today.
- jacquesm 11y ago
  You could apply that list to any interview process. Essentially you are saying that there is no such thing as a fair interview process (and I'd agree with that).
  - cousin_it 11y ago
    Yeah, I dislike "isolated demands for rigor" as much as the next guy, but... no. Just no. There are tons of peer reviewed studies on interviewing. In particular, there are tons of studies on measuring IQ, which is the best known predictor of job performance for a wide variety of jobs. (Yes, better than work sample tests or anything else you might name in five minutes.) See Schmidt & Hunter 1998 for a pretty good meta-analysis. That's the standard I'd like to see when we talk about measuring creativity.