7 ms·

This is seriously f'd up. When I was hired by IBM years ago, they also had a test called the IPAT (Initial Programming Aptitude Test) but, guess what?, there wa

This is seriously f'd up. When I was hired by IBM years ago, they also had a test called the IPAT (Initial Programming Aptitude Test) but, guess what?, there was no programming in it! Instead, the test was designed to see if one had the basic reasoning and logic skills which would be necessary to solve a wide variety of programming-related problems -- not one stupid, specific problem in a particular language. That's just asinine. Making sure that developers have the "needle in a haystack" solution that you're looking for almost guarantees that you'll weed out the most talented folks -- the ones who are programming generalists that can reason about and solve any kind of programming problem.

ixtli 7y ago
An extremely underrated point: Hire. Generalists.
- chrisseaton 7y ago
  Sometimes you really do need particular skills and experience that take decades to learn. Otherwise you don't even know what you don't know and you don't know what mistakes are usually made in a given area.
  - mimixco 7y ago
    Not knowing what to do or what others did can be an advantage. Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked. In my own experience at IBM, I solved a bug that had been repeatedly opened over a series of 9 years by simply asking the question, "Is this number a hex or a decimal?"
    - chrisseaton 7y ago
      > by simply discarding all the known theories about how physics worked But he wasn't ignorant of them. He knew the existing work. He was extremely well read in physics and maths. He wasn't a generalist! If you get a group of random generalists to build your compiler, or your graphics engine, or your cryptography stack, they aren't going to know what they don't know, and it'd take them decades to come up to speed.
      - averros 7y ago
        As somebody who wrote all of the above (and a lot more) I call B.S. on this one. The amount of unique stuff in each subfield is not that great, and if you have a wide and solid base in different subfields, picking up the unique ideas and techniques isn't that hard or time consuming. But after a generalist has it, he is a lot more productive and creative because he often applies stuff from other subfields to come up with better/novel solution to the problem at hand - something a specialist cannot do.
    - jerf 7y ago
      "Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked." That's ludicrous nonsense. The primary breakthrough of the theory of relatively was taking what the math had been telling physicists about the universe for years seriously. In hindsight, everything necessary was there before Einstein, it just was very deeply assumed not to be reasonable. Relativity builds on numerous pre-existing foundations and is very much in the "standing on the shoulder of giants" tradition. https://en.wikipedia.org/wiki/Theory_of_relativity#Development_and_acceptance https://en.wikipedia.org/wiki/Theory_of_relativity#Developme... , first paragraph
      - mimixco 7y ago
        Yes, he knew them and discarded them. Adam Frank's excellent About Time does a good job of telling the story.
        chrisseaton 7y ago
        > Yes, he knew them and discarded them. That's the sign of an expert, not a generalist.
        jerf 7y ago
        If you took out the bits of relativity that were developed by others, you'd be left with not very much. Relativity without the Lorentz transform... I'm not even sure how one would resolve that. As I said, the vast majority of the insight was just taking what the math was already saying seriously. Einstein put it into a final form, and this is important and valid work, but it's not like he invented all the mathematical tools from scratch. Either you've misinterpreted Adam Frank's work, or it isn't excellent. I'll counter-cite Reflections on Relativity, which contains a lot of deep analysis of the mathematical history of relativity both before and after Einstein: https://mathpages.com/rr/rrtoc.htm https://mathpages.com/rr/rrtoc.htm which makes it quite clear both what he did, and did not do.
        mimixco 7y ago
        I'm no astrophysicist, though Frank is. The two concepts I'm referring to as being "discarded" were Newton's idea that motion (ie: time and space) exist in a fixed frame of reference. I think it's fair to say Einstein tossed that out wholeheartedly. The other idea Frank mentions that Einstein threw away was the idea of an æther that filled the voids in the universe. According to Frank, Einstein didn't like this idea so he just worked without it.
- edoceo 7y ago
  Am generalist, I hire generalists. My favourite feature is that they/we don't know what we're not supposed to do. So we just go, damn the torpedoes. Smart generalists GSD
- atemerev 7y ago
  1) there are not even enough generalists, good generalists are expensive, and bad generalists... you don’t want them. 2) as a generalist, I am quite OK with hiring specialists, if they are really good at what they do. I am a “full-stack”, which means that while I can do both frontend and backend, I can’t shoot React and CSS like a machine gun, nor can I squeeze the last bits of performance from JVM. I hire specialists to do that, after I got the ball rolling.
  - potta_coffee 7y ago
    I think this can really depend on the size of the business. At a small company, you probably can't afford a specialist for each area of responsibility. At a big company, you probably can't afford _not_ to have a specialist.
- Frost1x 7y ago
  As a generalist, I can't agree more. Most opportunities look for some high degree of specificity. For contract work or a pre-defined task, that makes sense but for companies that need a rich talent pool, I don't know why there appears to be little opportunity for generalists.
mywittyname 7y ago
So basically, an IQ test. I believe these run into legal grey areas because they have to be obviously relevant to the job in question, which likely leads to the tests getting progressively more domain-specific as time goes on. Programming does seem to be a field in which general intelligence is a big predictor of success. Whereas a field like management probably personality and general intelligence are more equally weighted.
- andrewprock 7y ago
  So basically, an IQ test. Well yes. And that is what the current crop of "programming puzzle" interviews essentially are; the next iteration of attempting to measure people on a cardinal scale instead of an ordinal scale. The key to remember is that evaluating on a cardinal scale is much more efficient than evaluating on an ordinal scale, so companies will do everything they can to transform the "secretary problem" into one where they measure absolute metrics instead of relative metrics. Now, whether those metrics align with business value is a separate issue. All evidence suggests that they are largely independent of business value.
- bluGill 7y ago
  HR told me I couldn't ask such questions without proving they were useful first. Basically a scientific experiment, have someone else ask and grade the questions, but those results were not considered for hire/nohire. Then 6 months after a bunch of people were hired we pull the results of the test and compare to their performance reviews. If the questions were a good predictor or future success we could use them on future interviews. The overhead of this was more than we were willing to pay so we went back to HR's allowed questions.
- fma 7y ago
  My previous company issued the Wondelic Test (https://en.wikipedia.org/wiki/Wonderlic_test https://en.wikipedia.org/wiki/Wonderlic_test) for everyone. I don't know the details but there was a minimum for different positions. There was also a personality portion and they were looking mostly for "A" type personalities, and a small subset of others. I don't have the "A" type personality, but I'd like to think the IQ portion made up for it LOL. Legality...not sure.
cyclonetiger 7y ago
From what I know, the IPAT still exists. Had to take it in 2015 for a co-op with IBM.
- mimixco 7y ago
  Googling around shows the consensus that the IPAT is considered one of the most difficult hiring tests out there, yet it contains no programming.
  - akhilcacharya 7y ago
    I mean, when I did it it was considered a formality. I don’t think anyone considers it a difficult barrier compared to getting into FNG etc.
matthewbauer 7y ago
That seems much worse than programming tests IMO. If I have a bachelor's degree, and you're asking me to retake the SAT, you're just insulting my qualifications. At least programming tests have some legitimate rationale. Let's not pretend IBM has any fucking idea what they are doing.
- mimixco 7y ago
  It wasn't an IQ test, and IBM definitely knew what it was doing in ways today's "move fast and break things" culture cannot fathom. Their software was and still is used in many mission-critical applications because the company was rigorously focused on quality. When was the last time you heard of a bug in mainframe software affecting anyone?
  - rhombocombus 7y ago
    While I agree with you conceptually, as a daily DataStage/InfoSphere user, there are plenty of buggy IBM products. I do think a general problem solving test is way more useful for vetting candidates than some stupid algo whiteboard assessment, and my company has been using something like that to great effect to recruit jr devs.
  - flamtap 7y ago
    Maybe you're talking about something else, but have you heard about IBM's infamous Phoenix Pay System? https://en.wikipedia.org/wiki/Phoenix_pay_system https://en.wikipedia.org/wiki/Phoenix_pay_system In short, it's costing Canada billions of dollars and has affected countless people's livelihoods.
- Franciscouzo 7y ago
  Intelligence Is the Best Predictor of Job Performance https://journals.sagepub.com/doi/abs/10.1111/1467-8721.ep10768746 https://journals.sagepub.com/doi/abs/10.1111/1467-8721.ep107... Thought in the US you will run into problems due to Griggs v. Duke Power Co.
  - andrewla 7y ago

ixtli 7y ago

An extremely underrated point: Hire. Generalists.

chrisseaton 7y ago
Sometimes you really do need particular skills and experience that take decades to learn. Otherwise you don't even know what you don't know and you don't know what mistakes are usually made in a given area.
- mimixco 7y ago
  Not knowing what to do or what others did can be an advantage. Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked. In my own experience at IBM, I solved a bug that had been repeatedly opened over a series of 9 years by simply asking the question, "Is this number a hex or a decimal?"
  - chrisseaton 7y ago
    > by simply discarding all the known theories about how physics worked But he wasn't ignorant of them. He knew the existing work. He was extremely well read in physics and maths. He wasn't a generalist! If you get a group of random generalists to build your compiler, or your graphics engine, or your cryptography stack, they aren't going to know what they don't know, and it'd take them decades to come up to speed.
    - averros 7y ago
      As somebody who wrote all of the above (and a lot more) I call B.S. on this one. The amount of unique stuff in each subfield is not that great, and if you have a wide and solid base in different subfields, picking up the unique ideas and techniques isn't that hard or time consuming. But after a generalist has it, he is a lot more productive and creative because he often applies stuff from other subfields to come up with better/novel solution to the problem at hand - something a specialist cannot do.
  - jerf 7y ago
    "Einstein developed his theory of relatively by simply discarding all the known theories about how physics worked." That's ludicrous nonsense. The primary breakthrough of the theory of relatively was taking what the math had been telling physicists about the universe for years seriously. In hindsight, everything necessary was there before Einstein, it just was very deeply assumed not to be reasonable. Relativity builds on numerous pre-existing foundations and is very much in the "standing on the shoulder of giants" tradition. https://en.wikipedia.org/wiki/Theory_of_relativity#Development_and_acceptance https://en.wikipedia.org/wiki/Theory_of_relativity#Developme... , first paragraph
    - mimixco 7y ago
      Yes, he knew them and discarded them. Adam Frank's excellent About Time does a good job of telling the story.
      - chrisseaton 7y ago
        > Yes, he knew them and discarded them. That's the sign of an expert, not a generalist.
      - jerf 7y ago
        If you took out the bits of relativity that were developed by others, you'd be left with not very much. Relativity without the Lorentz transform... I'm not even sure how one would resolve that. As I said, the vast majority of the insight was just taking what the math was already saying seriously. Einstein put it into a final form, and this is important and valid work, but it's not like he invented all the mathematical tools from scratch. Either you've misinterpreted Adam Frank's work, or it isn't excellent. I'll counter-cite Reflections on Relativity, which contains a lot of deep analysis of the mathematical history of relativity both before and after Einstein: https://mathpages.com/rr/rrtoc.htm https://mathpages.com/rr/rrtoc.htm which makes it quite clear both what he did, and did not do.
        mimixco 7y ago
        I'm no astrophysicist, though Frank is. The two concepts I'm referring to as being "discarded" were Newton's idea that motion (ie: time and space) exist in a fixed frame of reference. I think it's fair to say Einstein tossed that out wholeheartedly. The other idea Frank mentions that Einstein threw away was the idea of an æther that filled the voids in the universe. According to Frank, Einstein didn't like this idea so he just worked without it.
edoceo 7y ago
Am generalist, I hire generalists. My favourite feature is that they/we don't know what we're not supposed to do. So we just go, damn the torpedoes. Smart generalists GSD
atemerev 7y ago
1) there are not even enough generalists, good generalists are expensive, and bad generalists... you don’t want them. 2) as a generalist, I am quite OK with hiring specialists, if they are really good at what they do. I am a “full-stack”, which means that while I can do both frontend and backend, I can’t shoot React and CSS like a machine gun, nor can I squeeze the last bits of performance from JVM. I hire specialists to do that, after I got the ball rolling.
- potta_coffee 7y ago
  I think this can really depend on the size of the business. At a small company, you probably can't afford a specialist for each area of responsibility. At a big company, you probably can't afford _not_ to have a specialist.
Frost1x 7y ago
As a generalist, I can't agree more. Most opportunities look for some high degree of specificity. For contract work or a pre-defined task, that makes sense but for companies that need a rich talent pool, I don't know why there appears to be little opportunity for generalists.

mywittyname 7y ago

So basically, an IQ test. I believe these run into legal grey areas because they have to be obviously relevant to the job in question, which likely leads to the tests getting progressively more domain-specific as time goes on. Programming does seem to be a field in which general intelligence is a big predictor of success. Whereas a field like management probably personality and general intelligence are more equally weighted.

andrewprock 7y ago
So basically, an IQ test. Well yes. And that is what the current crop of "programming puzzle" interviews essentially are; the next iteration of attempting to measure people on a cardinal scale instead of an ordinal scale. The key to remember is that evaluating on a cardinal scale is much more efficient than evaluating on an ordinal scale, so companies will do everything they can to transform the "secretary problem" into one where they measure absolute metrics instead of relative metrics. Now, whether those metrics align with business value is a separate issue. All evidence suggests that they are largely independent of business value.
bluGill 7y ago
HR told me I couldn't ask such questions without proving they were useful first. Basically a scientific experiment, have someone else ask and grade the questions, but those results were not considered for hire/nohire. Then 6 months after a bunch of people were hired we pull the results of the test and compare to their performance reviews. If the questions were a good predictor or future success we could use them on future interviews. The overhead of this was more than we were willing to pay so we went back to HR's allowed questions.
fma 7y ago
My previous company issued the Wondelic Test (https://en.wikipedia.org/wiki/Wonderlic_test https://en.wikipedia.org/wiki/Wonderlic_test) for everyone. I don't know the details but there was a minimum for different positions. There was also a personality portion and they were looking mostly for "A" type personalities, and a small subset of others. I don't have the "A" type personality, but I'd like to think the IQ portion made up for it LOL. Legality...not sure.

cyclonetiger 7y ago

From what I know, the IPAT still exists. Had to take it in 2015 for a co-op with IBM.

mimixco 7y ago
Googling around shows the consensus that the IPAT is considered one of the most difficult hiring tests out there, yet it contains no programming.
- akhilcacharya 7y ago
  I mean, when I did it it was considered a formality. I don’t think anyone considers it a difficult barrier compared to getting into FNG etc.

matthewbauer 7y ago

That seems much worse than programming tests IMO. If I have a bachelor's degree, and you're asking me to retake the SAT, you're just insulting my qualifications. At least programming tests have some legitimate rationale. Let's not pretend IBM has any fucking idea what they are doing.

mimixco 7y ago
It wasn't an IQ test, and IBM definitely knew what it was doing in ways today's "move fast and break things" culture cannot fathom. Their software was and still is used in many mission-critical applications because the company was rigorously focused on quality. When was the last time you heard of a bug in mainframe software affecting anyone?
- rhombocombus 7y ago
  While I agree with you conceptually, as a daily DataStage/InfoSphere user, there are plenty of buggy IBM products. I do think a general problem solving test is way more useful for vetting candidates than some stupid algo whiteboard assessment, and my company has been using something like that to great effect to recruit jr devs.
- flamtap 7y ago
  Maybe you're talking about something else, but have you heard about IBM's infamous Phoenix Pay System? https://en.wikipedia.org/wiki/Phoenix_pay_system https://en.wikipedia.org/wiki/Phoenix_pay_system In short, it's costing Canada billions of dollars and has affected countless people's livelihoods.
Franciscouzo 7y ago
Intelligence Is the Best Predictor of Job Performance https://journals.sagepub.com/doi/abs/10.1111/1467-8721.ep10768746 https://journals.sagepub.com/doi/abs/10.1111/1467-8721.ep107... Thought in the US you will run into problems due to Griggs v. Duke Power Co.
- andrewla 7y ago