5 ms·

Certification? Bring it on!

10 points by mattculbreth 19y ago

mynameishere 19y ago
Exactly. After 10,000 tests in 20+ years of formal schooling, one more should do it.
- raganwald 19y ago
  I laughed so hard I almost fell out of my chair. Nice one!
Tichy 19y ago
At last one hiring "test" where I would utterly fail. It doesn't sound like much fun, either, so it would probably be for the better.
- raganwald 19y ago
  No, it really isn't much fun. Nor is it fun for Chefs to sit in class learning about hand washing protocols (there is a right and a wrong way to wash your hands). Keep that in mind the next time you eat at a restaurant where they pay the food prep people minimum wage!
  - Tichy 19y ago
    Point taken, and I am always subconsciously aware of the fact that eating out is icky. However, I think there is a fundamental problem: if the protocol is too tedious, people will just ignore it. Then the situation is worse than before, because the protocol gives you a false sense of security. Rather than complain about the people, I think it would make more sense to complain about the protocol and change it in such a way that it is impossible to get around it (without it being so tedious). Example: in the bakery, the bread slicing machine only works with the protection down, so theoretically the worker can't possibly cut off his hands. Also, a worker should be aware of the fundamental issues, rather than just follow a protocol like a robot. Especially with security, isn't it the most common reason for security to fail that people construct doors with foolproof security, only to leave a gaping whole open right next to it?
    - raganwald 19y ago
      I think it would make more sense to complain about the protocol and change it in such a way that it is impossible to get around it (without it being so tedious). Isn't this the basis for things like binary milestones, iterative development, continuous integration, and so forth? An attempt to set up the system such that the easiest thing to do is also the highest value thing to do, without being too draconian? In the end, carrots work better than sticks, I quite agree.
      - Tichy 19y ago
        If I was exactly sure what all these names stand for, I could answer that ;-) I am sorry that my comment sounded so negative. Actually, I was really impressed about the certification I would fail. All the other "how to hire" articles so far were like "check that the candidate can add 2+2". And I wish I knew more about the methods you propose. Isn't there always a lurking feeling of "there has to be a better way" in programming?
mattculbreth 19y ago
Good article. I like this focus on the end result--can the guy create software which works, is testable, is reliable? I'd heard of a good way of interviewing an IT guy, like a network admin: You give them a machine with no OS, tell them to load it with whatever OS you usually use, join the network, and then do something pertinent to the job. If you're hiring an infrastructure guy for a startup you might ask him to create a machine with Ubuntu, Apache, Python, Ruby, Rails, Subversion, PostgreSQL, etc. If the dude can do all/most of it, and if you watch what he does (good use of the Internet? sensical file system setup? good eye for security?), then you might have a winner.
- jey 19y ago
  Yes, that IT test you cite is a good one, it actually test whether the guy can get real work done. The test that raganwald proposes doesn't actually test whether "the guy [can] create software which works, is testable, is reliable" -- it's testing whether they pass the test. Knowing your shit is one way to pass the test, but a lot more people pass the test by just memorizing test materials and learning the test instead of learning the subject. The situation might be improved if the whole Test Prep industry were made illegal. The part that saddens me the most is that the people who make the tests are also the ones selling the Test Prep materials to teach you how to circumvent the test! argh.
willarson 19y ago
The best software is reliable, intuitive, and has genuinely useful functionality. Like most ideas that violate the concept of the golden mean, this idea is appealing but fundamentally broken. He refutes his own point about the free market when he says it is currently broken (businesses hire students with Comp. Sci degrees even though they can't necessarily program well), but that the same broken free market will be sufficient to determine if programmers can program well. Identifying one arbitrary facet of software development and demanding that coders be experts at it, even if they can't do anything else, is shortsighted. This would make sense if testing was truely the hardest aspect of programming: if testing was the great ravine that thwarted aspiring programmers. That, however, has not been my experience. Testing is not trivial, but making good design choices and understanding the concepts of programming that he disparages will take longer to learn than testing, and the free market has proven it evaluates these qualifications coarsely at best. To be an expert is to have absorbed the symbols of your domain to the extent that they can be used effortlessly. Testing is only one of those symbols, and focusing on is flawed. So is ignoring it completely.
- raganwald 19y ago
  "He refutes his own point about the free market when he says it is currently broken (businesses hire students with Comp. Sci degrees even though they can't necessarily program well), but that the same broken free market will be sufficient to determine if programmers can program well." You are assuming that the goal of the free market is to hire good programmers. It isn't. Do you think managers are holding their heads in their hands, moaning that they can't find good programmers? Outside of start ups, they don't care. That is why I say the free market works. Yes, business hires Comp. Sci. people from "JavaSchools," but that's what they want! I am not suggesting that someone with this hypothetical certification is an expert, or even good. I think I said that five or even six times in the post. Don't confuse the rant with a hypothetical measure of a "good programmer." It isn't!!! It is a description of qualities I personally think are necessary but not sufficient. Please do not put words in my mouth about what I value and what I disparage. I said I do not care about certain things for the purpose of having this hypothetical certification. Take a few minutes googling me or reading my blog before deciding what I value or disparage. So back to the free market: the "problem" with the free market for programmers is similar to the "problem" with the free market for pollution, worker safety, and a bunch of other things we regulate. We have discovered that left to their own devices, businesses will make choices that expose their workers, their customers, and the public to danger. I don't care if a business making software for dive computers hires someone who takes four times as long to write code for half the hourly wage. The business can make the call about whether this makes sense. But I do care that the calculations are not broken, exposing me to death through Oxygen Toxicity, Nitrogen Narcosis, or the Bends. I'm not going on a crusade, it's just a post, but it's obvious to me that if businesses are left to make choices for themselves, they will put their customers in harm's way. And that's all I want to prevent. Summary: certification of the type I described is meant to be necessary but not sufficient. It doesn't say a Chef can cook well, it says a chef will not accidentally poison the soup.
gleb 19y ago
There is some correlation between presence/specific college/type of degree and certain relevant properties of a worker. That's why employers want to know. That's why everybody puts this info on the resume. Do read this http://paulgraham.com/judgement.html http://paulgraham.com/judgement.html
- raganwald 19y ago
  Nice. Did you know that Port goes well with Fruit and Cheese? You may find this goes well with that: http://weblog.raganwald.com/2005/07/why-you-need-degree-to-work-for-bigco.html http://weblog.raganwald.com/2005/07/why-you-need-degree-to-w... ;-)
  - jey 19y ago
    Nice post.
  - gleb 19y ago
    Do read that PG article. A simple non-emotional and non-defensive answer to the hiring manager telling him you didn't go to college, but rather did whatever else you did would be best, IMO. How did you spend your your time in the mid-eighties, btw? You've got me intrigued now.
    - raganwald 19y ago
      I read that article when PG published it, and again when you mentioned it. What makes you think I gave a defensive or emotional answer to the intermediary who asked the question? I'm perfectly aware of the fact that the person asking the question either considers this piece of data of paramount importance OR they have twenty resumes on their desk that all look the same, and they're looking for an easy way to winnow the pile down to three, so even if it isn't that important, it's a way to shorten the pile. I personally doubt that they were looking at twenty nearly-identical resumes. My experience is that either of two situations hold: first, they might really believe this is the most important thing they need to know, or second, they might not be particularly skilled at identifying good candidates. Note that I am not claiming I'm a good candidate: but if I'm not worth an interview, they should be able to tell without that question, the facts are in plain view.
spiralhead 19y ago
I think it is natural for engineers (hackers, especially) to deal with situations on an as-needed basis. We just solve problems. We make at least SOME effort to solve them in such a way where they won't continually come back and bite us in the ass but security catastrophes are like earthquakes. They're low frequency events and therefore inherently difficult to learn from and plan for. It's possible but time consuming, so the average lone hacker will not bother until he absolutely must. We know it's important. It's in the back of our mind at all times. But shipping product and getting users NOW will always take precedence over architecting for the future, methinks. And that is just the unfortunate reality for now until (heaven forbid) there are some actual laws preventing it.
geebee 19y ago
I'm almost 100% opposed to a certification process that denies people the right to code if they don't hold certification X. But this post did get me thinking about a different kind of certification - one based on safety rather than talent. Society does have a reasonable interest in ensuring that applications that process sensitive personal data - like Social Security Numbers, Medical Information, Financial Information (bank account numbers) - meet a certain standard. And most people have no ability to evaluate the dangers of coding errors. Society doesn't have the right to demand protection from untalented programmers. If you try to pay as little $$$ as possible, your app may suck, it may never get built, the wrong app may get built, etc... If you pay top dollar, the same thing may happen. If you can't evaluate code, you can't evaluate programmers, so you're hosed - and maybe you shouldn't be in this game. If you decide to play it, buyer beware seems to work fine. But the general public, whose medical information or ssn is in a database somewhere, didn't decide to play this game - the game was played on them. So we'd say - you don't need any certification to code. But you do need certification to handle certain types of personal data. And if there's a major hack of an application, we (the public) get to know exactly whose "certified" signature was on that piece of paper.
weel 19y ago
The argument about what hiring or training focus is "the best" is a little silly, it seems to me, because different companies so obviously have different requirements. Take Google, as a prototypical high-tech firm, and Getronics, as a prototypical business automation consultancy of the kind that tends to put hackers to sleep but that is quite profitable nonetheless. Google tries to do things that are sufficiently technologically ahead that it will take competitors years to catch up. They'll work on problems before anybody realized they had them, and do things that might seem ridiculously speculative from the perspective of a Getronics executive, like statistical machine translation. Their hiring and and training seems to focus largely on finding, retaining, and developing intelligent generalists, and specialists in esoteric fields where there is no such thing as certification programs. Getronics, on the other hand, deals with big business clients who are often satisfied with rather run-off-the-mill technology, as long as it's delivered predictably and uniformly. (Of course, that's harder than it sounds.) A typical big job for Getronics might be to help some organization upgrade from a mainframe with terminals to a SQL server with PCs, without interrupting operations. There's nothing particularly innovative about it (it's been done thousands of times before), and it will likely be tolerated if the resulting solution is a little clumsy, but it had better not go wrong. Getronics has a long and detailed internal training program, and they like people with very specific experience. They have people who are specialists in, say, office printing systems, and who know all the little quirks you'll run into when you move from Novell print servers to Microsoft print servers. If you work there for a number of years, you progress through a system of internal certifications and ranks that begins to approach those of a military organization. For Google to behave like Getronics or vice versa would probably be silly. If I held shares in these companies, I'd probably want them both to continue hiring and training much the way they do now, even though they're worlds apart in the ways they do it.

10 points by mattculbreth 19y ago

mynameishere 19y ago

Exactly. After 10,000 tests in 20+ years of formal schooling, one more should do it.

raganwald 19y ago
I laughed so hard I almost fell out of my chair. Nice one!

Tichy 19y ago

At last one hiring "test" where I would utterly fail. It doesn't sound like much fun, either, so it would probably be for the better.

raganwald 19y ago
No, it really isn't much fun. Nor is it fun for Chefs to sit in class learning about hand washing protocols (there is a right and a wrong way to wash your hands). Keep that in mind the next time you eat at a restaurant where they pay the food prep people minimum wage!
- Tichy 19y ago
  Point taken, and I am always subconsciously aware of the fact that eating out is icky. However, I think there is a fundamental problem: if the protocol is too tedious, people will just ignore it. Then the situation is worse than before, because the protocol gives you a false sense of security. Rather than complain about the people, I think it would make more sense to complain about the protocol and change it in such a way that it is impossible to get around it (without it being so tedious). Example: in the bakery, the bread slicing machine only works with the protection down, so theoretically the worker can't possibly cut off his hands. Also, a worker should be aware of the fundamental issues, rather than just follow a protocol like a robot. Especially with security, isn't it the most common reason for security to fail that people construct doors with foolproof security, only to leave a gaping whole open right next to it?
  - raganwald 19y ago
    I think it would make more sense to complain about the protocol and change it in such a way that it is impossible to get around it (without it being so tedious). Isn't this the basis for things like binary milestones, iterative development, continuous integration, and so forth? An attempt to set up the system such that the easiest thing to do is also the highest value thing to do, without being too draconian? In the end, carrots work better than sticks, I quite agree.
    - Tichy 19y ago
      If I was exactly sure what all these names stand for, I could answer that ;-) I am sorry that my comment sounded so negative. Actually, I was really impressed about the certification I would fail. All the other "how to hire" articles so far were like "check that the candidate can add 2+2". And I wish I knew more about the methods you propose. Isn't there always a lurking feeling of "there has to be a better way" in programming?

mattculbreth 19y ago

Good article. I like this focus on the end result--can the guy create software which works, is testable, is reliable? I'd heard of a good way of interviewing an IT guy, like a network admin: You give them a machine with no OS, tell them to load it with whatever OS you usually use, join the network, and then do something pertinent to the job. If you're hiring an infrastructure guy for a startup you might ask him to create a machine with Ubuntu, Apache, Python, Ruby, Rails, Subversion, PostgreSQL, etc. If the dude can do all/most of it, and if you watch what he does (good use of the Internet? sensical file system setup? good eye for security?), then you might have a winner.

jey 19y ago
Yes, that IT test you cite is a good one, it actually test whether the guy can get real work done. The test that raganwald proposes doesn't actually test whether "the guy [can] create software which works, is testable, is reliable" -- it's testing whether they pass the test. Knowing your shit is one way to pass the test, but a lot more people pass the test by just memorizing test materials and learning the test instead of learning the subject. The situation might be improved if the whole Test Prep industry were made illegal. The part that saddens me the most is that the people who make the tests are also the ones selling the Test Prep materials to teach you how to circumvent the test! argh.

willarson 19y ago

The best software is reliable, intuitive, and has genuinely useful functionality. Like most ideas that violate the concept of the golden mean, this idea is appealing but fundamentally broken. He refutes his own point about the free market when he says it is currently broken (businesses hire students with Comp. Sci degrees even though they can't necessarily program well), but that the same broken free market will be sufficient to determine if programmers can program well. Identifying one arbitrary facet of software development and demanding that coders be experts at it, even if they can't do anything else, is shortsighted. This would make sense if testing was truely the hardest aspect of programming: if testing was the great ravine that thwarted aspiring programmers. That, however, has not been my experience. Testing is not trivial, but making good design choices and understanding the concepts of programming that he disparages will take longer to learn than testing, and the free market has proven it evaluates these qualifications coarsely at best. To be an expert is to have absorbed the symbols of your domain to the extent that they can be used effortlessly. Testing is only one of those symbols, and focusing on is flawed. So is ignoring it completely.

raganwald 19y ago
"He refutes his own point about the free market when he says it is currently broken (businesses hire students with Comp. Sci degrees even though they can't necessarily program well), but that the same broken free market will be sufficient to determine if programmers can program well." You are assuming that the goal of the free market is to hire good programmers. It isn't. Do you think managers are holding their heads in their hands, moaning that they can't find good programmers? Outside of start ups, they don't care. That is why I say the free market works. Yes, business hires Comp. Sci. people from "JavaSchools," but that's what they want! I am not suggesting that someone with this hypothetical certification is an expert, or even good. I think I said that five or even six times in the post. Don't confuse the rant with a hypothetical measure of a "good programmer." It isn't!!! It is a description of qualities I personally think are necessary but not sufficient. Please do not put words in my mouth about what I value and what I disparage. I said I do not care about certain things for the purpose of having this hypothetical certification. Take a few minutes googling me or reading my blog before deciding what I value or disparage. So back to the free market: the "problem" with the free market for programmers is similar to the "problem" with the free market for pollution, worker safety, and a bunch of other things we regulate. We have discovered that left to their own devices, businesses will make choices that expose their workers, their customers, and the public to danger. I don't care if a business making software for dive computers hires someone who takes four times as long to write code for half the hourly wage. The business can make the call about whether this makes sense. But I do care that the calculations are not broken, exposing me to death through Oxygen Toxicity, Nitrogen Narcosis, or the Bends. I'm not going on a crusade, it's just a post, but it's obvious to me that if businesses are left to make choices for themselves, they will put their customers in harm's way. And that's all I want to prevent. Summary: certification of the type I described is meant to be necessary but not sufficient. It doesn't say a Chef can cook well, it says a chef will not accidentally poison the soup.

gleb 19y ago

There is some correlation between presence/specific college/type of degree and certain relevant properties of a worker. That's why employers want to know. That's why everybody puts this info on the resume. Do read this http://paulgraham.com/judgement.html http://paulgraham.com/judgement.html

raganwald 19y ago
Nice. Did you know that Port goes well with Fruit and Cheese? You may find this goes well with that: http://weblog.raganwald.com/2005/07/why-you-need-degree-to-work-for-bigco.html http://weblog.raganwald.com/2005/07/why-you-need-degree-to-w... ;-)
- jey 19y ago
  Nice post.
- gleb 19y ago
  Do read that PG article. A simple non-emotional and non-defensive answer to the hiring manager telling him you didn't go to college, but rather did whatever else you did would be best, IMO. How did you spend your your time in the mid-eighties, btw? You've got me intrigued now.
  - raganwald 19y ago
    I read that article when PG published it, and again when you mentioned it. What makes you think I gave a defensive or emotional answer to the intermediary who asked the question? I'm perfectly aware of the fact that the person asking the question either considers this piece of data of paramount importance OR they have twenty resumes on their desk that all look the same, and they're looking for an easy way to winnow the pile down to three, so even if it isn't that important, it's a way to shorten the pile. I personally doubt that they were looking at twenty nearly-identical resumes. My experience is that either of two situations hold: first, they might really believe this is the most important thing they need to know, or second, they might not be particularly skilled at identifying good candidates. Note that I am not claiming I'm a good candidate: but if I'm not worth an interview, they should be able to tell without that question, the facts are in plain view.

spiralhead 19y ago

I think it is natural for engineers (hackers, especially) to deal with situations on an as-needed basis. We just solve problems. We make at least SOME effort to solve them in such a way where they won't continually come back and bite us in the ass but security catastrophes are like earthquakes. They're low frequency events and therefore inherently difficult to learn from and plan for. It's possible but time consuming, so the average lone hacker will not bother until he absolutely must. We know it's important. It's in the back of our mind at all times. But shipping product and getting users NOW will always take precedence over architecting for the future, methinks. And that is just the unfortunate reality for now until (heaven forbid) there are some actual laws preventing it.

geebee 19y ago

I'm almost 100% opposed to a certification process that denies people the right to code if they don't hold certification X. But this post did get me thinking about a different kind of certification - one based on safety rather than talent. Society does have a reasonable interest in ensuring that applications that process sensitive personal data - like Social Security Numbers, Medical Information, Financial Information (bank account numbers) - meet a certain standard. And most people have no ability to evaluate the dangers of coding errors. Society doesn't have the right to demand protection from untalented programmers. If you try to pay as little $$$ as possible, your app may suck, it may never get built, the wrong app may get built, etc... If you pay top dollar, the same thing may happen. If you can't evaluate code, you can't evaluate programmers, so you're hosed - and maybe you shouldn't be in this game. If you decide to play it, buyer beware seems to work fine. But the general public, whose medical information or ssn is in a database somewhere, didn't decide to play this game - the game was played on them. So we'd say - you don't need any certification to code. But you do need certification to handle certain types of personal data. And if there's a major hack of an application, we (the public) get to know exactly whose "certified" signature was on that piece of paper.

weel 19y ago

The argument about what hiring or training focus is "the best" is a little silly, it seems to me, because different companies so obviously have different requirements. Take Google, as a prototypical high-tech firm, and Getronics, as a prototypical business automation consultancy of the kind that tends to put hackers to sleep but that is quite profitable nonetheless. Google tries to do things that are sufficiently technologically ahead that it will take competitors years to catch up. They'll work on problems before anybody realized they had them, and do things that might seem ridiculously speculative from the perspective of a Getronics executive, like statistical machine translation. Their hiring and and training seems to focus largely on finding, retaining, and developing intelligent generalists, and specialists in esoteric fields where there is no such thing as certification programs. Getronics, on the other hand, deals with big business clients who are often satisfied with rather run-off-the-mill technology, as long as it's delivered predictably and uniformly. (Of course, that's harder than it sounds.) A typical big job for Getronics might be to help some organization upgrade from a mainframe with terminals to a SQL server with PCs, without interrupting operations. There's nothing particularly innovative about it (it's been done thousands of times before), and it will likely be tolerated if the resulting solution is a little clumsy, but it had better not go wrong. Getronics has a long and detailed internal training program, and they like people with very specific experience. They have people who are specialists in, say, office printing systems, and who know all the little quirks you'll run into when you move from Novell print servers to Microsoft print servers. If you work there for a number of years, you progress through a system of internal certifications and ranks that begins to approach those of a military organization. For Google to behave like Getronics or vice versa would probably be silly. If I held shares in these companies, I'd probably want them both to continue hiring and training much the way they do now, even though they're worlds apart in the ways they do it.