4 ms·

I took a look at skills app -- I like the approach and the clean layout, but a couple thoughts from this hiring manager: * Most of the really good devs I know

by yajoe 15y ago

I took a look at skills app -- I like the approach and the clean layout, but a couple thoughts from this hiring manager: * Most of the really good devs I know and hire do not bother with easy metrics like number of tweets, followers, or StackOverflow points. They are perfect to paste together on summary reports and look "complete," but their absolute values or essence don't mean anything. * The high numbers on public sites seem to be for two classes of devs -- celebrities, who you already know are celebrities -- and people with way too much time on their hands (if they don't work for an open source company, which most people don't). Most people have average values. If someone is on LinkedIn, I'm more interested to see if they have X hundred followers per years in industry up to some maximum lifetime value. I generally ignore recommendations. * See http://news.ycombinator.com/item?id=3771286 http://news.ycombinator.com/item?id=3771286 for the discussion about what the number of points actually measures on SO, for example. It's a great site, but it's struggling to handle both breadth and depth of material well. High numbers does not necessarily correlate with expertise, just dedication. Maybe I want that skillset for a customer service rep, but I haven't yet wanted it for an engineer. * I have shipped 20M loc in my career in 32 regions, and none of it is available on github or sourceforge. Most engineers are like that -- again, the public numbers are biased to certain individuals. Maybe there is a way for candidates to quantify the projects they have worked on? * What I would love is a way to automatically assess FizzBuzz on candidates. I know there are some sites like interviewstreet.com that try to do this, but they don't yet have the critical mass like linkedin. And I fear, the moment they get critical mass then they'll become a new SAT or other standardized test that becomes too easy to game.

idan 15y ago
Thanks for sharing your brain (and experience) with us! We don't think number of tweets or followers is a relevant means of assessment at all—quite the opposite. Ashton Kutcher is not an interesting candidate for a rails job, and that should be reflected in our report for him (were he ever to apply for such a position.) We obviously gather a lot of metrics (hey, storage is cheap) but the ones we actually use in our scoring mix right now are tied to two simple concepts: does this person care about their craft, and do they enjoy any amount of professional respect for what they do. We aren't offering shoe size; what you need in early-stage hiring decisions is an indicator: something more fast than perfect. Resumes suck at this because everybody is trained to write sterile, "professional" resumes stuffed to the brim with keywords. We can tell a much more interesting story—today!—with the kinds of data that most developers in our industry are already sharing. SO reputation isn't a story, it's a data point. What tech people ask questions about versus what tech they answer questions about reveals something more interesting. They know Python very well but seem to be exploring JavaScript? Interesting. Maybe a hybrid in the making. Skills isn't for everyone and every industry; it's for one industry and a very specific breed of candidates that has this kind of publicly-visible track record. We think there are enough companies that are hiring in this space that there's a business to be built here now, and the Internet trends towards more public data. Do we really think that we'll still be using resumes in ten years to assess whether to spend more time on engineering candidate X?
- ericd 15y ago
  A point of feedback - I write a lot of code that gets used by lots of people, and I don't do any of it in a social fashion because I'm entirely uninterested in creating a brand for myself as a developer. All of these metrics you're talking about would screen me out, whereas a simple "created and scaled FOO with BAR set of technologies, used by 100,000/1,000,000/10,000,000 people/month" on a resume would probably get a good amount of employer attention, for good reason - that's an indication of a real full stack engineer who has faced a gamut of real problems. It's not as easy to quantify, but you should probably take that kind of thing into account, because it's a lot more meaningful than the number of people signed up to receive my quips about how much IE6 sucks or how many side projects I've open sourced.
- roguecoder 15y ago
  My resume isn't like that, and I think that's why I don't have any problem finding jobs. My resume tells the story of each role I've had, including concrete examples and metrics of success. I think your site is a perfect example of "lucky" candidates winning over unlucky candidates. A bunch of data probably unrelated to paying employment that a certain, lucky, subset of developers happen to score well on while others, especially those who traditionally hide on the Internet, look really bad in comparison.
idan 15y ago
Just FYI, we've been working hard on completely revamping ur github visualizationns to be a ton more meaningful. The current site is very much our MV(?)P. We've thought about automated coding assessments but for where we are right now, it doesn't feel like the right direction to swim in to get to viability. Any kind of exam this early in the hiring process will sour the best candidates on applying. If you are a solid developer with a good track record, chances are that you wouldn't want to work with a company that can't figure that out about you with a brief googling. Anybody using that sort of tool to replace resumes is locking themselves out of the top talent. Finally,mwe've definitely been thinking a lot recently about moving to a model without any kind of scores—just visualizations of data that help you figure out who a person is, maybe get a little bit of other personality in a way that resumes will never give you, and help you make that quick move-forward-or-not decision. We're not sure if scores really contribute something to that, and we'd love to hear what you think.
davidkatz 15y ago
Perhaps I can try to convince you of our long term vision: developers increasingly take their work online, which results in a mountain of professional data. A lot of it is meaningless, but some of it can be woven into interesting stories. We're still trying to find the best approach to this, but in essence, our mission is making sense of this data. Some of this will have nothing to do with popularity - say, we might give you a concise overview of someone's code or professional thoughts. Other metrics will use social activity only as a rough validation. For example, if someone has 20 watchers on GitHub, they're definitely not a celebrity, but they have crossed a very meaningful threshold of "someone cares about this person's work". Does that sound like a good direction to you?
- roguecoder 15y ago
  I think the problem I have is that it is not the case that "developers" are taking their work on line. It is the case that "some developers" are taking their work online. If all you care about is that subset of developers, I'm sure your approach is fine.
  - davidkatz 15y ago
    I get you. A lot of developers don't take their work online at all. But there are probably a hundred thousands developers who do take at least some parts of their work online. Perhaps speaking more practically, the market for developers is segmented. Not all companies want to hire all developers, and some companies specifically look for developers who do take their work online. It's those companies we want to help right now. Take us for example. We just hired Yuval Adam (http://y3xz.com/ http://y3xz.com/) for a project, and his online activity had a huge part in raising our interest. Check out his site. It's essentially a collection of links to professionally relevant online content. Sure, these are no guarantees. If you were hiring a developer though, wouldn't your interest be raised?
- amcintyre 15y ago
  Sometimes developers don't have a choice about putting their work online--many large companies actively discourage (or prohibit) their engineers from participating in any meaningful way in open source projects. Want to post your personal hacking online under your real name? Nope, sorry--"everything you do belongs to the company, and we can't take the chance that you might accidentally post some uber-valuable trade secret." So rather than risk losing the job, you just find some other, not-publicly-visible way to spend your time. Maybe this group of developers isn't viewed as interesting to many employers, and if that's the case my argument isn't worth much.

by yajoe 15y ago

idan 15y ago

Thanks for sharing your brain (and experience) with us! We don't think number of tweets or followers is a relevant means of assessment at all—quite the opposite. Ashton Kutcher is not an interesting candidate for a rails job, and that should be reflected in our report for him (were he ever to apply for such a position.) We obviously gather a lot of metrics (hey, storage is cheap) but the ones we actually use in our scoring mix right now are tied to two simple concepts: does this person care about their craft, and do they enjoy any amount of professional respect for what they do. We aren't offering shoe size; what you need in early-stage hiring decisions is an indicator: something more fast than perfect. Resumes suck at this because everybody is trained to write sterile, "professional" resumes stuffed to the brim with keywords. We can tell a much more interesting story—today!—with the kinds of data that most developers in our industry are already sharing. SO reputation isn't a story, it's a data point. What tech people ask questions about versus what tech they answer questions about reveals something more interesting. They know Python very well but seem to be exploring JavaScript? Interesting. Maybe a hybrid in the making. Skills isn't for everyone and every industry; it's for one industry and a very specific breed of candidates that has this kind of publicly-visible track record. We think there are enough companies that are hiring in this space that there's a business to be built here now, and the Internet trends towards more public data. Do we really think that we'll still be using resumes in ten years to assess whether to spend more time on engineering candidate X?

ericd 15y ago
A point of feedback - I write a lot of code that gets used by lots of people, and I don't do any of it in a social fashion because I'm entirely uninterested in creating a brand for myself as a developer. All of these metrics you're talking about would screen me out, whereas a simple "created and scaled FOO with BAR set of technologies, used by 100,000/1,000,000/10,000,000 people/month" on a resume would probably get a good amount of employer attention, for good reason - that's an indication of a real full stack engineer who has faced a gamut of real problems. It's not as easy to quantify, but you should probably take that kind of thing into account, because it's a lot more meaningful than the number of people signed up to receive my quips about how much IE6 sucks or how many side projects I've open sourced.
roguecoder 15y ago
My resume isn't like that, and I think that's why I don't have any problem finding jobs. My resume tells the story of each role I've had, including concrete examples and metrics of success. I think your site is a perfect example of "lucky" candidates winning over unlucky candidates. A bunch of data probably unrelated to paying employment that a certain, lucky, subset of developers happen to score well on while others, especially those who traditionally hide on the Internet, look really bad in comparison.

idan 15y ago

Just FYI, we've been working hard on completely revamping ur github visualizationns to be a ton more meaningful. The current site is very much our MV(?)P. We've thought about automated coding assessments but for where we are right now, it doesn't feel like the right direction to swim in to get to viability. Any kind of exam this early in the hiring process will sour the best candidates on applying. If you are a solid developer with a good track record, chances are that you wouldn't want to work with a company that can't figure that out about you with a brief googling. Anybody using that sort of tool to replace resumes is locking themselves out of the top talent. Finally,mwe've definitely been thinking a lot recently about moving to a model without any kind of scores—just visualizations of data that help you figure out who a person is, maybe get a little bit of other personality in a way that resumes will never give you, and help you make that quick move-forward-or-not decision. We're not sure if scores really contribute something to that, and we'd love to hear what you think.

davidkatz 15y ago

Perhaps I can try to convince you of our long term vision: developers increasingly take their work online, which results in a mountain of professional data. A lot of it is meaningless, but some of it can be woven into interesting stories. We're still trying to find the best approach to this, but in essence, our mission is making sense of this data. Some of this will have nothing to do with popularity - say, we might give you a concise overview of someone's code or professional thoughts. Other metrics will use social activity only as a rough validation. For example, if someone has 20 watchers on GitHub, they're definitely not a celebrity, but they have crossed a very meaningful threshold of "someone cares about this person's work". Does that sound like a good direction to you?

roguecoder 15y ago
I think the problem I have is that it is not the case that "developers" are taking their work on line. It is the case that "some developers" are taking their work online. If all you care about is that subset of developers, I'm sure your approach is fine.
- davidkatz 15y ago
  I get you. A lot of developers don't take their work online at all. But there are probably a hundred thousands developers who do take at least some parts of their work online. Perhaps speaking more practically, the market for developers is segmented. Not all companies want to hire all developers, and some companies specifically look for developers who do take their work online. It's those companies we want to help right now. Take us for example. We just hired Yuval Adam (http://y3xz.com/ http://y3xz.com/) for a project, and his online activity had a huge part in raising our interest. Check out his site. It's essentially a collection of links to professionally relevant online content. Sure, these are no guarantees. If you were hiring a developer though, wouldn't your interest be raised?
amcintyre 15y ago
Sometimes developers don't have a choice about putting their work online--many large companies actively discourage (or prohibit) their engineers from participating in any meaningful way in open source projects. Want to post your personal hacking online under your real name? Nope, sorry--"everything you do belongs to the company, and we can't take the chance that you might accidentally post some uber-valuable trade secret." So rather than risk losing the job, you just find some other, not-publicly-visible way to spend your time. Maybe this group of developers isn't viewed as interesting to many employers, and if that's the case my argument isn't worth much.