4 ms·

Peer validation platform for engineering skills (inspired by X community notes)

4 points by ms_sv 7mo ago

Hello Everyone. Resumes and CVs have a fundamental problem: anyone can write anything. As someone who's been job searching, I've wondered if there's a better way to separate genuine experience from creative writing, I am an engineer at the end of the day not a creative author. I've been thinking about applying something similar to X's Community Notes model to skill verification. The idea: engineers could "fact check" claims on each other's CVs - not as formal references, but as a crowd-sourced verification layer, where you get a check-mark on your skill like X check mark. If someone claims they're an expert in Kubernetes, other engineers who've worked with them (or reviewed their OSS contributions) could validate or challenge that. Also companies have repetitive interviews, why can't I simply do one interview and be "interviewed" fully for all other companies? I put together a rough prototype to illustrate the concept: https://skillverdict.com/ Some questions I'm trying to work through(ask more please): How useful will this be for engineers? Would this create its own set of problems? (gaming the system, bias, grudges) Could it scale beyond personal networks? Would companies even trust community-sourced verification? Curious what you guys think about the mechanism itself, not the prototype. Would something like this reduce friction in hiring, or just add another layer of noise?

unforbiddenYet 7mo ago
> engineers who've worked with them (or reviewed their OSS contributions) could validate or challenge that [skill] I like your idea of verifiable skills but I don’t think the proposal is solid. We need a trusted third-party as a validator so everyone is happy. Otherwise, we bring bias and corruption into this. The problem is currently solved by professional certification — yes it could be gamed but it provides an official confirmation from a reputable source rather than someone’s word that you’re good. Please keep iterating! We need better tools!
- ms_sv 7mo ago
  Thank you for the feedback, I appriciate it. The idea is to avoid the corruption and bias by hand picking a starter deck of super trusted few professionals who would be able to verify others. For example to prove someone is an senior engineer from google I can ask for a pay slip and references and yes certifications, so we would become that rusted partner. Still working on the proposal and iterations, I wanna talk to much engineers, hiring manages as possible since this is going to be for everyone, to navigate this wild job market. We definitely need better tools, at the moment it is wild that it takes 1000 of applications to just get an answer from any company, this not right at all
austin-cheney 7mo ago
As a hiring manager know what you want and be careful what you communicate to determine your desired audience. Secondly, skill verification is a form of leaky abstraction and may not be what hiring managers want. It’s most often not about finding the most skilled candidate but rather the most compatible candidate. This can commonly mean finding the least sucky person from a pool of sucky people. Hiring managers can game that by setting requirements criteria for a job. If you want extremely skilled people then get down to the metal and find candidates that like to work without tools or abstractions. If you super versatile candidates find people with experience in a bazillion different tools. If you just want a body to fill a seat that is quick to hire/fire select for the latest trendy framework. The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people. Dig in and see what they really want and then challenge it with questions only they can answer. Most people are bodies just wanting to be hired without bringing anything special to the table so that’s what most employers target for. The real challenge here is finding people that are highly skilled in a market where exceptional skills outside the bell curve are not commonly rewarded, because these people will not self identify as awesome when looking for work in compatibility driven system. The people that most typically do identify as awesome tend to not be as awesome as they believe.
- ms_sv 7mo ago
  This is a real insight "finding the most skilled candidate but rather the most compatible candidate", thank you very much, this is the kind of feedback I am looking for from engineers. I never looked at it from this perspective. I talked to hiring managers and most of them say skills is good but what we want is a personality because skills can be taught and personality cannot be. Personality I have experienced that, very skilled teammate exceptional, but personality was not a fit, so the management was not happy. "The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people.", that is true and this I have tried with when I was playing the role of hiring manager trying to find a team member to work with me in the company I worked for. I imagine the peer-to-peer verification framework like this talk to people and ask them personalized questions, this is how exceptional talent will be flushed out I think. Given that, if you were to imagine a tool that helps surface those 'quietly exceptional' people—the ones who are highly skilled but won't self-identify—what would that tool look like? Does it even need to be a tool to begin with?
tamimio 7mo ago
The idea is great, but everything else is not. A few points: - You say "engineering" verification, I checked and it's all computer science, monkey coding, full stack whatever, not only is that NOT engineering, it wasn't before generative AI, and it's definitely not now. This is programming. Programming is writing, and with AI it's more of storytelling rather than engineering. - How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc. - I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub? I think the best way to verify someone's skills in the engineering world is a portfolio that shows their projects and what was accomplished during them. A resume will never be enough, and interviews should be limited to personal interactions and assessing how the potential candidate communicates. Where needed, maybe an assignment to complete and return after a few days -one that mimics the potential project or role they will be working on- because asking questions during an interview is not enough to measure their skills at all.
- ms_sv 7mo ago
  Thank you for the feedback. It is meant to be all kinds of engineering, the vision is to have all type of engineers from software engineers(which you are correct it is focused full on that). > How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc - This will be all up to the hand picked person to measure your skills, for example the verifier will be assessing everything required to determine if someone is really good or not, that includes critical thinking, problem solving so on. Let's say this verifier they work/ed for Google, they will bring Google's interview framework to this candidate software engineer in our case to asses their skills, the expected result would be "I passed the assessment, I could be good as an engineer at Google" which helps with credibility. The platform itself will store the final decision on your profile (you can hide or show anything up to the user). > I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub? - Linkedin, github or gitlab are all optional fields, I have pushed a fix on that to communicate it better, thank you. I am thinking of making it a free link field to be portfolio rather than profile from github, gitlab or whatever else. Personally I understand this, not a fan of github at all, I just put it there since it is the most used. This is how I imagine it and actually tested this concept and it worked and you are 100% right it brings a lot of results. What I did was fist meet and greet talk, then assign take at home project, after the project was done, then we discussed the decisions made and approaches and how the problem was solved, and talked about past projects from the portfolio. This reduced the chance significantly of a bad hire, and the candidates liked the concept a lot. What don't you like the most from the current hiring processes? For example in software engineering for me is solving meaningless problems during interviews like reverse a binary tree that have nothing to do with the actual role.
lyaocean 7mo ago
The model gets interesting if reviewer credibility is scored by calibration over time, not follower count. Without that, it will drift toward reciprocal endorsements and people gaming each other’s profiles.
raw_anon_1111 7mo ago
Congratulations, you have just reinvented a feature that LinkedIn has had for years and not solved how easily it can be gamed. And honestly, validating skills like that is strictly something a mid level (no matter what your title is) “engineer” would do. Every interview I have had since 2014 - and I have had 5 since then including three as a strategic hire (still hands on keyboard) by a new to the company director/CTO and one at BigTech and now full time staff consultant (cloud + app dev) has been behavioral where I had to convince then that I was “smart and gets things done”. No matter what generic set of skills you have Kubernetes, AWS, etc, everyone and their dog have those skills - or at least enough people to make it difficult to stand out. If you do have a niche set of skills that not every one has (raises hand) you aren’t randomly spamming ATS’s and going through all of the standard interviewing processes. You’re being pushed through by a decision maker. You aren’t going to get hired in today’s market based on a generic set of skills without selling yourself as more than that. Every open rec gets hundreds of applications within a day. I am currently in the pool as the for lack of a better term and stolen from Amazon (former employer) as a “bar raiser”. I’m not going to ask you about K8s trivia. I’m going to ask you “what project are you most proud of” and then delve deep in the complexity of the project, your role and scope, your decision mskinv process etc.
- ms_sv 7mo ago
  Endorsements are good way, but limited and they can be bought and can be faked easily, which does not bring much credibility.Do you think there is a way to improve the endorsement system from LinkedIn into something that will be taken more seriously? Yes that is the main assumption, seniors would not be interested in getting their skill verified they will be doing the verifying to a junior or whatever entry title there is, in order to help them gain their credibility even mentor them, so it would be easier for them to stand out. That is a fair point, also that is going to be part of the verification process, engineers would be able to get badges verified for their projects, atm I can AI generate a whole project and add it to my portfolio, without having to do the system design work, but if I get into the verification process, then I would be able to prove I did it if I really did by explaining my system design, decisions, compromises so on.
  - 7mo ago
NimrodKramer 7mo ago
[dead]

4 points by ms_sv 7mo ago

unforbiddenYet 7mo ago

> engineers who've worked with them (or reviewed their OSS contributions) could validate or challenge that [skill] I like your idea of verifiable skills but I don’t think the proposal is solid. We need a trusted third-party as a validator so everyone is happy. Otherwise, we bring bias and corruption into this. The problem is currently solved by professional certification — yes it could be gamed but it provides an official confirmation from a reputable source rather than someone’s word that you’re good. Please keep iterating! We need better tools!

ms_sv 7mo ago
Thank you for the feedback, I appriciate it. The idea is to avoid the corruption and bias by hand picking a starter deck of super trusted few professionals who would be able to verify others. For example to prove someone is an senior engineer from google I can ask for a pay slip and references and yes certifications, so we would become that rusted partner. Still working on the proposal and iterations, I wanna talk to much engineers, hiring manages as possible since this is going to be for everyone, to navigate this wild job market. We definitely need better tools, at the moment it is wild that it takes 1000 of applications to just get an answer from any company, this not right at all

austin-cheney 7mo ago

As a hiring manager know what you want and be careful what you communicate to determine your desired audience. Secondly, skill verification is a form of leaky abstraction and may not be what hiring managers want. It’s most often not about finding the most skilled candidate but rather the most compatible candidate. This can commonly mean finding the least sucky person from a pool of sucky people. Hiring managers can game that by setting requirements criteria for a job. If you want extremely skilled people then get down to the metal and find candidates that like to work without tools or abstractions. If you super versatile candidates find people with experience in a bazillion different tools. If you just want a body to fill a seat that is quick to hire/fire select for the latest trendy framework. The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people. Dig in and see what they really want and then challenge it with questions only they can answer. Most people are bodies just wanting to be hired without bringing anything special to the table so that’s what most employers target for. The real challenge here is finding people that are highly skilled in a market where exceptional skills outside the bell curve are not commonly rewarded, because these people will not self identify as awesome when looking for work in compatibility driven system. The people that most typically do identify as awesome tend to not be as awesome as they believe.

ms_sv 7mo ago
This is a real insight "finding the most skilled candidate but rather the most compatible candidate", thank you very much, this is the kind of feedback I am looking for from engineers. I never looked at it from this perspective. I talked to hiring managers and most of them say skills is good but what we want is a personality because skills can be taught and personality cannot be. Personality I have experienced that, very skilled teammate exceptional, but personality was not a fit, so the management was not happy. "The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people.", that is true and this I have tried with when I was playing the role of hiring manager trying to find a team member to work with me in the company I worked for. I imagine the peer-to-peer verification framework like this talk to people and ask them personalized questions, this is how exceptional talent will be flushed out I think. Given that, if you were to imagine a tool that helps surface those 'quietly exceptional' people—the ones who are highly skilled but won't self-identify—what would that tool look like? Does it even need to be a tool to begin with?

tamimio 7mo ago

The idea is great, but everything else is not. A few points: - You say "engineering" verification, I checked and it's all computer science, monkey coding, full stack whatever, not only is that NOT engineering, it wasn't before generative AI, and it's definitely not now. This is programming. Programming is writing, and with AI it's more of storytelling rather than engineering. - How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc. - I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub? I think the best way to verify someone's skills in the engineering world is a portfolio that shows their projects and what was accomplished during them. A resume will never be enough, and interviews should be limited to personal interactions and assessing how the potential candidate communicates. Where needed, maybe an assignment to complete and return after a few days -one that mimics the potential project or role they will be working on- because asking questions during an interview is not enough to measure their skills at all.

ms_sv 7mo ago
Thank you for the feedback. It is meant to be all kinds of engineering, the vision is to have all type of engineers from software engineers(which you are correct it is focused full on that). > How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc - This will be all up to the hand picked person to measure your skills, for example the verifier will be assessing everything required to determine if someone is really good or not, that includes critical thinking, problem solving so on. Let's say this verifier they work/ed for Google, they will bring Google's interview framework to this candidate software engineer in our case to asses their skills, the expected result would be "I passed the assessment, I could be good as an engineer at Google" which helps with credibility. The platform itself will store the final decision on your profile (you can hide or show anything up to the user). > I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub? - Linkedin, github or gitlab are all optional fields, I have pushed a fix on that to communicate it better, thank you. I am thinking of making it a free link field to be portfolio rather than profile from github, gitlab or whatever else. Personally I understand this, not a fan of github at all, I just put it there since it is the most used. This is how I imagine it and actually tested this concept and it worked and you are 100% right it brings a lot of results. What I did was fist meet and greet talk, then assign take at home project, after the project was done, then we discussed the decisions made and approaches and how the problem was solved, and talked about past projects from the portfolio. This reduced the chance significantly of a bad hire, and the candidates liked the concept a lot. What don't you like the most from the current hiring processes? For example in software engineering for me is solving meaningless problems during interviews like reverse a binary tree that have nothing to do with the actual role.

lyaocean 7mo ago

The model gets interesting if reviewer credibility is scored by calibration over time, not follower count. Without that, it will drift toward reciprocal endorsements and people gaming each other’s profiles.

raw_anon_1111 7mo ago

Congratulations, you have just reinvented a feature that LinkedIn has had for years and not solved how easily it can be gamed. And honestly, validating skills like that is strictly something a mid level (no matter what your title is) “engineer” would do. Every interview I have had since 2014 - and I have had 5 since then including three as a strategic hire (still hands on keyboard) by a new to the company director/CTO and one at BigTech and now full time staff consultant (cloud + app dev) has been behavioral where I had to convince then that I was “smart and gets things done”. No matter what generic set of skills you have Kubernetes, AWS, etc, everyone and their dog have those skills - or at least enough people to make it difficult to stand out. If you do have a niche set of skills that not every one has (raises hand) you aren’t randomly spamming ATS’s and going through all of the standard interviewing processes. You’re being pushed through by a decision maker. You aren’t going to get hired in today’s market based on a generic set of skills without selling yourself as more than that. Every open rec gets hundreds of applications within a day. I am currently in the pool as the for lack of a better term and stolen from Amazon (former employer) as a “bar raiser”. I’m not going to ask you about K8s trivia. I’m going to ask you “what project are you most proud of” and then delve deep in the complexity of the project, your role and scope, your decision mskinv process etc.

ms_sv 7mo ago
Endorsements are good way, but limited and they can be bought and can be faked easily, which does not bring much credibility.Do you think there is a way to improve the endorsement system from LinkedIn into something that will be taken more seriously? Yes that is the main assumption, seniors would not be interested in getting their skill verified they will be doing the verifying to a junior or whatever entry title there is, in order to help them gain their credibility even mentor them, so it would be easier for them to stand out. That is a fair point, also that is going to be part of the verification process, engineers would be able to get badges verified for their projects, atm I can AI generate a whole project and add it to my portfolio, without having to do the system design work, but if I get into the verification process, then I would be able to prove I did it if I really did by explaining my system design, decisions, compromises so on.
- 7mo ago

NimrodKramer 7mo ago

[dead]