5 ms·

I wouldn't mind seeing education return to its roots of being about learning instead of credentialization. In an age where having a degree is increasingly meani

I wouldn't mind seeing education return to its roots of being about learning instead of credentialization. In an age where having a degree is increasingly meaningless in part due to many places simply becoming thinly veiled diploma treadmills (which are somehow nonetheless accredited), this is probably more important than ever. This is doubly so if the AI impact extremists end up being correct. So why is the issue you described an issue? Because it's about a grade. And the reason that's relevant is because that credential will then be used to determine where she can to to university which, in turn, is a credential that will determine her breadth of options for starting her career, and so on. But why is this all done by credentials instead of simple demonstrations of skill? What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting. This is how you used to apply to colleges. Here [1], for instance, is Harvard's exam from 1869. If you pass it, you're in. Simple as that. Obviously this creates a problem of institutions starting to 'teach the test', but with sufficiently broad testing I don't see this as a problem. If a writing class can teach somebody to write a compelling essay based on an arbitrary prompt, then that was simply a good writing class! As an aside this would also add a major selling point to all of the top universities that offer free educational courses online. Right now I think 'normal' people are mostly disinterested in those because of the lack of widely accepted credentials, which is just so backwards - people are actively seeking to maximize credentials over maximizing learning. This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization. [1] - https://graphics8.nytimes.com/packages/pdf/education/harvardexam.pdf https://graphics8.nytimes.com/packages/pdf/education/harvard...

disgruntledphd2 11mo ago
> This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization. Just so we're clear, the coding tests are in addition to credentialisation. I'll never forget when I worked at Big Tech (from Ireland) and I would constantly hear recruiters talk about the OK school list (basically the Ivy league). Additionally, I remember having to check the University a candidate had attended before she had an interview with one of our directors. He was fine with her, because she had gone to Oxford. Honestly, I'm surprised that I was able to get hired there given all this nonsense.
- oblio 11mo ago
  If anything, it will get worse. There was a deficit of tech workers, from now on, there will be an excess. Which means that differentiators will be even more important.
  - anoplus 11mo ago
    The smaller the job market relative to candidates, the more the differentiators are both important and also meaningless to the actual job
    - oblio 11mo ago
      We gotta play the game, though.
- rrrobert 11mo ago
  My experience with big tech has been the polar opposite - nobody has ever cared and I've never tried to hide it either. Which one was it if you don't mind me asking? I'm a drop out (didn't finish BSc) from a no name Northern European university and I've worked at or gotten offers from: - Meta - Amazon - Google - Microsoft - Uber - xAI + some unicorns that compete with FAANG+ locally. I didn't include some others that have reached out for interviews which I declined at the time. The lack of a degree has literally never come up for me.
  - disgruntledphd2 11mo ago
    It was Meta but twas a long time ago, when they were known as Facebook. It seems to be a US role thing in my experience.
    - Fripplebubby 11mo ago
      Hiring is still a pretty non-uniform thing despite attempts to make it less so - I'm sure there are some teams and orgs at all these large companies that do it well, and some that do it les well. I think it is pretty well accepted that university brand is not a good signal, but it is an easy signal and if the folks in the hiring process are a bit lazy and pressed for time, a bit overwhelmed by the number of inbound candidates, or don't really know how to evaluate for the role competencies, I think it's a tool that is still reached for today. In a way, I think the hiring process at second-tier (not FAANG) companies is actually better because you have to "moneyball" a little bit - you know that you're going to lose the most-credentialed people to other companies that can beat you dollar for dollar, so you actually have to think a little more deeply about what a role really needs to find the right person.
andrepd 11mo ago
That's an absolutely dreadful exam. Would you mind explaining the point you wanted to make with it?
- hbs18 11mo ago
  It's a product of its time, but what exactly do you think makes it bad?
thfuran 11mo ago
>What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting Sure, but it takes < 1 second to read a GPA.
- array_key_first 11mo ago
  Right, in an ideal world we'd peer into the minds of people and compute what they know. But if we did that, our eyes would probably catch on fire like that lady in Kingdom of the Crystal Skull. We need some way to distill the unbelievable amount of data in human brains into something that can be processed in a reasonable amount of time. We need a measurement - a degree, a GPA, something. Imagine if in every job interview they could assume absolutely nothing. They know nothing about your education. They might start by asking you to recite your ABCs and then, finally at sunset, you might get to a coding exam. Which still won't work, because you'll just AI cheat the coding exam. We require gatekeepers to make the system work. If we allow the gatekeepers to just rubber stamp based off of if stuff seems correct, that tells us nothing about the person itself. We want the measurement to get close to the real understanding. That means AI papers have to be given a 0, which means we need to know if something is AI generated. And we want to catch this at the education level, not above.
  - cindyllm 11mo ago
    [dead]
  - ghaff 11mo ago
    I did have interviews with a government agency many years ago that, among other things, involved a battery of tests including what I assume were foreign civil service exams. I got an offer though I didn't take it. But assuming in-person day long batteries of tests for universities and companies is probably not very practical. You can argue whether university is a very efficient use of time or money but it presumably does involve some learning and offers potential employers some level of a filter that roughly aligns with what they're looking for.
alpinisme 11mo ago
In a world where some but not all programs are “diploma treadmills,” you would expect that the reputation of the bad credentials would go down and the good credentials would go up. In some sense if the credentials were really being used (and not just as a perfunctory first pass elimination), you’d expect the most elite programs to have the highest signal to noise ratio. But the market doesn’t seem to respond to changes in credentialing capability (by hiring more from programs that start focusing on the “right” things to test). Instead it’s really just a background check.
- BeFlatXIII 11mo ago
  In the US, it's also because there are so many options that it's not feasible to have a clear ranking of schools outside of the extreme ends of the spectrum.
- sanjayjc 11mo ago
  > you would expect that the reputation of the bad credentials would go down and the good credentials would go up. We should expect this if employers can efficiently and objectively evaluate a candidate's skills without relying on credentials. When they're unable to, we should worry about this information asymmetry leading to a "market for lemons" [0]. I found an article [1] about how this could play out: > This scenario leads to a clear case of information asymmetry since only the graduate knows whether their degree reflects real proficiency, while employers have no reliable way to verify this. This mirrors the classic “Market for Lemons” concept introduced by economist George Akerlof in 1970, where the presence of low-quality goods (or in this case, under-skilled graduates) drives down the perceived value of all goods, due to a lack of trustworthy signals. [0] https://quickonomics.com/terms/market-for-lemons/ https://quickonomics.com/terms/market-for-lemons/ [1] https://competitiveness.in/how-ai-could-exacerbate-the-skill-shortage-crisis/ https://competitiveness.in/how-ai-could-exacerbate-the-skill...
- addaon 11mo ago
  > But the market doesn’t seem to respond to changes in credentialing capability I mean, it certainly seems to. I've been in hiring roles in tech for 20-ish years, and have definitely seen changes in how college hire patterns based on credential values. Some schools have gone way up in how much we value their credentials (Waterloo), some have gone somewhat down (MIT), etc.
snickerbockers 11mo ago
I still don't understand why standardized testing gets so much pushback. Having the students do their work in a controlled environment is the obvious solution to AI and many other problems related to academic integrity. Its also the only way that students can actually be held to the same standards. When I was a freshman in college with a 3.4 highschool GPA, I was absolutely gobsmacked by how many kids with perfect >= 4.0 GPAs couldn't pass the simple algebra test that the university administered to all undergraduates as a prerequisite for taking any advanced mathematics course.
- Viliam1234 11mo ago
  > I still don't understand why standardized testing gets so much pushback. Parents of the children who can't pass the standardized tests also get a vote.
- hamdingers 11mo ago
  > I still don't understand why standardized testing gets so much pushback. Goodhart's law.
  - bmacho 11mo ago
    Nah. Goodhart's law is literally just "if you play a matrix game don't announce your pick in advance". It is not a real law, or not different from common sense. (By matrix game I mean what wiki calls "Normal form game[0]", e.g. rock-paper-scissors or prisoner's dilemma.) In education, regarding exams, Goodhart's law just means that you should randomize your test questions instead of telling the students the questions before the exam. Have a wide set of questions, randomize them. The only way for students to pass is to learn the material. A randomized standardized test is not more susceptible to Goodhart's law than a randomized personal test. The latter however has many additional problems. [0] : https://en.wikipedia.org/wiki/Normal-form_game https://en.wikipedia.org/wiki/Normal-form_game
    - jerf 11mo ago
      That's not even remotely true. A randomized standardized test will still have some domain that it chooses its questions from and that domain will be perfectly susceptible to Goodhart's Law. It is already the case that no one is literally teaching "On the SAT you're going to get this problem about triangle similarity and the answer is C." When a fresh batch of students sits down in front of some year's SATs the test is still effectively "randomized" relative to the education they received. But that randomization is relative to a rigid standardized curriculum and the teaching was absolutely Goodhart'd relative to that curriculum. "The only way for students to pass is to learn the material." Part of Goodhart's law in this context is precisely that it overdetermines "the material" and there is no way around this. I wish Goodhart's law was as easy to dodge as you think it is, but it isn't.
eloisant 11mo ago
> In an age where having a degree is increasingly meaningless I wish I would agree with you, but I think that having a degree (or rather the right degree) is more important than ever. Basically grades exist to decide who gets a laid back high paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. As one teacher told me once: we could have all of you practice chess, make a big tournament and you get to choose your university based on your chess ranking. It wouldn't be any less stupid than the current system.
- aleph_minus_one 11mo ago
  > Basically grades exist to decide who gets a laid back high paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. Rather: Grades exist to decide who gets a stressful, but rather high-paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. Laid-back high-paying job (unluckily) barely exist.

disgruntledphd2 11mo ago

> This is one of the very few places I think big tech in the US has done a great job. Coding interviews can be justifiably critiqued in many ways, but it's still a much better system than raw credentialization. Just so we're clear, the coding tests are in addition to credentialisation. I'll never forget when I worked at Big Tech (from Ireland) and I would constantly hear recruiters talk about the OK school list (basically the Ivy league). Additionally, I remember having to check the University a candidate had attended before she had an interview with one of our directors. He was fine with her, because she had gone to Oxford. Honestly, I'm surprised that I was able to get hired there given all this nonsense.

oblio 11mo ago
If anything, it will get worse. There was a deficit of tech workers, from now on, there will be an excess. Which means that differentiators will be even more important.
- anoplus 11mo ago
  The smaller the job market relative to candidates, the more the differentiators are both important and also meaningless to the actual job
  - oblio 11mo ago
    We gotta play the game, though.
rrrobert 11mo ago
My experience with big tech has been the polar opposite - nobody has ever cared and I've never tried to hide it either. Which one was it if you don't mind me asking? I'm a drop out (didn't finish BSc) from a no name Northern European university and I've worked at or gotten offers from: - Meta - Amazon - Google - Microsoft - Uber - xAI + some unicorns that compete with FAANG+ locally. I didn't include some others that have reached out for interviews which I declined at the time. The lack of a degree has literally never come up for me.
- disgruntledphd2 11mo ago
  It was Meta but twas a long time ago, when they were known as Facebook. It seems to be a US role thing in my experience.
  - Fripplebubby 11mo ago
    Hiring is still a pretty non-uniform thing despite attempts to make it less so - I'm sure there are some teams and orgs at all these large companies that do it well, and some that do it les well. I think it is pretty well accepted that university brand is not a good signal, but it is an easy signal and if the folks in the hiring process are a bit lazy and pressed for time, a bit overwhelmed by the number of inbound candidates, or don't really know how to evaluate for the role competencies, I think it's a tool that is still reached for today. In a way, I think the hiring process at second-tier (not FAANG) companies is actually better because you have to "moneyball" a little bit - you know that you're going to lose the most-credentialed people to other companies that can beat you dollar for dollar, so you actually have to think a little more deeply about what a role really needs to find the right person.

andrepd 11mo ago

That's an absolutely dreadful exam. Would you mind explaining the point you wanted to make with it?

hbs18 11mo ago
It's a product of its time, but what exactly do you think makes it bad?

thfuran 11mo ago

>What somebody scored in a high school writing class should matter far less than the output somebody is capable of producing when given a prompt and an hour in a closed setting Sure, but it takes < 1 second to read a GPA.

array_key_first 11mo ago
Right, in an ideal world we'd peer into the minds of people and compute what they know. But if we did that, our eyes would probably catch on fire like that lady in Kingdom of the Crystal Skull. We need some way to distill the unbelievable amount of data in human brains into something that can be processed in a reasonable amount of time. We need a measurement - a degree, a GPA, something. Imagine if in every job interview they could assume absolutely nothing. They know nothing about your education. They might start by asking you to recite your ABCs and then, finally at sunset, you might get to a coding exam. Which still won't work, because you'll just AI cheat the coding exam. We require gatekeepers to make the system work. If we allow the gatekeepers to just rubber stamp based off of if stuff seems correct, that tells us nothing about the person itself. We want the measurement to get close to the real understanding. That means AI papers have to be given a 0, which means we need to know if something is AI generated. And we want to catch this at the education level, not above.
- cindyllm 11mo ago
  [dead]
- ghaff 11mo ago
  I did have interviews with a government agency many years ago that, among other things, involved a battery of tests including what I assume were foreign civil service exams. I got an offer though I didn't take it. But assuming in-person day long batteries of tests for universities and companies is probably not very practical. You can argue whether university is a very efficient use of time or money but it presumably does involve some learning and offers potential employers some level of a filter that roughly aligns with what they're looking for.

alpinisme 11mo ago

In a world where some but not all programs are “diploma treadmills,” you would expect that the reputation of the bad credentials would go down and the good credentials would go up. In some sense if the credentials were really being used (and not just as a perfunctory first pass elimination), you’d expect the most elite programs to have the highest signal to noise ratio. But the market doesn’t seem to respond to changes in credentialing capability (by hiring more from programs that start focusing on the “right” things to test). Instead it’s really just a background check.

BeFlatXIII 11mo ago
In the US, it's also because there are so many options that it's not feasible to have a clear ranking of schools outside of the extreme ends of the spectrum.
sanjayjc 11mo ago
> you would expect that the reputation of the bad credentials would go down and the good credentials would go up. We should expect this if employers can efficiently and objectively evaluate a candidate's skills without relying on credentials. When they're unable to, we should worry about this information asymmetry leading to a "market for lemons" [0]. I found an article [1] about how this could play out: > This scenario leads to a clear case of information asymmetry since only the graduate knows whether their degree reflects real proficiency, while employers have no reliable way to verify this. This mirrors the classic “Market for Lemons” concept introduced by economist George Akerlof in 1970, where the presence of low-quality goods (or in this case, under-skilled graduates) drives down the perceived value of all goods, due to a lack of trustworthy signals. [0] https://quickonomics.com/terms/market-for-lemons/ https://quickonomics.com/terms/market-for-lemons/ [1] https://competitiveness.in/how-ai-could-exacerbate-the-skill-shortage-crisis/ https://competitiveness.in/how-ai-could-exacerbate-the-skill...
addaon 11mo ago
> But the market doesn’t seem to respond to changes in credentialing capability I mean, it certainly seems to. I've been in hiring roles in tech for 20-ish years, and have definitely seen changes in how college hire patterns based on credential values. Some schools have gone way up in how much we value their credentials (Waterloo), some have gone somewhat down (MIT), etc.

snickerbockers 11mo ago

I still don't understand why standardized testing gets so much pushback. Having the students do their work in a controlled environment is the obvious solution to AI and many other problems related to academic integrity. Its also the only way that students can actually be held to the same standards. When I was a freshman in college with a 3.4 highschool GPA, I was absolutely gobsmacked by how many kids with perfect >= 4.0 GPAs couldn't pass the simple algebra test that the university administered to all undergraduates as a prerequisite for taking any advanced mathematics course.

Viliam1234 11mo ago
> I still don't understand why standardized testing gets so much pushback. Parents of the children who can't pass the standardized tests also get a vote.
hamdingers 11mo ago
> I still don't understand why standardized testing gets so much pushback. Goodhart's law.
- bmacho 11mo ago
  Nah. Goodhart's law is literally just "if you play a matrix game don't announce your pick in advance". It is not a real law, or not different from common sense. (By matrix game I mean what wiki calls "Normal form game[0]", e.g. rock-paper-scissors or prisoner's dilemma.) In education, regarding exams, Goodhart's law just means that you should randomize your test questions instead of telling the students the questions before the exam. Have a wide set of questions, randomize them. The only way for students to pass is to learn the material. A randomized standardized test is not more susceptible to Goodhart's law than a randomized personal test. The latter however has many additional problems. [0] : https://en.wikipedia.org/wiki/Normal-form_game https://en.wikipedia.org/wiki/Normal-form_game
  - jerf 11mo ago
    That's not even remotely true. A randomized standardized test will still have some domain that it chooses its questions from and that domain will be perfectly susceptible to Goodhart's Law. It is already the case that no one is literally teaching "On the SAT you're going to get this problem about triangle similarity and the answer is C." When a fresh batch of students sits down in front of some year's SATs the test is still effectively "randomized" relative to the education they received. But that randomization is relative to a rigid standardized curriculum and the teaching was absolutely Goodhart'd relative to that curriculum. "The only way for students to pass is to learn the material." Part of Goodhart's law in this context is precisely that it overdetermines "the material" and there is no way around this. I wish Goodhart's law was as easy to dodge as you think it is, but it isn't.

eloisant 11mo ago

> In an age where having a degree is increasingly meaningless I wish I would agree with you, but I think that having a degree (or rather the right degree) is more important than ever. Basically grades exist to decide who gets a laid back high paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. As one teacher told me once: we could have all of you practice chess, make a big tournament and you get to choose your university based on your chess ranking. It wouldn't be any less stupid than the current system.

aleph_minus_one 11mo ago
> Basically grades exist to decide who gets a laid back high paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. Rather: Grades exist to decide who gets a stressful, but rather high-paying job, and who has to work 2 low paying labor intensive job just to live paycheck to paycheck. Laid-back high-paying job (unluckily) barely exist.