7 ms·

> And we tuned it from there to into its own sort of mini-SAT with an accompanying preparatory industry and culture. Agreed - interview prep is a cottage indus

> And we tuned it from there to into its own sort of mini-SAT with an accompanying preparatory industry and culture. Agreed - interview prep is a cottage industry. It should be worth reflecting and discussing why that is. If tech interviews are not horribly flawed, why is there a small industry of paid prep resources for passing those interviews?

whimsicalism 5y ago
There are plenty of industries that have cropped up on convincing people they need something despite the fact that they probably don't need it. For instance: SAT prep schools, the evidence on private college counselors, etc. - there is little to no evidence these actually have a substantial impact, but people pay for it anyways. In the adult world, try managed funds. Again, little-to-no evidence that you have any way of knowing ahead of time which of these will beat the market and which will underperform, but people still pay massive fees to get included in an "exclusive" investment fund. This is a human phenomena, it doesn't mean the selection criteria isn't working - just like the exist of managed funds doesn't mean the stock market isn't working. The world is full of lots of people, and some proportion of those people will be marks.
- Apocryphon 5y ago
  > there is little to no evidence these actually have a substantial impact Is that true? https://slate.com/technology/2019/04/sat-prep-courses-do-they-work-bias.html https://slate.com/technology/2019/04/sat-prep-courses-do-the... > Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University. > A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37. Also, managed funds are not analogous to interview/test preparation in any way.
  - whimsicalism 5y ago
    That demonstrates my point. 10 to 20 points on a 2400 scale (at the time of study) test is not a substantial impact. I also think there are probably test-oriented lurking variables they are not able to capture. Managed funds also sell you on the idea that you can buy your way into performance, and there is also little evidence that they actually work.
    - Apocryphon 5y ago
      Read on to the conclusion: > Let’s assume the effects of short-term coaching are really just a 20- or 30-point jump in students’ scores. That means they ought to be irrelevant to college admissions officers. Briggs found otherwise, however. Analyzing a 2008 survey conducted by the National Association for College Admission Counseling, he noted that one-third of respondents described a jump from 750 to 770 on the math portion of the SAT as having a significant effect on a student’s chances of admissions, and this was true among counselors at more and less selective schools alike. Even a minor score improvement for a high-achieving student, then—and one that falls within the standard measurement error for the test—can make a real difference. > Compare that with students from families that earn less than $20,000 per year, who have a mean score on the math portion of the SAT around 450. According to the same admissions counselor survey, a 20-point improvement to a score in this range would have no practical meaning for students who are trying to get into more selective schools. Even when it comes to less selective schools, just 20 percent of the counselors said a boost from 430 to 450 would make a difference. In other words, research has debunked the myth that pricey test prep gives a major bump to students’ scores, but it’s also hinted that whatever modest bumps they do provide are more likely to help the people who are already at the top. ====== As for managed funds, they promise to sell performance within an environment composed of many many people. Test prep and interview prep promise to improve performance for individuals. One is about mastery over an external environment, nature. The other is about internal improvement. I don't think promises of personal improvement through coaching, training, or teaching is comparable to promises of gains within a large chaotic unpredictable environment. They're entirely different categories of "buying your way into performance."
      - 5y ago

whimsicalism 5y ago

There are plenty of industries that have cropped up on convincing people they need something despite the fact that they probably don't need it. For instance: SAT prep schools, the evidence on private college counselors, etc. - there is little to no evidence these actually have a substantial impact, but people pay for it anyways. In the adult world, try managed funds. Again, little-to-no evidence that you have any way of knowing ahead of time which of these will beat the market and which will underperform, but people still pay massive fees to get included in an "exclusive" investment fund. This is a human phenomena, it doesn't mean the selection criteria isn't working - just like the exist of managed funds doesn't mean the stock market isn't working. The world is full of lots of people, and some proportion of those people will be marks.

Apocryphon 5y ago
> there is little to no evidence these actually have a substantial impact Is that true? https://slate.com/technology/2019/04/sat-prep-courses-do-they-work-bias.html https://slate.com/technology/2019/04/sat-prep-courses-do-the... > Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University. > A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37. Also, managed funds are not analogous to interview/test preparation in any way.
- whimsicalism 5y ago
  That demonstrates my point. 10 to 20 points on a 2400 scale (at the time of study) test is not a substantial impact. I also think there are probably test-oriented lurking variables they are not able to capture. Managed funds also sell you on the idea that you can buy your way into performance, and there is also little evidence that they actually work.
  - Apocryphon 5y ago
    Read on to the conclusion: > Let’s assume the effects of short-term coaching are really just a 20- or 30-point jump in students’ scores. That means they ought to be irrelevant to college admissions officers. Briggs found otherwise, however. Analyzing a 2008 survey conducted by the National Association for College Admission Counseling, he noted that one-third of respondents described a jump from 750 to 770 on the math portion of the SAT as having a significant effect on a student’s chances of admissions, and this was true among counselors at more and less selective schools alike. Even a minor score improvement for a high-achieving student, then—and one that falls within the standard measurement error for the test—can make a real difference. > Compare that with students from families that earn less than $20,000 per year, who have a mean score on the math portion of the SAT around 450. According to the same admissions counselor survey, a 20-point improvement to a score in this range would have no practical meaning for students who are trying to get into more selective schools. Even when it comes to less selective schools, just 20 percent of the counselors said a boost from 430 to 450 would make a difference. In other words, research has debunked the myth that pricey test prep gives a major bump to students’ scores, but it’s also hinted that whatever modest bumps they do provide are more likely to help the people who are already at the top. ====== As for managed funds, they promise to sell performance within an environment composed of many many people. Test prep and interview prep promise to improve performance for individuals. One is about mastery over an external environment, nature. The other is about internal improvement. I don't think promises of personal improvement through coaching, training, or teaching is comparable to promises of gains within a large chaotic unpredictable environment. They're entirely different categories of "buying your way into performance."
    - 5y ago