7 ms·

(squeezing eyes shut and focusing my psychic powers) This college I've never heard of will turn out to have tiny enrollment, and the entire effect size will be

(squeezing eyes shut and focusing my psychic powers) This college I've never heard of will turn out to have tiny enrollment, and the entire effect size will be selection bias + random noise. >Olin College, a small engineering school in Needham, MA, which graduates around 75 students per year, turns out an alumni population where 2.77% of alumni found a successfully venture-backed startup, more than five times the rate of Stanford (0.51%), MIT (0.75%), Harvard (0.28%), Wow, who would have guessed. https://fredrikdeboer.com/2017/03/29/why-selection-bias-is-the-most-powerful-force-in-education/ https://fredrikdeboer.com/2017/03/29/why-selection-bias-is-t...

jostmey 9y ago
And maybe that's how you end up with the harvards and yales. The initial reputation is luck and the schools coast on it. Smart talented people attend but maybe they were going to succeed anyway
mathattack 9y ago
That might be extreme. We'd expect the largest density to come from a small school, but Olin also as the benefit of being 100% engineers with much less debt than comparable schools. That contributes a lot too. If it were purely size plus noise there would be a few other weird schools on the list. The bigger question isn't funding, it's how they'll do. No unicorns yet, let alone massive exits. Win the market share for that, and I'll really be impressed.
ISL 9y ago
2/75 = 0.027
cjbprime 9y ago
Sorry, I guess I'm being slow -- I get that the sample size is too small to confirm the effect, but I'm not seeing where the selection bias is. Is it that the type of people who would be attracted to founding a startup are more likely to apply/be accepted to Olin than Stanford? That seems to confirm the original article's thesis rather than invalidate it, in my eyes, so I don't expect that's what you're thinking..
- koolba 9y ago
  With enough tiny schools you can find outliers for anything. Give me the stats and I'll show you why a seemingly random school is the epicenter of modern opioid addiction.
  - btilly 9y ago
    There aren't enough tiny schools to explain this result. The p-value against this result if this school was as good as Stanford by pure chance is 1/10k. There are only 3k 4 year colleges in the USA. The vast majority of which are significantly worse than Stanford. It is therefore extremely unlikely that pure chance alone would explain this data point.
- rjtobin 9y ago
  One source of bias here: there are lots of small schools, chances are high that one of them will exhibit a large number of founders due to noise, and this will be the one picked up by the report cited.
- kolbe 9y ago
  It's possible that Olin exclusively having engineering degrees produces a bias. Stanford has a significant population who study subjects that do not aid in pursuing a career in entrepreneurship.
- sbierwagen 9y ago
  If you assume that, out of all the students who applied to Olin, only 70 were entrepreneur-y, then a class size of 75 would capture all of them, and Olin would have an impressive number of successful entrepreneurs among its alumni. But it doesn't scale. If you made Olin accept ten times as many students, then its numbers would regress to the mean.
pbiggar 9y ago
I asked my cofounder about this (she's mentioned in the article). She said "This occurred to us. We're a school of engineers - we talk about sample sizes a lot"? Here's her thoughts on why it's not just sample size: https://twitter.com/ellenchisa/status/905448602142572545 https://twitter.com/ellenchisa/status/905448602142572545
- valuearb 9y ago
  None of her arguments make any sense. Most apply to any school.
  - pbiggar 9y ago
    Would you care to provide evidence for your aggressively-worded comment?
    - bllguo 9y ago
      Hypothesis 3 - that alumni or faculty are supportive - applies to any number of schools. Need more to show this is a distinguishing factor. Hypothesis 2 - the school encourages growth and "spiral learning" - is vague. Going to need more evidence to prove this is a unique characteristic of Olin. Hypothesis 1 - the school attracts students with a certain risk profile - is precisely the parent's point about selection bias. There is a healthy discussion of this above.
      - BeetleB 9y ago
        Your requests for info are fair, but for 2 and 3, it's not hard to find hints of evidence with a mere Google search. I did not go to Olin, but knew about them from when they were fairly new. Their very existence from the first day has been to produce entrepreneurial students, and their curriculum has always reflected that. IIRC, they don't separate courses based on "engineering" and "entrepreneurship". They aim for every course they teach to be a mix of both. Very few pure "theory" courses. Taking a course in communications? You will learn both the theory (Shannon, etc) and you better build something significant by the end of the semester. Honestly, from the comments here, I'm surprised at the level of doubt. I thought more people here would be familiar with Olin. (None of this is to say that they are or are not effective in 2 & 3, but it is a foundational principle of theirs, and I imagine a metric they track - as opposed to typical top schools that claim 3 but are pretty bad at it).
arcticfox 9y ago
Actually, it's not selection bias at all! This result wasn't just drawn out of a 1000-count small college hat, so naive Bayesian priors shouldn't be used here. As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do. So my priors are that Olin is likely to do this at a higher rate (since Stanford / MIT etc. have entire swathes of the school not interested in entrepreneurship at all); this evidence makes that seem overwhelmingly likely now. Consider evaluating all the departments of a school and finding out that one of the smallest turns out the highest percentage of high-energy physicists. Selection bias? Nope, just the high-energy physics department.
- sbierwagen 9y ago
  >Actually, it's not selection bias at all! [...] As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do. From the link: >In other words, what we might have perceived as a difference in education quality was really the product of systematic differences in how the considered populations were put together. The groups we considered had a hidden non-random distribution. This is selection bias. As you just stated, entrepreneur-y students self-select into Olin. The fact the school produces entrepreneurs doesn't have anything to do with the teachers, the curriculum, or the chemicals Olin puts in the drinking water. It's the non-random distribution of students.
  - pataoAoC 9y ago
    It depends what the conclusion in question is, whether selection bias is a factor. Does Olin produce / graduate founders at ~5 times the rate of Stanford? Yes - no bias. Is the effect size due to self-selection? Likely at least partially.
    - valuearb 9y ago
      If by "partially" you mean "entirely" I agree.
      - 9y ago
blfr 9y ago
Tiny enrolment is a good point but selection bias is not. You could easily argue that virtually all of college effects are selection effects.
- deleted 9y ago
  [deleted]
leblancfg 9y ago
I've regularly heard good things of Olin College in the past years, especially when it comes to teaching CS early to all engineering students. Was not surprised to read this headline, but have never met any grads personally.
wellok 9y ago
Well, it's easy to dismiss. Do you have anything substantive to add to the discussion?

jostmey 9y ago

And maybe that's how you end up with the harvards and yales. The initial reputation is luck and the schools coast on it. Smart talented people attend but maybe they were going to succeed anyway

mathattack 9y ago

That might be extreme. We'd expect the largest density to come from a small school, but Olin also as the benefit of being 100% engineers with much less debt than comparable schools. That contributes a lot too. If it were purely size plus noise there would be a few other weird schools on the list. The bigger question isn't funding, it's how they'll do. No unicorns yet, let alone massive exits. Win the market share for that, and I'll really be impressed.

ISL 9y ago

2/75 = 0.027

cjbprime 9y ago

Sorry, I guess I'm being slow -- I get that the sample size is too small to confirm the effect, but I'm not seeing where the selection bias is. Is it that the type of people who would be attracted to founding a startup are more likely to apply/be accepted to Olin than Stanford? That seems to confirm the original article's thesis rather than invalidate it, in my eyes, so I don't expect that's what you're thinking..

koolba 9y ago
With enough tiny schools you can find outliers for anything. Give me the stats and I'll show you why a seemingly random school is the epicenter of modern opioid addiction.
- btilly 9y ago
  There aren't enough tiny schools to explain this result. The p-value against this result if this school was as good as Stanford by pure chance is 1/10k. There are only 3k 4 year colleges in the USA. The vast majority of which are significantly worse than Stanford. It is therefore extremely unlikely that pure chance alone would explain this data point.
rjtobin 9y ago
One source of bias here: there are lots of small schools, chances are high that one of them will exhibit a large number of founders due to noise, and this will be the one picked up by the report cited.
kolbe 9y ago
It's possible that Olin exclusively having engineering degrees produces a bias. Stanford has a significant population who study subjects that do not aid in pursuing a career in entrepreneurship.
sbierwagen 9y ago
If you assume that, out of all the students who applied to Olin, only 70 were entrepreneur-y, then a class size of 75 would capture all of them, and Olin would have an impressive number of successful entrepreneurs among its alumni. But it doesn't scale. If you made Olin accept ten times as many students, then its numbers would regress to the mean.

pbiggar 9y ago

I asked my cofounder about this (she's mentioned in the article). She said "This occurred to us. We're a school of engineers - we talk about sample sizes a lot"? Here's her thoughts on why it's not just sample size: https://twitter.com/ellenchisa/status/905448602142572545 https://twitter.com/ellenchisa/status/905448602142572545

valuearb 9y ago
None of her arguments make any sense. Most apply to any school.
- pbiggar 9y ago
  Would you care to provide evidence for your aggressively-worded comment?
  - bllguo 9y ago
    Hypothesis 3 - that alumni or faculty are supportive - applies to any number of schools. Need more to show this is a distinguishing factor. Hypothesis 2 - the school encourages growth and "spiral learning" - is vague. Going to need more evidence to prove this is a unique characteristic of Olin. Hypothesis 1 - the school attracts students with a certain risk profile - is precisely the parent's point about selection bias. There is a healthy discussion of this above.
    - BeetleB 9y ago
      Your requests for info are fair, but for 2 and 3, it's not hard to find hints of evidence with a mere Google search. I did not go to Olin, but knew about them from when they were fairly new. Their very existence from the first day has been to produce entrepreneurial students, and their curriculum has always reflected that. IIRC, they don't separate courses based on "engineering" and "entrepreneurship". They aim for every course they teach to be a mix of both. Very few pure "theory" courses. Taking a course in communications? You will learn both the theory (Shannon, etc) and you better build something significant by the end of the semester. Honestly, from the comments here, I'm surprised at the level of doubt. I thought more people here would be familiar with Olin. (None of this is to say that they are or are not effective in 2 & 3, but it is a foundational principle of theirs, and I imagine a metric they track - as opposed to typical top schools that claim 3 but are pretty bad at it).

arcticfox 9y ago

Actually, it's not selection bias at all! This result wasn't just drawn out of a 1000-count small college hat, so naive Bayesian priors shouldn't be used here. As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do. So my priors are that Olin is likely to do this at a higher rate (since Stanford / MIT etc. have entire swathes of the school not interested in entrepreneurship at all); this evidence makes that seem overwhelmingly likely now. Consider evaluating all the departments of a school and finding out that one of the smallest turns out the highest percentage of high-energy physicists. Selection bias? Nope, just the high-energy physics department.

sbierwagen 9y ago
>Actually, it's not selection bias at all! [...] As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do. From the link: >In other words, what we might have perceived as a difference in education quality was really the product of systematic differences in how the considered populations were put together. The groups we considered had a hidden non-random distribution. This is selection bias. As you just stated, entrepreneur-y students self-select into Olin. The fact the school produces entrepreneurs doesn't have anything to do with the teachers, the curriculum, or the chemicals Olin puts in the drinking water. It's the non-random distribution of students.
- pataoAoC 9y ago
  It depends what the conclusion in question is, whether selection bias is a factor. Does Olin produce / graduate founders at ~5 times the rate of Stanford? Yes - no bias. Is the effect size due to self-selection? Likely at least partially.
  - valuearb 9y ago
    If by "partially" you mean "entirely" I agree.
    - 9y ago

blfr 9y ago

Tiny enrolment is a good point but selection bias is not. You could easily argue that virtually all of college effects are selection effects.

deleted 9y ago
[deleted]

leblancfg 9y ago

I've regularly heard good things of Olin College in the past years, especially when it comes to teaching CS early to all engineering students. Was not surprised to read this headline, but have never met any grads personally.

wellok 9y ago

Well, it's easy to dismiss. Do you have anything substantive to add to the discussion?