12 ms·

I teach computer science at a well-known, large and mediocre university. I see bimodal grade distributions all the time. The more technical a subject, the more

by mafribe 10y ago

I teach computer science at a well-known, large and mediocre university. I see bimodal grade distributions all the time. The more technical a subject, the more bimodal the grades. In fluff subjects like software engineering where grading relates to essay writing and group projects, grade distribution tends to be gaussian. OTOH in technical subjects where I grade by automated test suits, basically students fail almost all tests or pass almost all tests. These grade distributions have been constant, regardless of university, regardless of who teaches the course. There really is a sharp threshold that separates those who get programming from those who don't. And those who get programming really are a lot more productive than those who don't get it. Moreover, once you get it, learning further parts of CS (e.g. new algorithms, new programming languages, new OSes etc) is much easier then if you don't get it, and learning new things makes you even more powerful. The mastery of doing great things is joyful and makes you want to learn more. The frustration of failure drives you away from the subject. I find it plausible that grade distributions turn into similar income distributions later in life.

meric 10y ago
I have a theory where the bimodal distributions comes from two sets of people: one set focused only passing the course/assignment/test, the other set focused on actually knowing the content. Appearance and substance.
- mafribe 10y ago
  Interesting theory. I have a different one, derived from teaching 1000s of students over 2 decades. the bimodal distributions comes from two sets of people: one set working hard and putting the hours in until the master the subject, the other set being lazy, cheating, and don't put the hours in, so they never achieve mastery. Appearance and substance.
  - kgwgk 10y ago
    How is it different?
    - meric 10y ago
      "The ten-thousand thing-kinds are ultimately alike and ultimately different. This is called the great similarity-difference." - Zhuangzi
    - Bartweiss 10y ago
      In meric's model, the student who get good grades aren't the same ones achieving mastery. In mafribe's, they are.
      - meric 10y ago
        I must insist although "achieving mastery" is clearer than "knowing the content", I meant the same. It would probably help if I wrap "knowing" in asterisks like "knowing the content".
- splatcollision 10y ago
  FYI, you have an untested hypothesis, not a theory. Unless you have done some research somewhere. Sorry for the pedantics!
  - hueving 10y ago
    Poster was using the dictionary definition[1], the the scientific one. 1. http://www.merriam-webster.com/dictionary/theory http://www.merriam-webster.com/dictionary/theory
- bonoboTP 10y ago
  That's my experience too. There are those who want to get a degree with the least amount of work and don't really find the topic interesting. And then there are those who will discuss it and debate it before/after class, etc. Good test to tell them apart: if the lecturer mistakenly starts to talk about stuff that he already said in the last lecture (i.e. mistakenly doing the last one again), the first group will be happy and silent because there will be less stuff to learn for the test. The second group will tell the lecturer because they want to learn. This is then often mistaken by the first kind of people as trying to please the professor or "siding" with the prof instead of the fellow students. Because they think in a school-mindset.
- walshemj 10y ago
  I suspect that some do cs because they are interested and some because its a hot field or because of family/social pressures
- collyw 10y ago
  I did a batchelors in Molecular Biology years ago, and I noticed that some of my friends seemed to just learn the material, whereas I tended to understand the material.
deanCommie 10y ago
Actually doesn't your experience hint at the opposite trend in the working sector? If everyone (roughly) either fails programming, or aces it, then the working professionals are the ones who passed AND excelled.
- jon-wood 10y ago
  Oh if only that were true. I've worked with people in the past who know just enough to hack together a front end on top of a simple database, but who would completely fall apart when presented with anything more complex than that. These are the sort of people who when asked to build an internationalised website "solve" the problem by copying an entire PHP application into directories called "english" and "german" and then proceed to change the text in the German version.
  - charlesdm 10y ago
    :D
  - deleted 10y ago
    [deleted]
  - Bartweiss 10y ago
    There's that XKCD about asking how novice users solve computer problems. "I save the video to iCloud, then copy the url into this video downloader, then put the download into this transcription site, then upload that to..." I think this is a big chunk of explaining how bad programming goes, too. It's not "too incompetent to build a thing", it's "incompetent enough that everything runs on voodoo and evil".
- douche 10y ago
  In my experience, there is enough fluff and non-programming, abstract theory classes in a CS major that people who cannot program their way out of a wet paper bag can complete the degree, and even get high marks. These people can get jobs, and coast along for years at sub-mediocre levels of production, in most cases.
stdbrouw 10y ago
I don't think the conclusion follows from the data. It's not implausible that in computer science you need to "get" a whole bunch of things before it clicks and you start being productive, and a bimodal distribution of grades would then be indicative only of people who are further along in the learning process vs. those that have not yet reached that critical point, rather than any huge difference in intrinsic ability. This is just speculation, of course, but so is any other explanation and I prefer giving people the benefit of the doubt.
- mafribe 10y ago
  I agree, I don't think there is much difference in intrinsic ability. I'm absolutely convinced that every healthy person can become a really good programmer. But becoming a really good programmer requires putting in a lot of time doing programming, and a lot of students don't put in that time.
  - charlesdm 10y ago
    Well, if you enjoy something, it's much easier to put a lot of time in to figure it out. For example, I became a software entrepreneur about five years ago making money from my own projects. Dealing with a lot of business related "work", I actually discovered I also really like law, in particular tax law. A friend of mine was a pretty good lawyer (hung up his hat to do more entrepreneurial stuff) and I occasionally ask him to look over contracts I've drafted. He said some of the contracts I put together are better than what some of his colleagues put together. I've also defended myself in a few tax disputes with legal arguments, and always won. If I knew more about it before, I think I would've genuinely enjoyed being a lawyer. If you enjoy something, it's not that hard to become really good at something. If you don't, it is extremely hard to become better than the guy who loves what he does.
    - henrik_w 10y ago
      I think the question of whether anybody can get good at programming is very interesting. I often hear "anybody can do it, but you can only become really good if you enjoy it". So then the question is: why do you enjoy something? Are we born that way, or is it something we do? (I don't know what the answer is) Edit: also, it is interesting to replace "programmer" with "math professor". Can anybody become a good math professor if they enjoy doing it? If not, what is the difference (assuming it is true for programmers).
gotchange 10y ago
Or these tests were designed from the first moment to exhibit this distribution of grades since they favor only geniuses and insiders alike. I am not really buying that this is some kind of a reflection of a natural or deterministic phenomenon.
jackmott 10y ago
Please show data to support claim.
- Bartweiss 10y ago
  Check out this paper, which inspired Jeff Atwood's "two humps" model of programming education: https://web.archive.org/web/20090401003425/http://www.cs.mdx.ac.uk/research/PhDArea/saeed/ https://web.archive.org/web/20090401003425/http://www.cs.mdx... It's been retracted, but it's the only case I've ever seen where I found the retraction unconvincing. The retraction effectively says "After this paper came out, I made a lot of wild claims about it that the data didn't support. Therefore ignore the paper, even though the results replicated successfully."
Fiahil 10y ago
I had the same experience for the 3 years I was teaching assistant. Some people "clicked" very early and some didn't, but it's not set in stone. I've seen people succeed with only hard work and motivation. Once you've unlocked yourself there is not much differences between you and the ones who have unlocked themselves earlier.
driverdan 10y ago
> In fluff subjects like software engineering where grading relates to essay writing and group projects How is SE a "fluff subject?" Why are they writing essays? When I took SE we had individual programming projects and a group project to write a compiler. Not fluff at all.
- whorleater 10y ago
  SE is often relegated to a "fluff" subject in schools simply because it's, well, not very challenging. Classes based around the topic of SE are often topics like "Formal Software Development Methods" where you learn about verification of concurrent programs, formal symbolic testing, contracts, explicit model checking, etc. These are all "soft skills" and hard to directly generalize into a programming assignment, so students often times write essays formalizing their logic. Frankly, I'm surprised that your SE class did compilers at all, many schools in CS have their own "operating systems programming"/"low level programming"/"compiler design" sort of class, where the topic is fully fleshed out, rather than just a tiny project.
  - sidlls 10y ago
    Wow, things have changed since I took my SE electives (17-18 years ago). We had to write code and there were final projects that required the development of a non-trivial application. One that compiled and at least nominally worked. Granted I only took a course of about three electives at the sophomore and junior level, so maybe I missed the "fluff" in other levels.
    - whorleater 10y ago
      Yeah the first "formal" sophomore SE elective I took was literally "hey isn't OO the best? Don't you love drawing UML diagrams? What word is P-O-L-Y-M-O-R-P-H-I-S-M, and can you write a 1 page essay on it?" Once you get up to the actual SE electives, they're all about the formalization of software and how it can be represented. These classes are usually avoided by students as well, since students opt for fun (from the student's pov) classes like robotics programming, compiler design, web development, crypto, etc, which only makes the SE classes seem like more "fluff".
lucasnemeth 10y ago
There's a paper from the University of Toronto called "Evidence That Computer Science Grades Are Not Bimodal": http://www.cs.toronto.edu/~sme/papers/2016/icer_2016_bimodal.pdf http://www.cs.toronto.edu/~sme/papers/2016/icer_2016_bimodal...
Bartweiss 10y ago
I've held this basic theory for a while, and it seems to explain a lot of things about the state of software engineering. Most visibly, there's a wild disagreement about whether we have too many programmers, or not enough. Big tech companies consistently say that they struggle to find programmers, while contrarians at places like The Atlantic love writing pieces about how we're education way more programmers than we hire each year, and the bubble is going to burst. The ultra-cynical take is that companies are trying to drive a crash to drop salaries. It fits with the wage-fixing collusion, but doesn't really explain things like paying massive referral bonuses for new hires. The more interesting take is that it's about this bimodal distribution. "A degree in computer science" is a pretty amorphous thing, and thinkpiece writers who assume that every CS program produces employable CS-job-seekers aren't going to get decent data. So I think the explanation might be that CS isn't harder than EE or chemical engineering, but rather less prescriptive. There's a huge range of programs producing "CS degrees", and jobs titled "software engineering", and they're not really describing the same things.
- gautamdivgi 10y ago
  Then the point remains. Why are people who can't code getting the CS or EE degrees? Generally, the degree is supposed to indicate some sort of sufficiency with development. Probably universities are dumbing down requirements for $$$.
  - roel_v 10y ago
    Programming is to CS what telescopes are to astronomy.
    - sqeaky 10y ago
      Programming Telescopes is what CS is to Astronomy.
  - goostavos 10y ago
    >Probably universities are dumbing down requirements for $$$. I 100% believe this is true. When I was in college it was pretty staggering to see my fellow classmates receive passing grades and get shoveled through the programs despite never grasping the basic fundamentals of CS. These were people who, at the end of CS 101, still couldn't tell you the difference between a string and an int, who after CS 202 _still_ couldn't tell you the difference between a string and an int, and so on and so on. Yet happily through the system they continued. I eventually became part of the lowly drop-out club (I found attending college later in life to be a maddening experience of pointless busy work, rather than a challenging mind-expanding one). However, those people who fumbled around randomly swapping out types until things compile are now CS degree holders. It seems true even for the higher tier degrees too. It seems even a masters degree isn't a good indicator. In the past year alone I've watched several guys with masters enter and then promptly leave our startup due to having just absolutely no idea how to write software. :(
Alex3917 10y ago
> I find it plausible that grade distributions turn into similar income distributions later in life. Given that the people earning the most money aren't necessarily the ones with the most technical ability, why would this be the case?

by mafribe 10y ago

meric 10y ago

I have a theory where the bimodal distributions comes from two sets of people: one set focused only passing the course/assignment/test, the other set focused on actually knowing the content. Appearance and substance.

mafribe 10y ago
Interesting theory. I have a different one, derived from teaching 1000s of students over 2 decades. the bimodal distributions comes from two sets of people: one set working hard and putting the hours in until the master the subject, the other set being lazy, cheating, and don't put the hours in, so they never achieve mastery. Appearance and substance.
- kgwgk 10y ago
  How is it different?
  - meric 10y ago
    "The ten-thousand thing-kinds are ultimately alike and ultimately different. This is called the great similarity-difference." - Zhuangzi
  - Bartweiss 10y ago
    In meric's model, the student who get good grades aren't the same ones achieving mastery. In mafribe's, they are.
    - meric 10y ago
      I must insist although "achieving mastery" is clearer than "knowing the content", I meant the same. It would probably help if I wrap "knowing" in asterisks like "knowing the content".
splatcollision 10y ago
FYI, you have an untested hypothesis, not a theory. Unless you have done some research somewhere. Sorry for the pedantics!
- hueving 10y ago
  Poster was using the dictionary definition[1], the the scientific one. 1. http://www.merriam-webster.com/dictionary/theory http://www.merriam-webster.com/dictionary/theory
bonoboTP 10y ago
That's my experience too. There are those who want to get a degree with the least amount of work and don't really find the topic interesting. And then there are those who will discuss it and debate it before/after class, etc. Good test to tell them apart: if the lecturer mistakenly starts to talk about stuff that he already said in the last lecture (i.e. mistakenly doing the last one again), the first group will be happy and silent because there will be less stuff to learn for the test. The second group will tell the lecturer because they want to learn. This is then often mistaken by the first kind of people as trying to please the professor or "siding" with the prof instead of the fellow students. Because they think in a school-mindset.
walshemj 10y ago
I suspect that some do cs because they are interested and some because its a hot field or because of family/social pressures
collyw 10y ago
I did a batchelors in Molecular Biology years ago, and I noticed that some of my friends seemed to just learn the material, whereas I tended to understand the material.

deanCommie 10y ago

Actually doesn't your experience hint at the opposite trend in the working sector? If everyone (roughly) either fails programming, or aces it, then the working professionals are the ones who passed AND excelled.

jon-wood 10y ago
Oh if only that were true. I've worked with people in the past who know just enough to hack together a front end on top of a simple database, but who would completely fall apart when presented with anything more complex than that. These are the sort of people who when asked to build an internationalised website "solve" the problem by copying an entire PHP application into directories called "english" and "german" and then proceed to change the text in the German version.
- charlesdm 10y ago
  :D
- deleted 10y ago
  [deleted]
- Bartweiss 10y ago
  There's that XKCD about asking how novice users solve computer problems. "I save the video to iCloud, then copy the url into this video downloader, then put the download into this transcription site, then upload that to..." I think this is a big chunk of explaining how bad programming goes, too. It's not "too incompetent to build a thing", it's "incompetent enough that everything runs on voodoo and evil".
douche 10y ago
In my experience, there is enough fluff and non-programming, abstract theory classes in a CS major that people who cannot program their way out of a wet paper bag can complete the degree, and even get high marks. These people can get jobs, and coast along for years at sub-mediocre levels of production, in most cases.

stdbrouw 10y ago

I don't think the conclusion follows from the data. It's not implausible that in computer science you need to "get" a whole bunch of things before it clicks and you start being productive, and a bimodal distribution of grades would then be indicative only of people who are further along in the learning process vs. those that have not yet reached that critical point, rather than any huge difference in intrinsic ability. This is just speculation, of course, but so is any other explanation and I prefer giving people the benefit of the doubt.

mafribe 10y ago
I agree, I don't think there is much difference in intrinsic ability. I'm absolutely convinced that every healthy person can become a really good programmer. But becoming a really good programmer requires putting in a lot of time doing programming, and a lot of students don't put in that time.
- charlesdm 10y ago
  Well, if you enjoy something, it's much easier to put a lot of time in to figure it out. For example, I became a software entrepreneur about five years ago making money from my own projects. Dealing with a lot of business related "work", I actually discovered I also really like law, in particular tax law. A friend of mine was a pretty good lawyer (hung up his hat to do more entrepreneurial stuff) and I occasionally ask him to look over contracts I've drafted. He said some of the contracts I put together are better than what some of his colleagues put together. I've also defended myself in a few tax disputes with legal arguments, and always won. If I knew more about it before, I think I would've genuinely enjoyed being a lawyer. If you enjoy something, it's not that hard to become really good at something. If you don't, it is extremely hard to become better than the guy who loves what he does.
  - henrik_w 10y ago
    I think the question of whether anybody can get good at programming is very interesting. I often hear "anybody can do it, but you can only become really good if you enjoy it". So then the question is: why do you enjoy something? Are we born that way, or is it something we do? (I don't know what the answer is) Edit: also, it is interesting to replace "programmer" with "math professor". Can anybody become a good math professor if they enjoy doing it? If not, what is the difference (assuming it is true for programmers).

gotchange 10y ago

Or these tests were designed from the first moment to exhibit this distribution of grades since they favor only geniuses and insiders alike. I am not really buying that this is some kind of a reflection of a natural or deterministic phenomenon.

jackmott 10y ago

Please show data to support claim.

Bartweiss 10y ago
Check out this paper, which inspired Jeff Atwood's "two humps" model of programming education: https://web.archive.org/web/20090401003425/http://www.cs.mdx.ac.uk/research/PhDArea/saeed/ https://web.archive.org/web/20090401003425/http://www.cs.mdx... It's been retracted, but it's the only case I've ever seen where I found the retraction unconvincing. The retraction effectively says "After this paper came out, I made a lot of wild claims about it that the data didn't support. Therefore ignore the paper, even though the results replicated successfully."

Fiahil 10y ago

I had the same experience for the 3 years I was teaching assistant. Some people "clicked" very early and some didn't, but it's not set in stone. I've seen people succeed with only hard work and motivation. Once you've unlocked yourself there is not much differences between you and the ones who have unlocked themselves earlier.

driverdan 10y ago

> In fluff subjects like software engineering where grading relates to essay writing and group projects How is SE a "fluff subject?" Why are they writing essays? When I took SE we had individual programming projects and a group project to write a compiler. Not fluff at all.

whorleater 10y ago
SE is often relegated to a "fluff" subject in schools simply because it's, well, not very challenging. Classes based around the topic of SE are often topics like "Formal Software Development Methods" where you learn about verification of concurrent programs, formal symbolic testing, contracts, explicit model checking, etc. These are all "soft skills" and hard to directly generalize into a programming assignment, so students often times write essays formalizing their logic. Frankly, I'm surprised that your SE class did compilers at all, many schools in CS have their own "operating systems programming"/"low level programming"/"compiler design" sort of class, where the topic is fully fleshed out, rather than just a tiny project.
- sidlls 10y ago
  Wow, things have changed since I took my SE electives (17-18 years ago). We had to write code and there were final projects that required the development of a non-trivial application. One that compiled and at least nominally worked. Granted I only took a course of about three electives at the sophomore and junior level, so maybe I missed the "fluff" in other levels.
  - whorleater 10y ago
    Yeah the first "formal" sophomore SE elective I took was literally "hey isn't OO the best? Don't you love drawing UML diagrams? What word is P-O-L-Y-M-O-R-P-H-I-S-M, and can you write a 1 page essay on it?" Once you get up to the actual SE electives, they're all about the formalization of software and how it can be represented. These classes are usually avoided by students as well, since students opt for fun (from the student's pov) classes like robotics programming, compiler design, web development, crypto, etc, which only makes the SE classes seem like more "fluff".

lucasnemeth 10y ago

There's a paper from the University of Toronto called "Evidence That Computer Science Grades Are Not Bimodal": http://www.cs.toronto.edu/~sme/papers/2016/icer_2016_bimodal.pdf http://www.cs.toronto.edu/~sme/papers/2016/icer_2016_bimodal...

Bartweiss 10y ago

I've held this basic theory for a while, and it seems to explain a lot of things about the state of software engineering. Most visibly, there's a wild disagreement about whether we have too many programmers, or not enough. Big tech companies consistently say that they struggle to find programmers, while contrarians at places like The Atlantic love writing pieces about how we're education way more programmers than we hire each year, and the bubble is going to burst. The ultra-cynical take is that companies are trying to drive a crash to drop salaries. It fits with the wage-fixing collusion, but doesn't really explain things like paying massive referral bonuses for new hires. The more interesting take is that it's about this bimodal distribution. "A degree in computer science" is a pretty amorphous thing, and thinkpiece writers who assume that every CS program produces employable CS-job-seekers aren't going to get decent data. So I think the explanation might be that CS isn't harder than EE or chemical engineering, but rather less prescriptive. There's a huge range of programs producing "CS degrees", and jobs titled "software engineering", and they're not really describing the same things.

gautamdivgi 10y ago
Then the point remains. Why are people who can't code getting the CS or EE degrees? Generally, the degree is supposed to indicate some sort of sufficiency with development. Probably universities are dumbing down requirements for $$$.
- roel_v 10y ago
  Programming is to CS what telescopes are to astronomy.
  - sqeaky 10y ago
    Programming Telescopes is what CS is to Astronomy.
- goostavos 10y ago
  >Probably universities are dumbing down requirements for $$$. I 100% believe this is true. When I was in college it was pretty staggering to see my fellow classmates receive passing grades and get shoveled through the programs despite never grasping the basic fundamentals of CS. These were people who, at the end of CS 101, still couldn't tell you the difference between a string and an int, who after CS 202 _still_ couldn't tell you the difference between a string and an int, and so on and so on. Yet happily through the system they continued. I eventually became part of the lowly drop-out club (I found attending college later in life to be a maddening experience of pointless busy work, rather than a challenging mind-expanding one). However, those people who fumbled around randomly swapping out types until things compile are now CS degree holders. It seems true even for the higher tier degrees too. It seems even a masters degree isn't a good indicator. In the past year alone I've watched several guys with masters enter and then promptly leave our startup due to having just absolutely no idea how to write software. :(

Alex3917 10y ago

> I find it plausible that grade distributions turn into similar income distributions later in life. Given that the people earning the most money aren't necessarily the ones with the most technical ability, why would this be the case?