9 ms·

CS50 was a gigantic intro class designed to whet non-comp-sci students appetite for coding and computer science, generally. (I took it when it was C/PHP, so the

CS50 was a gigantic intro class designed to whet non-comp-sci students appetite for coding and computer science, generally. (I took it when it was C/PHP, so they may have changed things.) While the lectures are very straightforward, the smaller 'sections' lectures with the dozen-or-so teaching fellows were more intensive, and p-sets were pretty challenging for newbies. The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality. The assignments were often automatically graded upon submission and you often have some equivalent of a unit test to see if you got it right before submission, so you know if you got it wrong, but not how wrong. Just having a resource for sanity checks and walkthroughs of lesson concepts without trudging to the hall-o-TAs is prolly really useful. Totally legit philosophical debates aside, I think this is a great use of the technology. It's a very, very well-funded class (bigger dedicated full-time year-round staff than some entire departments,) so I'm sure it's not a half-assed effort. Malan is a great lecturer and his CS50 classes are all on YouTube. I sometimes send new coders to them but they're a fun watch, in general.

eesmith 3y ago
I find it hard to reconcile "The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality." with "very, very well-funded class". Why didn't the funding go to hiring more staff? My experience in a state school had far more access to education staff than what you describe, and for less funding.
- vanviegen 3y ago
  My reading is that this class just has a huge number of students, each bringing in a bit of funding. This allows for relatively large investments in the quality of the teaching materials, but teacher/student ratios will probably not be great. Hiring capable teachers is hard, especially at scale.
  - eesmith 3y ago
    I must be missing something. Harvard has 7,153 undergrads says Wikipedia. At most that's 1,790 students per year taking the class each year. But surely not everyone is taking the course? So why does it need to scale? Harvard tuition is $54,269, says https://registrar.fas.harvard.edu/tuition-and-fees https://registrar.fas.harvard.edu/tuition-and-fees . Assuming it's a 3 hour class with a 120 credit hours to graduate, that's 30 hours per year, so each student pays about $5,000 for what seems to be a horrible staff/student ratio. FWIW, I found https://cse.engin.umich.edu/stories/popular-intro-cs-course-continues-to-grow-over-870-students-present-final-projects-at-showcase https://cse.engin.umich.edu/stories/popular-intro-cs-course-... as a comparison, the EECS 183 course at the University of Michigan. Three professors (Dr. Dorf, Dr. Bill Arthur, and Dr. Héctor García-Ramírez), and 24 student instructors for the 870 students who finished the course. (It doesn't say how many started the course.) Full-time tuition at UMich is $8,780/in-state and $27,663/out-of-state. Without class size numbers for Harvard, I can't directly compare the two. I can understand that the distance learning version of the course does need to scale. But chefandy was talking about the in-person version of the course. (Unless "in a giant hall waiting" means some sort of VR-based course. :)
    - wil421 3y ago
      My coworker’s kid just graduated from Harvard. He said it cost about $315,000k. Does your number include all the fees and such most colleges tack on to tuition?
      - rocho 3y ago
        You probably mean $315k.
        deleted 3y ago
        [deleted]
        chefandy 3y ago
        If they lived in off-campus housing in Cambridge then $315m would be on the low side.
- vasco 3y ago
  Feom what I understand of these universities with endowments they use money to hire administrative staff and to "last forever". Teaching staff is secondary to those.
  - chefandy 3y ago
    Yes. They lay off scores of staff in nearly every economic downturn. There's a whole lot of 'blah blah blah' about why they can't use the endowment to avoid screwing people who've given decades of their professional life to Harvard, but I think it mostly comes down to their wanting a big shiny endowment balance long after everyone on earth right now has died.
- chefandy 3y ago
  Harvard is bizzaro land. I don't know what CS50s setup looks like internally, but after taking the course, and working there for a couple of decades— one full time— I think I'm pretty close. I learned that they always have money for stuff but rarely for staff. The class seemed to have significant corporate support from Facebook and Microsoft, and probably others— both Zuck and Ballmer have given guest lectures— but I'll bet that Harvard's governance rules didn't let them hire academic staff with outside funding or something like that. They def could use it on regular staff positions— I spent years in a full-time, permanent staff position that was funded through a project partner. The support academics in CS50 are all done by student TAs at the lowest level, or teaching fellows— grad students maybe— for lecturing or more advanced help. The staff are for things like video production, technical infrastructure, etc. There's only so many qualified students willing to do that work and no way you're getting existing faculty to do it. I'll also bet that they have rules about hiring staff to do anything resembling instructional work. Like I said, though, I don't know what the situation is exactly but I'm pretty familiar with Harvard.
  - lioeters 3y ago
    > grad students Why buy the cow, amirite?
    - chefandy 3y ago
      Yeah though they're unionized now I think. I wonder if anything changed with cs50?
  - chefandy 3y ago
    > I learned that they always have money for stuff but rarely for staff. (By "they," I mean Harvard, generally. I have no reason to believe CS50 is exempt from this, but I have no special insight, here.)
- some_random 3y ago
  Yeah, not exactly helping with my prior of "college is a massive, societal level scam"
  - chefandy 3y ago
    College is great and very useful, it's just heavily abused by our society. I think that a lot of well-meaning people in the 80s and 90s saw that people with college degrees were better off, and confusing correlation with causation, insisted everyone needed to go to college so they could also be better off. Well, now we have way more people with college degrees than we need. They've taken on debt that their (government-employed) guidance counselors insisted would pay for itself. Decent paying white collar positions for non-degree holders, like administrative assistant, now require them because of lazy recruiting. And income inequality is worse because you can't resolve structural economic issues with credentials. So you have someone with an expensive English degree from BU answering phones in a CS pool for 35k, and the 'dumb' kids who went to trade school and became electricians have to turn away work at hundreds of dollars per hour because they're in such high demand.
- devwastaken 3y ago
  Harvard isn't about learning. It's about paying the TA. Connections and money.
Paul-Craft 3y ago
> The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality. The assignments were often automatically graded upon submission and you often have some equivalent of a unit test to see if you got it right before submission, so you know if you got it wrong, but not how wrong. Just having a resource for sanity checks and walkthroughs of lesson concepts without trudging to the hall-o-TAs is prolly really useful. It's been a while since I was an undergrad, and I didn't study CS when I was (I was in math -- we didn't even get the unit test to tell us we were wrong lol). I understand auto-grading like this is very common in lower level CS classes, but wouldn't it be possible to write these "test suites" in such a way that they could provide some hints on exactly what was wrong? I have to imagine, based on my experience as a graduate TA in math, that undergrads as a whole make mistakes that typically fall into certain classes, and that those classes could be reflected or captured in the pattern of which "unit tests" end up failing. It doesn't strike me as terribly difficult to analyze these patterns of failures, after running the course at least once, and come up with useful hints as feedback. Does nobody do this? Or, perhaps more to the point, does nobody ask their TAs to do this? > Totally legit philosophical debates aside, I think this is a great use of the technology. It's a very, very well-funded class (bigger dedicated full-time year-round staff than some entire departments,) so I'm sure it's not a half-assed effort. I agree broadly with this. If you're going to do this experiment, in principle, a class like Harvard CS-50 sounds like the right place to start, for the reasons you've listed. However, if this chatbot is just an equivalent of ChatGPT with the GPT-4 model and some additional training/fine tuning, I don't think the technology is quite there yet. We've seen a lot of examples already of how GPT-x has a tendency to give "confidently incorrect" answers, and that's the absolute last thing you want to be telling beginners. I've seen answers that could plausibly fool experts, yet be wrong enough as to be useless once you try to verify the details. The danger here is that beginners may not recognize this, and may not be able to verify the details. That said, I think there is a future for this sort of technology in precisely this type of instructional environment. The "confidently incorrect" problem is something that can be mitigated with fine-tuning, proper prompt engineering, and other techniques, but, to my knowledge, can't be eliminated if the system is essentially a bare text interface into an LLM. I think the work being done with web search is a possible direction here, since I've seen GPT-4 essentially give up and go search the web when pressed for details or when the actual answer would become too complex. For a restricted domain like CS50, maybe even some kind of formally encoded knowledge base would help it (similar to how I mentioned analyzing patterns of failures to provided hints). What I do know is that the gigantic "hall-o-TAs" is absolutely not a great user experience for undergrad learners, so anything that offers the potential for improvement on that UX is definitely something I'd like to see pursued and developed.
chefandy 3y ago
(to be clear, I was never a Harvard College undergrad. I was a degree candidate in a Harvard Extension program before transferring, and I worked for Harvard for over two decades, one of them full-time.)

eesmith 3y ago

I find it hard to reconcile "The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality." with "very, very well-funded class". Why didn't the funding go to hiring more staff? My experience in a state school had far more access to education staff than what you describe, and for less funding.

vanviegen 3y ago
My reading is that this class just has a huge number of students, each bringing in a bit of funding. This allows for relatively large investments in the quality of the teaching materials, but teacher/student ratios will probably not be great. Hiring capable teachers is hard, especially at scale.
- eesmith 3y ago
  I must be missing something. Harvard has 7,153 undergrads says Wikipedia. At most that's 1,790 students per year taking the class each year. But surely not everyone is taking the course? So why does it need to scale? Harvard tuition is $54,269, says https://registrar.fas.harvard.edu/tuition-and-fees https://registrar.fas.harvard.edu/tuition-and-fees . Assuming it's a 3 hour class with a 120 credit hours to graduate, that's 30 hours per year, so each student pays about $5,000 for what seems to be a horrible staff/student ratio. FWIW, I found https://cse.engin.umich.edu/stories/popular-intro-cs-course-continues-to-grow-over-870-students-present-final-projects-at-showcase https://cse.engin.umich.edu/stories/popular-intro-cs-course-... as a comparison, the EECS 183 course at the University of Michigan. Three professors (Dr. Dorf, Dr. Bill Arthur, and Dr. Héctor García-Ramírez), and 24 student instructors for the 870 students who finished the course. (It doesn't say how many started the course.) Full-time tuition at UMich is $8,780/in-state and $27,663/out-of-state. Without class size numbers for Harvard, I can't directly compare the two. I can understand that the distance learning version of the course does need to scale. But chefandy was talking about the in-person version of the course. (Unless "in a giant hall waiting" means some sort of VR-based course. :)
  - wil421 3y ago
    My coworker’s kid just graduated from Harvard. He said it cost about $315,000k. Does your number include all the fees and such most colleges tack on to tuition?
    - rocho 3y ago
      You probably mean $315k.
      - deleted 3y ago
        [deleted]
      - chefandy 3y ago
        If they lived in off-campus housing in Cambridge then $315m would be on the low side.
vasco 3y ago
Feom what I understand of these universities with endowments they use money to hire administrative staff and to "last forever". Teaching staff is secondary to those.
- chefandy 3y ago
  Yes. They lay off scores of staff in nearly every economic downturn. There's a whole lot of 'blah blah blah' about why they can't use the endowment to avoid screwing people who've given decades of their professional life to Harvard, but I think it mostly comes down to their wanting a big shiny endowment balance long after everyone on earth right now has died.
chefandy 3y ago
Harvard is bizzaro land. I don't know what CS50s setup looks like internally, but after taking the course, and working there for a couple of decades— one full time— I think I'm pretty close. I learned that they always have money for stuff but rarely for staff. The class seemed to have significant corporate support from Facebook and Microsoft, and probably others— both Zuck and Ballmer have given guest lectures— but I'll bet that Harvard's governance rules didn't let them hire academic staff with outside funding or something like that. They def could use it on regular staff positions— I spent years in a full-time, permanent staff position that was funded through a project partner. The support academics in CS50 are all done by student TAs at the lowest level, or teaching fellows— grad students maybe— for lecturing or more advanced help. The staff are for things like video production, technical infrastructure, etc. There's only so many qualified students willing to do that work and no way you're getting existing faculty to do it. I'll also bet that they have rules about hiring staff to do anything resembling instructional work. Like I said, though, I don't know what the situation is exactly but I'm pretty familiar with Harvard.
- lioeters 3y ago
  > grad students Why buy the cow, amirite?
  - chefandy 3y ago
    Yeah though they're unionized now I think. I wonder if anything changed with cs50?
- chefandy 3y ago
  > I learned that they always have money for stuff but rarely for staff. (By "they," I mean Harvard, generally. I have no reason to believe CS50 is exempt from this, but I have no special insight, here.)
some_random 3y ago
Yeah, not exactly helping with my prior of "college is a massive, societal level scam"
- chefandy 3y ago
  College is great and very useful, it's just heavily abused by our society. I think that a lot of well-meaning people in the 80s and 90s saw that people with college degrees were better off, and confusing correlation with causation, insisted everyone needed to go to college so they could also be better off. Well, now we have way more people with college degrees than we need. They've taken on debt that their (government-employed) guidance counselors insisted would pay for itself. Decent paying white collar positions for non-degree holders, like administrative assistant, now require them because of lazy recruiting. And income inequality is worse because you can't resolve structural economic issues with credentials. So you have someone with an expensive English degree from BU answering phones in a CS pool for 35k, and the 'dumb' kids who went to trade school and became electricians have to turn away work at hundreds of dollars per hour because they're in such high demand.
devwastaken 3y ago
Harvard isn't about learning. It's about paying the TA. Connections and money.

Paul-Craft 3y ago

> The weekly help sessions ended up being a million lost students in a giant hall waiting to see one of a few dozen TAs offering uneven pedagogical quality. The assignments were often automatically graded upon submission and you often have some equivalent of a unit test to see if you got it right before submission, so you know if you got it wrong, but not how wrong. Just having a resource for sanity checks and walkthroughs of lesson concepts without trudging to the hall-o-TAs is prolly really useful. It's been a while since I was an undergrad, and I didn't study CS when I was (I was in math -- we didn't even get the unit test to tell us we were wrong lol). I understand auto-grading like this is very common in lower level CS classes, but wouldn't it be possible to write these "test suites" in such a way that they could provide some hints on exactly what was wrong? I have to imagine, based on my experience as a graduate TA in math, that undergrads as a whole make mistakes that typically fall into certain classes, and that those classes could be reflected or captured in the pattern of which "unit tests" end up failing. It doesn't strike me as terribly difficult to analyze these patterns of failures, after running the course at least once, and come up with useful hints as feedback. Does nobody do this? Or, perhaps more to the point, does nobody ask their TAs to do this? > Totally legit philosophical debates aside, I think this is a great use of the technology. It's a very, very well-funded class (bigger dedicated full-time year-round staff than some entire departments,) so I'm sure it's not a half-assed effort. I agree broadly with this. If you're going to do this experiment, in principle, a class like Harvard CS-50 sounds like the right place to start, for the reasons you've listed. However, if this chatbot is just an equivalent of ChatGPT with the GPT-4 model and some additional training/fine tuning, I don't think the technology is quite there yet. We've seen a lot of examples already of how GPT-x has a tendency to give "confidently incorrect" answers, and that's the absolute last thing you want to be telling beginners. I've seen answers that could plausibly fool experts, yet be wrong enough as to be useless once you try to verify the details. The danger here is that beginners may not recognize this, and may not be able to verify the details. That said, I think there is a future for this sort of technology in precisely this type of instructional environment. The "confidently incorrect" problem is something that can be mitigated with fine-tuning, proper prompt engineering, and other techniques, but, to my knowledge, can't be eliminated if the system is essentially a bare text interface into an LLM. I think the work being done with web search is a possible direction here, since I've seen GPT-4 essentially give up and go search the web when pressed for details or when the actual answer would become too complex. For a restricted domain like CS50, maybe even some kind of formally encoded knowledge base would help it (similar to how I mentioned analyzing patterns of failures to provided hints). What I do know is that the gigantic "hall-o-TAs" is absolutely not a great user experience for undergrad learners, so anything that offers the potential for improvement on that UX is definitely something I'd like to see pursued and developed.

chefandy 3y ago

(to be clear, I was never a Harvard College undergrad. I was a degree candidate in a Harvard Extension program before transferring, and I worked for Harvard for over two decades, one of them full-time.)