7 ms·

OP here (shocked to see this on HN): The context is missing but is important. When I say MOOCs have failed I mean something very specific: asynchronous online

OP here (shocked to see this on HN): The context is missing but is important. When I say MOOCs have failed I mean something very specific: asynchronous online learning has proven to be so dramatically less effective than synchronous, guided learning. Better than nothing existing, but we’re learning that only a small subset of people are able to effectively learn from them. In other words, take a random sample of 100 people with the same level of motivation. 90% will complete a live, guided course. 5% will complete a MOOC of the same quality. Which is unfortunate because the live/synchronous part is what is expensive. There was a time when we thought all education was is just a bunch of video lectures, and that if we got a bunch of professors to record their lectures college would be free. We now know that doesn’t work, at least for the vast, vast majority of people, both because we’re unable to discipline ourselves and because most true learning happens in a multi-way environment with accountability measures.

kbutler 9y ago
How are you measuring the same level of motivation? From my anecdotal experience, the people taking the "live, guided course" have committed to: - synchronous attendance at a particular time, probably at a particular place - probably a dollar investment - face-to-face contact with instructor - near-traditional learning environment People taking a MOOC have committed to: - browsing on an electronic device There's generally a really high barrier to entry to the synchronous, guided learning, so it seems like comparing students who have paid tuition and enrolled in a class vs prospective students expressing interest on a university recruitment page.
- edraferi 9y ago
  I think the time commitment is the biggest issue. Let's assume the costs are the same to me (perhaps due to employer subsidy): When I take a MOOC, I'm pledging at least 5 hours a week. That covers 2 hours of videos, 2 hours of homework and 1 hour of additional research / review. If I pledge 5 hours /week to a traditional class, I don't get nearly as much learning. I have to commute to the class, which is at least 30 minutes each way. Now my 5 hours buys me 2 hours of class, 2 hours of commuting and 1 hour of homework. I've lost an hour of homework and an hour of research. I know which one is better for me.
- austenallred 9y ago
  There have been quite a few studies that have done this in a scientific manner. Internally we have taken sets of students that apply to our free “mini code bootcamp” and sent half to an archived version published each night and send the other half a live link that becomes an archive each night. So we didn’t control scientifically, jsut an AB test, but the shift has been breathtaking. We can also take students that are struggling learning from our MOOCs and they finish the live course almost every time. I know the latter is unscientific, but it becomes obvious over time.
  - kbutler 9y ago
    This Harvard study & Atlantic article by the study authors give the same 5% completion rate you cited, but give a completely opposite interpretation: basically, that the MOOC broadened access, allowing more people to get what they wanted out of the course. https://www.theatlantic.com/education/archive/2014/01/the-tricky-task-of-figuring-out-what-makes-a-mooc-successful/283274/ https://www.theatlantic.com/education/archive/2014/01/the-tr... https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 "In these course[sic], “dropping out” is not a breach of expectations but the natural result of an open, free, and asynchronous registration process, where students get just as much as they wish out of a course and registering for a course does not imply a commitment to completing it." ... Also the discussion of the "Colbert Bump", where increased publicity to less-motivated (just curious) students doubled completion but tripled registration (thus reducing completion rate). Also the point about whether to allow or disallow registration after the course begins (late registrants are ineligible for certification, so they are automatically "failed"). "HarvardX could boost its certification rate by closing those courses to new registrants or restrict courses to those most likely to complete them. Instead, it keeps courses open to maximize the number of students who are learning something new." I absolutely agree that (at least some) users can be helped by live instruction or supplemented online instruction, but I disagree that commonly cited, low completion rates measure students with the "same level of motivation".
  - gliese1337 9y ago
    As long which ones you send to each link is determined entirely randomly, you have indeed done a "scientific" (i.e., statistically valid) control.
dsr_ 9y ago
Major counterproof: The Open University. Sounds like MOOCs need to learn from educational institutions that actually have a track record in distance learning.
- tonyedgecombe 9y ago
  The big difference is you have to pay for the OU, there is a commitment up front that filters out a lot of people. Saying MOOCS are a failure because only 2% complete the course is like saying the OU is a failure because only 2% of their web site visitors sign up.
randomdata 9y ago
> 90% will complete a live, guided course. 5% will complete a MOOC of the same quality. What does complete mean here? Complete within a given timeframe? Doesn't that miss the entire gain that MOOCs bring to the table? Which is that education becomes a lifelong endeavour instead of something you try and fit into a few years and, for most, never return to again?
moosey 9y ago
What is the purpose of a MOOC? Is it to make training available, or to provide academic success? If it is the former, then it is definitely a success. If it is the latter, then I think that not only percentage rates of success are important, but also the nominal completion rate itself. A MOOC supplies the opportunity for someone getting a degree in communications, business, or some other not-entirely-technical degree, that might be wonder what a computer science or high level mathematics course is like to go and find out. Does their failure in such a course mean that the class itself is a failure, or can the fact that they had access and were able to put a toe in the water mark success? I would suggest the latter.
- austenallred 9y ago
  Right. They're only a failure inasmuch as the initial promise was "these will basically replace college but for free!" which is now clearly not true.
VLM 9y ago
Ah OK interesting I think I get what you are saying in that given an authority figure's specification, the student in a class will be far more likely to succeed at reaching that arbitrary externally imposed spec than a MOOC subscriber. With the side issue that 100% of everyone in my 20 person automata theory class back in ninety-something is a much smaller number than 2% of 24000 people signing up for Ullman's automata theory MOOC. The fundamental problem is authority along with excessive broadness. If I want to learn Y and the MOOC teaches X Y and Z, I will fail the MOOC. There are also Skinner box conditioning issues where my assignment for something like using bootstrap framework, will be scratching my own itch of project or doing some project at work, such that again I'll fail the MOOC because I'll refuse the assignment; I already have a more ambitious self assigned assignment and am not doing both my assignment and someone elses much less interesting assignment. Whereas if I sit in class I'll have peer pressure and financial pressure to do the class assignment regardless of work or hobby interests. Sort of a traditional in-person or online class is a goal in and of itself; MOOCs are a means to an end for work or hobby for most.
ThrustVectoring 9y ago
How well are you controlling for the sheepskin effect? Students are highly motivated to do what it takes to get a degree from a university, and much less motivated to actually learn the topics nominally offered. I suspect that a MOOC offered with the same incentives and structure as a live class - ie, as part of a degree-granting program with mandatory tests at set dates - would have significantly higher completion rates.

kbutler 9y ago

How are you measuring the same level of motivation? From my anecdotal experience, the people taking the "live, guided course" have committed to: - synchronous attendance at a particular time, probably at a particular place - probably a dollar investment - face-to-face contact with instructor - near-traditional learning environment People taking a MOOC have committed to: - browsing on an electronic device There's generally a really high barrier to entry to the synchronous, guided learning, so it seems like comparing students who have paid tuition and enrolled in a class vs prospective students expressing interest on a university recruitment page.

edraferi 9y ago
I think the time commitment is the biggest issue. Let's assume the costs are the same to me (perhaps due to employer subsidy): When I take a MOOC, I'm pledging at least 5 hours a week. That covers 2 hours of videos, 2 hours of homework and 1 hour of additional research / review. If I pledge 5 hours /week to a traditional class, I don't get nearly as much learning. I have to commute to the class, which is at least 30 minutes each way. Now my 5 hours buys me 2 hours of class, 2 hours of commuting and 1 hour of homework. I've lost an hour of homework and an hour of research. I know which one is better for me.
austenallred 9y ago
There have been quite a few studies that have done this in a scientific manner. Internally we have taken sets of students that apply to our free “mini code bootcamp” and sent half to an archived version published each night and send the other half a live link that becomes an archive each night. So we didn’t control scientifically, jsut an AB test, but the shift has been breathtaking. We can also take students that are struggling learning from our MOOCs and they finish the live course almost every time. I know the latter is unscientific, but it becomes obvious over time.
- kbutler 9y ago
  This Harvard study & Atlantic article by the study authors give the same 5% completion rate you cited, but give a completely opposite interpretation: basically, that the MOOC broadened access, allowing more people to get what they wanted out of the course. https://www.theatlantic.com/education/archive/2014/01/the-tricky-task-of-figuring-out-what-makes-a-mooc-successful/283274/ https://www.theatlantic.com/education/archive/2014/01/the-tr... https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 "In these course[sic], “dropping out” is not a breach of expectations but the natural result of an open, free, and asynchronous registration process, where students get just as much as they wish out of a course and registering for a course does not imply a commitment to completing it." ... Also the discussion of the "Colbert Bump", where increased publicity to less-motivated (just curious) students doubled completion but tripled registration (thus reducing completion rate). Also the point about whether to allow or disallow registration after the course begins (late registrants are ineligible for certification, so they are automatically "failed"). "HarvardX could boost its certification rate by closing those courses to new registrants or restrict courses to those most likely to complete them. Instead, it keeps courses open to maximize the number of students who are learning something new." I absolutely agree that (at least some) users can be helped by live instruction or supplemented online instruction, but I disagree that commonly cited, low completion rates measure students with the "same level of motivation".
- gliese1337 9y ago
  As long which ones you send to each link is determined entirely randomly, you have indeed done a "scientific" (i.e., statistically valid) control.

dsr_ 9y ago

Major counterproof: The Open University. Sounds like MOOCs need to learn from educational institutions that actually have a track record in distance learning.

tonyedgecombe 9y ago
The big difference is you have to pay for the OU, there is a commitment up front that filters out a lot of people. Saying MOOCS are a failure because only 2% complete the course is like saying the OU is a failure because only 2% of their web site visitors sign up.

randomdata 9y ago

> 90% will complete a live, guided course. 5% will complete a MOOC of the same quality. What does complete mean here? Complete within a given timeframe? Doesn't that miss the entire gain that MOOCs bring to the table? Which is that education becomes a lifelong endeavour instead of something you try and fit into a few years and, for most, never return to again?

moosey 9y ago

What is the purpose of a MOOC? Is it to make training available, or to provide academic success? If it is the former, then it is definitely a success. If it is the latter, then I think that not only percentage rates of success are important, but also the nominal completion rate itself. A MOOC supplies the opportunity for someone getting a degree in communications, business, or some other not-entirely-technical degree, that might be wonder what a computer science or high level mathematics course is like to go and find out. Does their failure in such a course mean that the class itself is a failure, or can the fact that they had access and were able to put a toe in the water mark success? I would suggest the latter.

austenallred 9y ago
Right. They're only a failure inasmuch as the initial promise was "these will basically replace college but for free!" which is now clearly not true.

VLM 9y ago

Ah OK interesting I think I get what you are saying in that given an authority figure's specification, the student in a class will be far more likely to succeed at reaching that arbitrary externally imposed spec than a MOOC subscriber. With the side issue that 100% of everyone in my 20 person automata theory class back in ninety-something is a much smaller number than 2% of 24000 people signing up for Ullman's automata theory MOOC. The fundamental problem is authority along with excessive broadness. If I want to learn Y and the MOOC teaches X Y and Z, I will fail the MOOC. There are also Skinner box conditioning issues where my assignment for something like using bootstrap framework, will be scratching my own itch of project or doing some project at work, such that again I'll fail the MOOC because I'll refuse the assignment; I already have a more ambitious self assigned assignment and am not doing both my assignment and someone elses much less interesting assignment. Whereas if I sit in class I'll have peer pressure and financial pressure to do the class assignment regardless of work or hobby interests. Sort of a traditional in-person or online class is a goal in and of itself; MOOCs are a means to an end for work or hobby for most.

ThrustVectoring 9y ago

How well are you controlling for the sheepskin effect? Students are highly motivated to do what it takes to get a degree from a university, and much less motivated to actually learn the topics nominally offered. I suspect that a MOOC offered with the same incentives and structure as a live class - ie, as part of a degree-granting program with mandatory tests at set dates - would have significantly higher completion rates.