7 ms·

Perhaps, but if there is one thing we know, it's that massive online courses do not work well as we have currently tried them. Students don't come back year to

by wespiser_2018 6y ago

Perhaps, but if there is one thing we know, it's that massive online courses do not work well as we have currently tried them. Students don't come back year to year, and engagement throughout the course is very low [1]. Using a video game might help solve the engagement problem, but I would be hard pressed to think benefit of using a video game (increased engagement in the learning process) would be worth the 10-100x development cost with considerably more risk. Where I think video games can excel is in more niche applications like Kerbal Space Program, not teaching something like European Literature. [1] https://joseruiperez.me/papers/journals/2019_Science_MOOCPivot_postprint.pdf https://joseruiperez.me/papers/journals/2019_Science_MOOCPiv...

KaoruAoiShiho 6y ago
Video games don't mean 3d whatever, there are very low cost browser games too that sufficiently gamify learning. Sites like coursera are actually kinda terrible in terms of gamification. The best examples are sites like lichess or duolingo that have great retention.
Animats 6y ago
"but I would be hard pressed to think benefit of using a video game (increased engagement in the learning process) would be worth the 10-100x development cost with considerably more risk." That's the problem with the MOOC industry. Those guys produced content at the production level of the average high school or college class, endless talking head video. They thought that all the advantages of scale accrued to them - they got to make cheap content and sell it in huge quantity. Didn't work.
tsumnia 6y ago
While I do agree, it is not without caveats. One issue is that "engagement" is not a good metric for measuring learning. One of the early studies made on the DragonBox math game system[1] showed less benefit than allowing re-practice opportunities with a traditional tutoring system with no reward system. This was despite students showing higher levels of engagement, enjoyment, and completing more exercises using DragonBox. Engagement and enjoyment are tricky metrics because they often report positive numbers; however, when compared to less engaging or less enjoyable methods they can perform worse for the end goal of learning. They can create motivation to learn the topic, which I support, but at some point that benefit has a ceiling effect. [1] https://www.researchgate.net/profile/Yanjin_Long/publication/268577301_Gamification_of_Joint_StudentSystem_Control_Over_Problem_Selection_in_a_Linear_Equation_Tutor/links/5470fb2b0cf24af340c3b8fd.pdf https://www.researchgate.net/profile/Yanjin_Long/publication...
- wespiser_2018 6y ago
  Indeed, and I think if you optimize for engagement, you are really failing to consider that once you target a metric, that metric is no longer useful. For the MOOC example, what I meant by engagement was, "Week over week, does the student return?" or Weekly Active Users as a percentage of total students at the start of the course. For a MOOCs, the number of students disengaging is so high, that there is obviously a problem, and this is what I meant to communicate. Thanks for the article!
  - tsumnia 6y ago
    MOOCs are an interesting area as well, as they share similar drop off to free mobile games - high initial recruitment and engagement, but immediate drop off. I love a lot of the components that MOOCs use and utilize them in my course design, but the high drop off is why I support traditional class-based environments. The cost of tuition and fear of failure ensure they stick with it. In another world, I think it's one of the reasons why many physical sports like martial arts have high drop offs as well, no risk from failure. This is one of the reasons I don't particularly care for motivation studies, as I believe it's only a temporary method to generate traction. Once learning becomes difficult, it requires a higher order of motivation I call "discipline" to maintain. But back to MOOCs, there is work on MOOC forum analysis[1]. In short, forum activity and the people you associate with in an educational forum can have a significant impact on your grade. Note this study was post-hoc analysis, but students that primarily discussed non-course relevant topics were more likely to fail the course. There is also the fact that MOOC students come from a variety of life experiences. Students of different countries have different cultures and as a result, will behave differently [2]. France, for example, only wanted to view the course in [2], not engage. I think the beauty of a MOOC is the accessibility and automated feedback feature that from their design. Beyond that they suffer from a lot of human conditions that can't really be controlled without removing or reducing those features. Plus, as I mention, there isn't a lot of cog psych research on "discipline" yet to identify the qualities that encourage its development besides Duckworth's "grit" (if I'm wrong I'd love to be pointed towards them). [1] http://ceur-ws.org/Vol-1446/GEDM_2015_Submission_2.pdf http://ceur-ws.org/Vol-1446/GEDM_2015_Submission_2.pdf [2] https://files.eric.ed.gov/fulltext/ED592695.pdf https://files.eric.ed.gov/fulltext/ED592695.pdf
hanselot 6y ago
Opportunity exists for developing the AI training program from Ender's Game. That would be solving the problem, permanently.

KaoruAoiShiho 6y ago

Video games don't mean 3d whatever, there are very low cost browser games too that sufficiently gamify learning. Sites like coursera are actually kinda terrible in terms of gamification. The best examples are sites like lichess or duolingo that have great retention.

Animats 6y ago

"but I would be hard pressed to think benefit of using a video game (increased engagement in the learning process) would be worth the 10-100x development cost with considerably more risk." That's the problem with the MOOC industry. Those guys produced content at the production level of the average high school or college class, endless talking head video. They thought that all the advantages of scale accrued to them - they got to make cheap content and sell it in huge quantity. Didn't work.

tsumnia 6y ago

While I do agree, it is not without caveats. One issue is that "engagement" is not a good metric for measuring learning. One of the early studies made on the DragonBox math game system[1] showed less benefit than allowing re-practice opportunities with a traditional tutoring system with no reward system. This was despite students showing higher levels of engagement, enjoyment, and completing more exercises using DragonBox. Engagement and enjoyment are tricky metrics because they often report positive numbers; however, when compared to less engaging or less enjoyable methods they can perform worse for the end goal of learning. They can create motivation to learn the topic, which I support, but at some point that benefit has a ceiling effect. [1] https://www.researchgate.net/profile/Yanjin_Long/publication/268577301_Gamification_of_Joint_StudentSystem_Control_Over_Problem_Selection_in_a_Linear_Equation_Tutor/links/5470fb2b0cf24af340c3b8fd.pdf https://www.researchgate.net/profile/Yanjin_Long/publication...

wespiser_2018 6y ago
Indeed, and I think if you optimize for engagement, you are really failing to consider that once you target a metric, that metric is no longer useful. For the MOOC example, what I meant by engagement was, "Week over week, does the student return?" or Weekly Active Users as a percentage of total students at the start of the course. For a MOOCs, the number of students disengaging is so high, that there is obviously a problem, and this is what I meant to communicate. Thanks for the article!
- tsumnia 6y ago
  MOOCs are an interesting area as well, as they share similar drop off to free mobile games - high initial recruitment and engagement, but immediate drop off. I love a lot of the components that MOOCs use and utilize them in my course design, but the high drop off is why I support traditional class-based environments. The cost of tuition and fear of failure ensure they stick with it. In another world, I think it's one of the reasons why many physical sports like martial arts have high drop offs as well, no risk from failure. This is one of the reasons I don't particularly care for motivation studies, as I believe it's only a temporary method to generate traction. Once learning becomes difficult, it requires a higher order of motivation I call "discipline" to maintain. But back to MOOCs, there is work on MOOC forum analysis[1]. In short, forum activity and the people you associate with in an educational forum can have a significant impact on your grade. Note this study was post-hoc analysis, but students that primarily discussed non-course relevant topics were more likely to fail the course. There is also the fact that MOOC students come from a variety of life experiences. Students of different countries have different cultures and as a result, will behave differently [2]. France, for example, only wanted to view the course in [2], not engage. I think the beauty of a MOOC is the accessibility and automated feedback feature that from their design. Beyond that they suffer from a lot of human conditions that can't really be controlled without removing or reducing those features. Plus, as I mention, there isn't a lot of cog psych research on "discipline" yet to identify the qualities that encourage its development besides Duckworth's "grit" (if I'm wrong I'd love to be pointed towards them). [1] http://ceur-ws.org/Vol-1446/GEDM_2015_Submission_2.pdf http://ceur-ws.org/Vol-1446/GEDM_2015_Submission_2.pdf [2] https://files.eric.ed.gov/fulltext/ED592695.pdf https://files.eric.ed.gov/fulltext/ED592695.pdf

hanselot 6y ago

Opportunity exists for developing the AI training program from Ender's Game. That would be solving the problem, permanently.