5 ms·

Real time monitoring during a live task is certainly a use case I've never thought of. A very good point. I remember a piece on programming education research

• falcor84 4y ago
  > education outcomes improved considerably if you constantly bombard students with multiple choice questions I would love to read more about that, if you happen to have a link to share.
  • urthor 4y ago
    https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006023 https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
    • falcor84 4y ago
      Much obliged. I see the relevant part seems to be the following, in the "Tip 4" section: """ The key to making demonstrations more effective is to make learners predict the outcome of the demonstration before performing it. Crucially, their prediction should be in some way recorded or public, e.g., by a show of hands, by holding up cue cards marked with A, B, C, or D, or by talking to their neighbour. We speculate that the sting of being publicly wrong leads learners to pay more attention and to reflect on what they are learning; regardless of whether this hypothesis is true, instructors should be careful not to punish or criticise students who predicted wrongly but rather to use those incorrect predictions as a spur to further exploration and explanation. """
      • blagie 4y ago
        That part has been replicated (see e.g. vicarious learning, from Derek Muller as well as Miki Chi, studies on Khan Academy in-video questions, etc.). However, it's not the most relevant part. The most relevant part is simple use of active learning. The classic paper here is Hake (Interactive-Engagement vs. Traditional Methods: A Six-Thousand-Student Survey of Mechanics Test Data for Introductory Physics Courses). However, it's been replicated discipline-by-discipline (lots of papers in PNAS). Best summary is ICAP by Miki Chi. Miki didn't invent it, and ICAP isn't the most rigorous, but she's pretty good at these summaries. Having students do anything beyond listening leads to huge improvements in student learning, basically no matter what it is. Some things lead to bigger improvements than others, of course. Even clicking a "next" button occasionally shows a (very modest) gain. For good interactive engagement, doubling of learning gains is a good lower-bound over passive listening.
        • urthor 4y ago
          Note that it's Active Recall according to other education research. Not only active participation, but actively being forced to remember the knowledge, in a different format to the original, is the key. Teaching another student is definitely an approach to this. But a gentle touch multiple choice question also works.
          • blagie 4y ago
            No, that's not really fair. There's a broad range of techniques which have significant learning gains beyond passive learning. Active recall is one of them. We have no clear sense of how they compare to each other or how they work in combination. There is ample evidence for active recall, but there is equally ample evidence for a dozen other techniques much like it. We don't have great studies comparing them side-by-side. The best hypothesis for how they compare is ICAP, referenced. It's still a hypothesis. You can look at the evidence behind it. Much of this is context-dependent too. Active recall is great for factoids. If you're trying to learn a date in history, memorize a word in a foreign language, or a formula in math, your best bet is a spaced repetition system (how you time active recall makes a big difference too; spaced repetition > simple active recall > passive recall). If you're trying to understand a concept like force (in physics), what an integral means (in math), make sense of the tradition away from serfdom, or compare ethical systems between two cultures, active recall is almost irrelevant. At that point, the best methods are probably constructive and interactive learning.
            • urthor 4y ago
              I agree. In my mind, active recall is an excellent technique that's appropriate for the first lecture covering a topic. Which is why I like it as a prompt in a lecture, not so much later on. Constructive learning in my mind comes later. Re: the balance between constructive learning and other types. I feel that we often accelerate the students too quickly to constructive learning, whilst they can still benefit from rote learning some worked examples. The balance between rote learning and "copying" worked examples, and constructive learning from first principles, is a very difficult one. I truly think that, certainly for programming education, all my professional life I've gotten huge mileage out of genuinely copying in example projects and tutorials into a text editor. I think we underestimate just how much students benefit from rote learning. I think programming education would benefit far more from 19 year old rote learning from retired 65 year olds. I find the college education system is fairly broken, in that it deploys researchers who are excellent lecturers, but far too few 65 year olds. And there's an ocean of extremely willing to teach 65 year olds out there.