5 ms·

The Engineered Student: On B. F. Skinner’s Teaching Machine

46 points by benbreen 5y ago

richardatlarge 5y ago
As the article suggests, Skinner was a very rational person and often made the mistake of expecting the rest of the world to be just the same. He once said that if you come across something interesting, stop everything and study it. It's good advice. Earning a Ph.D. in this area, I was always frustrated that - as utterly brilliant as Skinner was (e.g. he wrote a dozen books, including best-sellers, in the middle of the night after a short rest) - he always overlooked an obvious fact: people saw themselves as special from other animals, and would not accept any theory or technology that treated them as something less. While 'operant principles' could explain everything from the emotional lives of people to the conditioning of neural networks, Skinner's approach was, once again, too rational to survive in the public sphere.
- avgcorrection 5y ago
  > As the article suggests, Skinner was a very rational person In an audio recording he dismissed Chomsky’s critique of his work. He said that he hadn’t read it. He then went on to agree with people who had had read and disagreed with the critique... but how could he agree with them if he hadn’t read the critique.
  - richardatlarge 5y ago
    Not sure. Skinner was at Harvard and Chomsky was at MIT. But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review. Skinner didn't believe Chomsky, or linguists generally, understood Verbal Behavior, which further encouraged him to ignore the review. At the end of the day, you can't have a real debate between two views unless both views are properly understood. Useful reference: https://www.researchgate.net/publication/223959896_Skinner_and_Chomsky_30_Years_Later_Or_The_Return_of_the_Repressed https://www.researchgate.net/publication/223959896_Skinner_a...
    - avgcorrection 5y ago
      > Not sure. […] But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review. But that’s not the point. Say you haven’t read X: 1. It is rational to say “I haven’t read X so I can’t comment on it” 2. It is less rational (or dishonest) to say “I haven’t read X but I agree with these other people who have read it and dislike it!”
- tsimionescu 5y ago
  But Skinner's ideas have been proven to be entirely limited to simplistic training. His most ambitious idea - that all complex behaviors are acquired through these simplistic reward mechanisms - is well known to be wrong now, a hopelessly limited understanding of the problem space. As far as I know, he never engaged with the obvious arguments against his work - particularly the poverty of the stimulus argument. I don't believe he was a particularly rational individual - he was simply convinced of his simplistic model, and didn't want to see all the ways it didn't succeed in predicting all animal behavior. It's not that humans are special, by the way. There are human behaviors that can be trained with behaviorist approaches. There are also animal behaviors that are obviously not trained in similar ways. For a good example of the latter, look at how long it takes quadruped animals to learn to walk - many do it within seconds or minutes after birth. Deer are leaping over obstacles hours after birth. That's not operant conditioning.
  - avgcorrection 5y ago
    “Rational” in this context seems to mean “prone to reductive and simple explanations”.[1] But I have never heard of a scientific endeavor where people found out that things were more simple than they first hypothesized. [1] And to be contrasted with those idiots who think that humans are “special” since they think that simple theories are not sufficient... or you know, maybe that organisms in general are complex machines or whatever you want to call it and that, yes, there are differences between a maggot, a bird and a human.
    - dharmaturtle 5y ago
      > "All models are wrong, but some are useful."... It is usually considered to be applicable to not only statistical models, *but to scientific models generally*. The aphorism recognizes that statistical or scientific models always fall short of the complexities of reality but can still be of use. Emphasis mine. Human behavior is not physics. Even in that domain though, Newtonian physics is still useful - we just happen to know its limits. https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong
    - 5y ago
travisd 5y ago
Audrey Waters’s book is on my shelf and next in my queue. If anyone is interested in education technology and the intersection there in (especially with a critical eye towards what technology can, and should, so in a classroom), please drop me a line! travis (at) pathbird.com. As far as I can see, education is full of really perverse incentives which are, if not created by, greatly exacerbated by the prevalence of technology. Gamification (in general) ended poorly. MOOCs never changed the world. And so much of edtech forgets that education is about making better people, not better test scores.
throwaway05112 5y ago
modern machine learning (and reinforcement learning in particular) is a grandchild of B.F. Skinner work
kragen 5y ago
Brian Dear's The Friendly Orange Glow covers some of this history, as well as the fascinating PLATO system that grew out of it at the University of Illinois. The flat-screen plasma display developed for PLATO was Alan Kay's inspiration for the vision of the Dynabook and Smalltalk, which is where we got the modern GUI. (I'd say "and OOP", but the OOP part of Smalltalk came from Burroughs, SKETCHPAD, SIMULA, and a self-describing tape library system from about 01960; I don't think plasma displays had much to do with it.) I learned the basics of linear algebra in high school largely from a "programmed instruction" book inspired by Skinner's work. The Little Lisper family of books is the best example I've seen of this in recent decades. It's a shame that Watters is committing so many malapropisms in this article ("operating conditioning", plus incorrect uses of "ascribing" and "negative reinforcement"). You'd think she could hire an editor.
- cyberbanjo 5y ago
  Do you remember the name of the Linear Algebra text?
  - kragen 5y ago
    No, I'm sorry.
- leobg 5y ago
  Oh, wow. I love these old books. There is another one using this “Direct Instruction” technique in aviation. It’s called “Stick and Rudder”. I wish modern books were written like that. Do you know anymore?
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
tgb 5y ago
I just finished reading Don't Shoot the Dog by Karen Pryor, which is apparently a classic popular account of animal training based off Skinners ideas. It's pretty interesting but a little too self congratulatory. The short summary is: - punishment usually comes too late and also makes the trainee dislike the trainer and so makes future training harder - rewards should come immediately at success - since most rewards can't be given immediately, develop a cue (a "clicker" is popular) which you give at the immediate moment that informs them they're about to get the reward. This bridges the time span between act and reward, and the cue itself becomes valuable to the trainee later - variable rewards often work better - trainees in a rut can sometimes be jolted out of the rut by giving them an undeserved reward jackpot - teaching a trainee to not do something is harder but one strategy is to teach them to do it on command. Then simply don't never give them the command. Most behaviors die out once trainned to command except when commanded - or train them to do something that is incompatible with the undesired behavior - bizarrely, stopping doing an undesirable task can be reward enough to train them to do the undesirable task. I'm not super clear on how you do this in practice - giving cues and rewards too late is a common error. It's counterintuitive because you often must interrupt the behavior you are training. Rewarding your dog for sitting will cause the dog to stop sitting, but you must do this. Waiting until the dog stops sitting is too late since then you are rewarding the stopping of sitting, not sitting itself. - all of this applies to humans (if you don't let them know what you're doing) and also to yourself, supposedly I'm trying to train my misbehaving cat. We're making some progress but it's not as fast and magical as the book makes it sound (my inexperienced training skills likely contributing). I'm also learning Mandarin and a lot of what I've learned is through apps like Anki. Feedback is instant. But it doesn't really feel that rewarding to get a card right, so I'm not sure how successful it is in a Skinnerian sense. Teaching machines are here and work but I think they're still suboptimal.
- david_allison 5y ago
  > I'm also learning Mandarin and a lot of what I've learned is through apps like Anki. Feedback is instant. But it doesn't really feel that rewarding to get a card right, so I'm not sure how successful it is in a Skinnerian sense. Teaching machines are here and work but I think they're still suboptimal. A lot of users use addons to provide positive feedback in Anki. You may want to give them a try https://ankiweb.net/shared/info/1722658993 https://ankiweb.net/shared/info/1722658993 - Puppy Reinforcement https://ankiweb.net/shared/info/579111794 https://ankiweb.net/shared/info/579111794 - Anki Killstreaks https://ankiweb.net/shared/info/41708974 https://ankiweb.net/shared/info/41708974 - Leaderboard https://www.youtube.com/watch?v=UkveLkAgXiM https://www.youtube.com/watch?v=UkveLkAgXiM - Anki Add-on Guide: Gamify Your Reviews
jvvw 5y ago
I worked in a large academic educational technology department at a university for 13 years (as a developer) - and spent a lot of that time immersed in that world of research etc. in the area. The general vibe I would say was quite anti-behaviourist and certainly with more constructivist leanings. I certainly got the feeling that in those academic circles, anybody who admitted to being a behaviourist would be looked down upon. There are large differences between school education and higher education though. Both my children's school used some software called TT Rockstars for learning times tables which is very much a gamification of that and it did seem to work and motivate the children fairly effectively. A lot of schools obviously experimented with various online systems in the lockdowns here in the UK (and I also did as a parent!) and it was interesting seeing which engaged my children's attention and which didn't.
- geomark 5y ago
  My kid's school also used TT Rockstars in school before the pandemic. Seemed to work well for kids that age for giving them fast recall of memorized times tables. Beyond that, I didn't see any other effective gamified learning tools. Seems like it would be very much subject and age dependent.
cycomanic 5y ago
One thing that I find still both fascinating and shocking at the same time is how little we know about learning. If you look at the literature on learning, there are lots of schools, theories... almost all with only very little empirical work to verify and falsify. Take something relatively simple like learning to read. It's relatively straight forward to test, it's usually learned in relatively tightly controlled environments (schools) and yet we har no idea what is the best way to teach kids how to read. It would be comparitavely easy to conduct large scale experiments: make several groups of schools use different approaches to teach reading over several years, conduct standardised tests and compare. The populations could easily be large enough to average out other variables and after some time we would have a reasonable good idea which approach works and which doesn't. Instead we have a bunch of deductively derived theories paired with small scale qualitative studies ("it worked for me") , with dubious statistical significance. It is quite frustrating really.
geomark 5y ago
What happened to the comment about a learning platform called Primer? Was that removed for being too spammy?
richardatlarge 5y ago
Old Skinner would have a laugh today, debating immediacy as a reward on HN, where points are so prized ...

46 points by benbreen 5y ago

As the article suggests, Skinner was a very rational person and often made the mistake of expecting the rest of the world to be just the same. He once said that if you come across something interesting, stop everything and study it. It's good advice. Earning a Ph.D. in this area, I was always frustrated that - as utterly brilliant as Skinner was (e.g. he wrote a dozen books, including best-sellers, in the middle of the night after a short rest) - he always overlooked an obvious fact: people saw themselves as special from other animals, and would not accept any theory or technology that treated them as something less. While 'operant principles' could explain everything from the emotional lives of people to the conditioning of neural networks, Skinner's approach was, once again, too rational to survive in the public sphere.

avgcorrection 5y ago
> As the article suggests, Skinner was a very rational person In an audio recording he dismissed Chomsky’s critique of his work. He said that he hadn’t read it. He then went on to agree with people who had had read and disagreed with the critique... but how could he agree with them if he hadn’t read the critique.
- richardatlarge 5y ago
  Not sure. Skinner was at Harvard and Chomsky was at MIT. But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review. Skinner didn't believe Chomsky, or linguists generally, understood Verbal Behavior, which further encouraged him to ignore the review. At the end of the day, you can't have a real debate between two views unless both views are properly understood. Useful reference: https://www.researchgate.net/publication/223959896_Skinner_and_Chomsky_30_Years_Later_Or_The_Return_of_the_Repressed https://www.researchgate.net/publication/223959896_Skinner_a...
  - avgcorrection 5y ago
    > Not sure. […] But Skinner was much more established in his respective field at that time, which is why Skinner made the mistake of ignoring the review. But that’s not the point. Say you haven’t read X: 1. It is rational to say “I haven’t read X so I can’t comment on it” 2. It is less rational (or dishonest) to say “I haven’t read X but I agree with these other people who have read it and dislike it!”
tsimionescu 5y ago
But Skinner's ideas have been proven to be entirely limited to simplistic training. His most ambitious idea - that all complex behaviors are acquired through these simplistic reward mechanisms - is well known to be wrong now, a hopelessly limited understanding of the problem space. As far as I know, he never engaged with the obvious arguments against his work - particularly the poverty of the stimulus argument. I don't believe he was a particularly rational individual - he was simply convinced of his simplistic model, and didn't want to see all the ways it didn't succeed in predicting all animal behavior. It's not that humans are special, by the way. There are human behaviors that can be trained with behaviorist approaches. There are also animal behaviors that are obviously not trained in similar ways. For a good example of the latter, look at how long it takes quadruped animals to learn to walk - many do it within seconds or minutes after birth. Deer are leaping over obstacles hours after birth. That's not operant conditioning.
- avgcorrection 5y ago
  “Rational” in this context seems to mean “prone to reductive and simple explanations”.[1] But I have never heard of a scientific endeavor where people found out that things were more simple than they first hypothesized. [1] And to be contrasted with those idiots who think that humans are “special” since they think that simple theories are not sufficient... or you know, maybe that organisms in general are complex machines or whatever you want to call it and that, yes, there are differences between a maggot, a bird and a human.
  - dharmaturtle 5y ago
    > "All models are wrong, but some are useful."... It is usually considered to be applicable to not only statistical models, *but to scientific models generally*. The aphorism recognizes that statistical or scientific models always fall short of the complexities of reality but can still be of use. Emphasis mine. Human behavior is not physics. Even in that domain though, Newtonian physics is still useful - we just happen to know its limits. https://en.wikipedia.org/wiki/All_models_are_wrong https://en.wikipedia.org/wiki/All_models_are_wrong
  - 5y ago

travisd 5y ago

Audrey Waters’s book is on my shelf and next in my queue. If anyone is interested in education technology and the intersection there in (especially with a critical eye towards what technology can, and should, so in a classroom), please drop me a line! travis (at) pathbird.com. As far as I can see, education is full of really perverse incentives which are, if not created by, greatly exacerbated by the prevalence of technology. Gamification (in general) ended poorly. MOOCs never changed the world. And so much of edtech forgets that education is about making better people, not better test scores.

throwaway05112 5y ago

modern machine learning (and reinforcement learning in particular) is a grandchild of B.F. Skinner work

kragen 5y ago

Brian Dear's The Friendly Orange Glow covers some of this history, as well as the fascinating PLATO system that grew out of it at the University of Illinois. The flat-screen plasma display developed for PLATO was Alan Kay's inspiration for the vision of the Dynabook and Smalltalk, which is where we got the modern GUI. (I'd say "and OOP", but the OOP part of Smalltalk came from Burroughs, SKETCHPAD, SIMULA, and a self-describing tape library system from about 01960; I don't think plasma displays had much to do with it.) I learned the basics of linear algebra in high school largely from a "programmed instruction" book inspired by Skinner's work. The Little Lisper family of books is the best example I've seen of this in recent decades. It's a shame that Watters is committing so many malapropisms in this article ("operating conditioning", plus incorrect uses of "ascribing" and "negative reinforcement"). You'd think she could hire an editor.

cyberbanjo 5y ago
Do you remember the name of the Linear Algebra text?
- kragen 5y ago
  No, I'm sorry.
leobg 5y ago
Oh, wow. I love these old books. There is another one using this “Direct Instruction” technique in aviation. It’s called “Stick and Rudder”. I wish modern books were written like that. Do you know anymore?

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

tgb 5y ago

I just finished reading Don't Shoot the Dog by Karen Pryor, which is apparently a classic popular account of animal training based off Skinners ideas. It's pretty interesting but a little too self congratulatory. The short summary is: - punishment usually comes too late and also makes the trainee dislike the trainer and so makes future training harder - rewards should come immediately at success - since most rewards can't be given immediately, develop a cue (a "clicker" is popular) which you give at the immediate moment that informs them they're about to get the reward. This bridges the time span between act and reward, and the cue itself becomes valuable to the trainee later - variable rewards often work better - trainees in a rut can sometimes be jolted out of the rut by giving them an undeserved reward jackpot - teaching a trainee to not do something is harder but one strategy is to teach them to do it on command. Then simply don't never give them the command. Most behaviors die out once trainned to command except when commanded - or train them to do something that is incompatible with the undesired behavior - bizarrely, stopping doing an undesirable task can be reward enough to train them to do the undesirable task. I'm not super clear on how you do this in practice - giving cues and rewards too late is a common error. It's counterintuitive because you often must interrupt the behavior you are training. Rewarding your dog for sitting will cause the dog to stop sitting, but you must do this. Waiting until the dog stops sitting is too late since then you are rewarding the stopping of sitting, not sitting itself. - all of this applies to humans (if you don't let them know what you're doing) and also to yourself, supposedly I'm trying to train my misbehaving cat. We're making some progress but it's not as fast and magical as the book makes it sound (my inexperienced training skills likely contributing). I'm also learning Mandarin and a lot of what I've learned is through apps like Anki. Feedback is instant. But it doesn't really feel that rewarding to get a card right, so I'm not sure how successful it is in a Skinnerian sense. Teaching machines are here and work but I think they're still suboptimal.

david_allison 5y ago
> I'm also learning Mandarin and a lot of what I've learned is through apps like Anki. Feedback is instant. But it doesn't really feel that rewarding to get a card right, so I'm not sure how successful it is in a Skinnerian sense. Teaching machines are here and work but I think they're still suboptimal. A lot of users use addons to provide positive feedback in Anki. You may want to give them a try https://ankiweb.net/shared/info/1722658993 https://ankiweb.net/shared/info/1722658993 - Puppy Reinforcement https://ankiweb.net/shared/info/579111794 https://ankiweb.net/shared/info/579111794 - Anki Killstreaks https://ankiweb.net/shared/info/41708974 https://ankiweb.net/shared/info/41708974 - Leaderboard https://www.youtube.com/watch?v=UkveLkAgXiM https://www.youtube.com/watch?v=UkveLkAgXiM - Anki Add-on Guide: Gamify Your Reviews

jvvw 5y ago

I worked in a large academic educational technology department at a university for 13 years (as a developer) - and spent a lot of that time immersed in that world of research etc. in the area. The general vibe I would say was quite anti-behaviourist and certainly with more constructivist leanings. I certainly got the feeling that in those academic circles, anybody who admitted to being a behaviourist would be looked down upon. There are large differences between school education and higher education though. Both my children's school used some software called TT Rockstars for learning times tables which is very much a gamification of that and it did seem to work and motivate the children fairly effectively. A lot of schools obviously experimented with various online systems in the lockdowns here in the UK (and I also did as a parent!) and it was interesting seeing which engaged my children's attention and which didn't.

geomark 5y ago
My kid's school also used TT Rockstars in school before the pandemic. Seemed to work well for kids that age for giving them fast recall of memorized times tables. Beyond that, I didn't see any other effective gamified learning tools. Seems like it would be very much subject and age dependent.

cycomanic 5y ago

One thing that I find still both fascinating and shocking at the same time is how little we know about learning. If you look at the literature on learning, there are lots of schools, theories... almost all with only very little empirical work to verify and falsify. Take something relatively simple like learning to read. It's relatively straight forward to test, it's usually learned in relatively tightly controlled environments (schools) and yet we har no idea what is the best way to teach kids how to read. It would be comparitavely easy to conduct large scale experiments: make several groups of schools use different approaches to teach reading over several years, conduct standardised tests and compare. The populations could easily be large enough to average out other variables and after some time we would have a reasonable good idea which approach works and which doesn't. Instead we have a bunch of deductively derived theories paired with small scale qualitative studies ("it worked for me") , with dubious statistical significance. It is quite frustrating really.

geomark 5y ago

What happened to the comment about a learning platform called Primer? Was that removed for being too spammy?

richardatlarge 5y ago

Old Skinner would have a laugh today, debating immediacy as a reward on HN, where points are so prized ...