16 ms·

Stanford study shows success of different programming styles in CS class

156 points by kornish 12y ago

mellavora 12y ago
Fantastic! Good steps towards "artificial wisdom". Define wisdom-- skill in a craft (here: knowing how to think). Using ML to identify "wise" programming mindsets. Then (presumably) developing educational programs which teach these. “Educational data mining should not be used to reinforce the existing ineffective forms of assessment, but to reimagine it completely,” Blikstein said. “Pre- and post-assessment is a black-box. We do not know much about what is happening in between. “Ultimately, this work may help us better understand some forms of human cognition because you can see what people are doing and how they are thinking in real time.”
partition 12y ago
What was interesting is that the higher-performing group (Alpha) was stuck less often that the lower-performing group (Gamma). Is this the next programming motto after "Don't Repeat Yourself"? Don't Get Stuck! Now how does one avoid getting stuck while programming? I still get stuck a lot, but these really helped: - Test what you write as quickly as you can. - Hold it in your head! Or you won't have a clue what all your code, together, is doing. - To get started, ask yourself, what is the simplest thing that could possibly work?
- BenderV 12y ago
  > - To get started, ask yourself, what is the simplest thing that could possibly work? I'm not sure I would go for what appear to be the simplest now ; I try to find the implementation which will have the simplest design to use/modify/delete in the future.
- danieltillett 12y ago
  Overcoming getting stuck is the difference between a highly productive dev and the average dev. This is especially the case if the problem being worked on is hard. I am a very average dev in regards coding skills, but due to my background I have learned strategies to “unstick” myself after ending up in the middle of a massive mud pit. This has allowed me to solve problems better devs have not been able to accomplish.
- VMG 12y ago
  > What was interesting is that the higher-performing group (Alpha) was stuck less often that the lower-performing group (Gamma). Which may just be correlated with high performance, not necessarily the cause. If you make "don't get stuck" a motto, you can easily imagine people skipping over difficult problems and not thinking about a good solution.
  - tarpherder 12y ago
    I share your concern that this will result in a lot of sloppy solution to lots of little problems which in turn results in one big mess of sloppy-but-it-works solutions which can be a big problem (but doesn't have to be). However, even when working out a good solution I think it's important not to get stuck. Make sure you are making some sort of progress; it's okay that it takes time but it is not okay if someone is just moving in circles instead of inching closer to the solution. Basically if something takes "too long" to get working, try a different approach or come back to it at a later time to try again. I quite like the "Don't get stuck" motto if we look at it that way.
  - bjwbell 12y ago
    "don't get stuck" reminds me of the feynman method of problem solving: 1. Write down the problem. 2. Think very hard. 3. Write down the answer. Rich Hickey's Hammock Driven Development, https://www.youtube.com/watch?v=f84n5oFoZBc https://www.youtube.com/watch?v=f84n5oFoZBc, is IMO very good for actually not getting stuck in a bad place problem solving.
    - partition 12y ago
      Wow, this was a big wake-up call. It's ironic because I am in research and I always did a lot of theory and problem definition up front for whatever I did, but I am now in this "reckless rapid prototyping" phase where if I get some idea I have to try it out RIGHT NOW, stepping back be damned. Ironically it results in getting stuck more than I would have wanted! I guess this is the fourth tip? - Step away from the computer
  - mironathetin 12y ago
- diminoten 12y ago
  Each of your three bullet points has a related developer concept: * Test early, test often test automatically [0] * Remember the big picture [0] * The minimum viable product [1] [0] - https://pragprog.com/the-pragmatic-programmer/extracts/tips [1] - http://practicetrumpstheory.com/minimum-viable-product/
- pjc50 12y ago
  I'd offer two bits of marketable wisdom from this. - think back. Students who change style do better. This implies that those students are reflecting on their own style and trying out techniques (performing experiments) to find out what works best for them. - think ahead. Use lookahead to avoid backtracking. Some amount of top-down planning will make your life easier and avoid traps, even if your natural style is bottom-up.
- datenwolf 12y ago
  > Test what you write as quickly as you can. Until you run into that one problem where you can't test between the small steps, because you need the whole thing to be up (to some degree) and working for any test to work. Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails. Another example would be refactoring a complex library into something more manageable. If you keep working in small, testable steps you can move only along the gradient, bordered by "can execute" and "doesn't execute". So you'll be able to reach only a local extremum. Now if you're in the fog and don't know where to go, that's fine. And for most development this is exactly how it happens. But sometimes you can see that summit on the other side of that rift and you know you have to take a leap to get over there. I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it. > - Hold it in your head! Or you won't have a clue what all your code, together, is doing. Or, sometimes it's important to get it out of your head, take a week or two off and look at it again with a fresh mind and from a different angle. Often problems seem only hard because you're approaching them from that one angle and you're so stuck with wanting to get it done, that you don't see the better way. Instead you should write code in a way that it's easy to get back into it. > - To get started, ask yourself, what is the simplest thing that could possibly work? And then wonder: What would it take to make this simple thing break. Make things as simple as necessary but not simpler.
  - Dewie 12y ago
    > Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails. How do you test kernel? Run a virtualised kernel? I think at least DragonflyBSD supports that.
- VLM 12y ago
  My interpretation of that part of the article was "refactor early and reimplement often" That grabs in the concept of switching between planning and experimenting modes with the end result of not getting stuck. I may be biased in having learned this stuff a long time ago so when I get stuck its from getting hopelessly backed into a corner at which point the best solution is pretty much start over differently, and first semester programmers might get trapped a little more easily, like they don't even know common ancient anti-patterns so they don't even know where to start looking when debugging. Reading between the lines it almost sounded like the A level programmers wrote their tests first (which is usually easy) then wrote code to pass the tests (usually not too awful) then they were done, whereas the lower grade level programmers wrote the code first, then tried to debug (and debugging is always harder than test writing or coding)
- random_coder 12y ago
  It's also likely that these guys had already programmed before coming to the class. We all tend to get stuck when programming something for the first time. Second and third times are much smoother experiences.
  - evincarofautumn 12y ago
    You’ve reminded me of a quote from Chuck Moore: > Before you can write your own subroutine, you have to know how. This means, to be practical, that you have written it before; which makes it difficult to get started. But give it a try. After writing the same subroutine a dozen times on as many computers and languages, you'll be pretty good at it. Or, as Fred Brooks put it, “plan one to throw away”.
- vanderZwan 12y ago
  > Hold it in your head! Or you won't have a clue what all your code, together, is doing. Do you mean to say: if you can't hold it all in your head, you're overcomplicating things? Because if so, I agree. I just finished giving an introduction to programming course, using Processing. I tried teaching my students to write structured code from the start, splitting functionality into smaller methods as much as sensible. That way they never have to juggle too much functionality in their head at once. On the first day I had them call built-in methods. Then I introduced the concept of variables and types, then I introduced defining your own methods, and only after that did I move on to booleans, if/else, for-loops and arrays. It seems to have worked well for getting them to structure their code from the start, using methods more or less the same way you would use chapters, headers, sub-headers and paragraphs to structure a paper. I know this isn't the best way to program, obviously, but this analogy is (relatively) easy for them to understand and loads better than the ball of muddy spaghetti you often see with people who just start out. It's also probably fairly easy to make the jump to other more advanced forms of code organisation. I also completely agree with your other two points by the way, I kept repeating similar statements throughout the course: - By splitting functionality into small methods, it's easy to test if something does what it is supposed to do (and I could easily correct them: "Hey, this method drawBall() is also doing hit detection. That's not what the method is supposed to do, restructure the code!"). - "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?"
  - chiph 12y ago
    When I see students posting on /r/programming, what they usually seem to have a problem with is problem decomposition. i.e.: What are the steps needed to solve this part of the problem? Or algorithm design, critical thinking, or whatever you want to call it. How do I get from A to B, basically. Remembering back to my first year, it seems that some people "get this" and some do not. Which is nothing against them -- everyone is different.
  - partition 12y ago
    > - "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?" I feel like after you really know a system you almost start thinking only in terms of its components. "Hmm I want to do X...somehow I just automatically thought to modify function Y and call it in module Z."
- humanarity 12y ago
  Yeah I really reckon that "what is the simplest thing that could possibly work" is the vital question. If you stray from that, getting stuck wastes what could otherwise be productive effort. I do have to wonder tho -- why is programming so complicated? Silicon and neurons so fundamentally different that we're always going to be fighting ourselves -- or one day we're going to have interfaces make programming way more natural?
nerdy 12y ago
Begs the question: "Is there a 'right' way to think about programming problems?" My knee-jerk reaction would be "No! Everyone learns and thinks differently"-- but being able to put programming styles into buckets and correlate final grades from it suggests otherwise, doesn't it?
- brudgers 12y ago
  I think the emphasis is more about the way people think about solving programming problems than the way they think about the problem itself. An analogy is that two people may differ in their mental models at the level of syntactic sugar rather than Turing completeness.
- atmosx 12y ago
  It might suggest something, but it's just a hint. In order to take that approach as conclusive much more comprehensive studies in to highly diversified environments should take place. All that leaving aside that it's a social science we're talking so... It's not exactly science as in scientific method. It's just models that (under X circumstances) might or might not apply... Go figure ;-)
inglor 12y ago
From the actual article (and not the news article): "The results show that students’ change in programming patterns is only weakly predictive of course performance" Not as "juicy" as the website - but then again, it's the actual results :)
ynniv 12y ago
It sounds like they discovered that some people already knew how to program.
- FinnDS 12y ago
  It seems to me that there are way too many variables that aren't really measurable for this to be completely accurate -- such as this example, some people probably had programmed before.
- kifler 12y ago
  Having recently (last year) enrolled in a Computer Programming program, I can say this is likely the case. The class has a number of people who are simply enrolled to get a piece of paper that states they know how to code and then there are those that are breaking new ground in learning. I have the logic down quite well but my biggest issue is the syntax - it would appear that Stanford thinks I'm doomed to fail.
  - spion 12y ago
    One of the most touted advantages of Scheme (and other Lisps) as a teaching language is precisely the (lack of) syntax which allows the student to focus on logic.
    - marssaxman 12y ago
      One might as well claim that C++ has no syntax, as its source code is nothing but a linear series of undifferentiated octets. Scheme and other lisps have plenty of syntax; it's just expressed at a different level of abstraction than the one most languages use.
      - spion 12y ago
        C++ is an abomination - it should not only be avoided when teaching, but we should make all the effort to avoid it whenever possible. Learning programming using C++ as a language is like learning to walk on a grass-covered minefield - almost every feature hides something that can blow your leg off. Anyway, the fact remains that you need much fewer syntax lessons to get started with Scheme compared to anything else.
kriro 12y ago
I think data mining editor/IDE use is a pretty useful idea overall and the "paradigm shift" of constant monitoring which allows immediate teacher feedback is much needed (not just for programming). The other interesting approach I read about a while back was using tracing quiz data to identify "blind spots". A typical example was identifying students that had problems understanding loop constructs (they would always assume one iteration for example). I talked to one of the authors of this paper and he had some interesting ideas fro CS education in general: http://dl.acm.org/citation.cfm?id=2526978 http://dl.acm.org/citation.cfm?id=2526978 My gut instinct and thinking back to my university days make me think that a blind spot for recursion may very well be a thing. I'd be pretty interested in identifying students that struggle with loops and/or recursion and investigating that further. For example...is there a correlation? If not what happens if you give a loop-struggler group only recursive tasks and a recursion-struggler group only loop tasks? Etc.
skywhopper 12y ago
It's not necessarily surprising that they might be able to detect students who will likely perform poorly in the course overall by monitoring their behavior early on. But the real challenge is not in determining which students are struggling, but rather in when and how you should intervene to improve these students' chances of success.
markbnj 12y ago
>> Alpha students, explained co-author Piech, moved efficiently from one point to another as they wrote code, creating a streamlined program. Beta and gamma students, on the other hand, wrote themselves into so-called “sink states” in which their programs slammed into dead ends from which they had to back out before they could resume progress. Very interesting, and intuitively right to me. The difference is likely in the ability to create and maintain a consistent mental model of the program's structure and behavior during the implementation.
AustinBGibbons 12y ago
Hey! I worked on the preliminary efforts of this project in 2012, and was also a grader for the course. (I'm "Gibbons, A." in the citation list) Many people are making great points - there's a wealth of uncontrolled variables (ie some students already know how to program), but it was a fascinating dataset and I could even see the paradigms play out over the quarter during discussions with my students, both experienced programmers and students getting their first exposure to programming. I would offer as a larger takeaway that work like this has the potential to vastly increase our understanding of learning and development with the rise of MOOCs, ipads-in-every-classroom, and of course IDEs that save every incremental compilation :-) Oh and as another mention - working with Marcelo and Paulo was awesome, both are super passionate, intelligent dudes.
the_watcher 12y ago
>> the authors did not find any correlation between the amount of “tinkering” — which is often thought to signify lack of programming expertise — and course performance. However, the authors found that students who changed their programming style — going from being a “planner” to a “tinkerer,” or vice versa — were the ones who performed best, suggesting that behavior change (rather than learning one single behavior) was determinant for course performance. Not being flippant: The conclusion here seems to be evidence of an ability to learn something new, regardless of what, is strongly indicative of performance.
GrinningFool 12y ago
> "The discovery of these “sink states,” and how students > got into them, offers opportunities beyond predicting > grades: They open the door for developing systems to > encourage students to go down more fruitful paths before > they become lost in the programming weeds." In the real world, my most valuable learning experiences have been those dead ends. Because I don't repeat those mistakes - better to make them and learn from them than to be steered away from them before they happen.
- taeric 12y ago
  Relatedly, I have seen plenty of very smart people that were able to bowl over these "sink states" completely unaware of just how unmaintainable their ultimate solution was.
convivialdingo 12y ago
Completely anecdotal, but I vividly remember my own experience learning to program. I learned in a vacuum, having nobody around with any software experience. I was given a PC and then saved up for an Amiga 500 (and later a $500 Lattice C compiler). My only link to software was through the bookstore and magazines. Being a nerd, my main motivation at the time (of course) was to make a D&D game. I remember my first code was in Basic (no subroutines!) so I eventually learned to create subroutines based on line numbers (100-900 was main, 1000 was map, 1100 was character A, etc) I managed to make about 50% of a single game before I ran out of memory. So I then set out to shrink (optimize) what I had made. After reading an article on dungeon map creation, I figured out that large arrays could replace the hand-coded drawing I had written. Check. Ok, so now I start doing everything in arrays and pretty soon my code is smaller but "fixing stuff" (maintaining code) was stupidly hard. Oh okay, so I learned then some that data and code should be together rather than global (encapsulation). I then needed to create random maps. After struggling with that, I read a paper on fractals, and implemented my first algorithm. You can probably guess by now where I'm going with all this... I had crudely recreated many things by knocking up against hardware and brain limitations and figuring out how to work around them. I progressed into Pascal, C, C++, assembly and eventually created my own assembler and graphical operating system at 16 years old. It was not a thing of beauty, but it did work. (Time was free, and compilers weren't cheap.) I think that most people learn programming by failing, hitting traps, finding limits, crashing, burning and trying again. The only real identifier of outcomes, in the long term, is the willingness to continue trying and learning from mistakes. So from my point of view, the outcomes of the paper are arbitrary in the long term. I would definitely had been a "Gamma" programmer. I hit every problem, and got terribly stuck. It took years for me to progress, learn good practices and become a proficient developer.
20edward 12y ago
Good Article, Thanks.

156 points by kornish 12y ago

Fantastic! Good steps towards "artificial wisdom". Define wisdom-- skill in a craft (here: knowing how to think). Using ML to identify "wise" programming mindsets. Then (presumably) developing educational programs which teach these. “Educational data mining should not be used to reinforce the existing ineffective forms of assessment, but to reimagine it completely,” Blikstein said. “Pre- and post-assessment is a black-box. We do not know much about what is happening in between. “Ultimately, this work may help us better understand some forms of human cognition because you can see what people are doing and how they are thinking in real time.”

partition 12y ago

What was interesting is that the higher-performing group (Alpha) was stuck less often that the lower-performing group (Gamma). Is this the next programming motto after "Don't Repeat Yourself"? Don't Get Stuck! Now how does one avoid getting stuck while programming? I still get stuck a lot, but these really helped: - Test what you write as quickly as you can. - Hold it in your head! Or you won't have a clue what all your code, together, is doing. - To get started, ask yourself, what is the simplest thing that could possibly work?

BenderV 12y ago
> - To get started, ask yourself, what is the simplest thing that could possibly work? I'm not sure I would go for what appear to be the simplest now ; I try to find the implementation which will have the simplest design to use/modify/delete in the future.
danieltillett 12y ago
Overcoming getting stuck is the difference between a highly productive dev and the average dev. This is especially the case if the problem being worked on is hard. I am a very average dev in regards coding skills, but due to my background I have learned strategies to “unstick” myself after ending up in the middle of a massive mud pit. This has allowed me to solve problems better devs have not been able to accomplish.
VMG 12y ago
> What was interesting is that the higher-performing group (Alpha) was stuck less often that the lower-performing group (Gamma). Which may just be correlated with high performance, not necessarily the cause. If you make "don't get stuck" a motto, you can easily imagine people skipping over difficult problems and not thinking about a good solution.
- tarpherder 12y ago
  I share your concern that this will result in a lot of sloppy solution to lots of little problems which in turn results in one big mess of sloppy-but-it-works solutions which can be a big problem (but doesn't have to be). However, even when working out a good solution I think it's important not to get stuck. Make sure you are making some sort of progress; it's okay that it takes time but it is not okay if someone is just moving in circles instead of inching closer to the solution. Basically if something takes "too long" to get working, try a different approach or come back to it at a later time to try again. I quite like the "Don't get stuck" motto if we look at it that way.
- bjwbell 12y ago
  "don't get stuck" reminds me of the feynman method of problem solving: 1. Write down the problem. 2. Think very hard. 3. Write down the answer. Rich Hickey's Hammock Driven Development, https://www.youtube.com/watch?v=f84n5oFoZBc https://www.youtube.com/watch?v=f84n5oFoZBc, is IMO very good for actually not getting stuck in a bad place problem solving.
  - partition 12y ago
    Wow, this was a big wake-up call. It's ironic because I am in research and I always did a lot of theory and problem definition up front for whatever I did, but I am now in this "reckless rapid prototyping" phase where if I get some idea I have to try it out RIGHT NOW, stepping back be damned. Ironically it results in getting stuck more than I would have wanted! I guess this is the fourth tip? - Step away from the computer
- mironathetin 12y ago
diminoten 12y ago
Each of your three bullet points has a related developer concept: * Test early, test often test automatically [0] * Remember the big picture [0] * The minimum viable product [1] [0] - https://pragprog.com/the-pragmatic-programmer/extracts/tips [1] - http://practicetrumpstheory.com/minimum-viable-product/
pjc50 12y ago
I'd offer two bits of marketable wisdom from this. - think back. Students who change style do better. This implies that those students are reflecting on their own style and trying out techniques (performing experiments) to find out what works best for them. - think ahead. Use lookahead to avoid backtracking. Some amount of top-down planning will make your life easier and avoid traps, even if your natural style is bottom-up.
datenwolf 12y ago
> Test what you write as quickly as you can. Until you run into that one problem where you can't test between the small steps, because you need the whole thing to be up (to some degree) and working for any test to work. Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails. Another example would be refactoring a complex library into something more manageable. If you keep working in small, testable steps you can move only along the gradient, bordered by "can execute" and "doesn't execute". So you'll be able to reach only a local extremum. Now if you're in the fog and don't know where to go, that's fine. And for most development this is exactly how it happens. But sometimes you can see that summit on the other side of that rift and you know you have to take a leap to get over there. I spent the past 3 weeks doing exactly that, refactoring a code base. I knew exactly where I wanted to go, but eventually it meant working for about a week on code without being able to compile, not even think of testing it, because everything was moving around and getting reorganized. However now I'm enjoying the fruits of that week; a much cleaner codebase, easier to work with and I even managed to eliminate some Voodoo code nobody knew why it was there, except that it made things work and things broke if you touched it. > - Hold it in your head! Or you won't have a clue what all your code, together, is doing. Or, sometimes it's important to get it out of your head, take a week or two off and look at it again with a fresh mind and from a different angle. Often problems seem only hard because you're approaching them from that one angle and you're so stuck with wanting to get it done, that you don't see the better way. Instead you should write code in a way that it's easy to get back into it. > - To get started, ask yourself, what is the simplest thing that could possibly work? And then wonder: What would it take to make this simple thing break. Make things as simple as necessary but not simpler.
- Dewie 12y ago
  > Operating system kernels would be an example of that: The best test for a *nix OS kernel is, if it can run a shell. You need all the essential syscalls to do something sensible and if any of the required parts doesn't work the whole thing fails. How do you test kernel? Run a virtualised kernel? I think at least DragonflyBSD supports that.
VLM 12y ago
My interpretation of that part of the article was "refactor early and reimplement often" That grabs in the concept of switching between planning and experimenting modes with the end result of not getting stuck. I may be biased in having learned this stuff a long time ago so when I get stuck its from getting hopelessly backed into a corner at which point the best solution is pretty much start over differently, and first semester programmers might get trapped a little more easily, like they don't even know common ancient anti-patterns so they don't even know where to start looking when debugging. Reading between the lines it almost sounded like the A level programmers wrote their tests first (which is usually easy) then wrote code to pass the tests (usually not too awful) then they were done, whereas the lower grade level programmers wrote the code first, then tried to debug (and debugging is always harder than test writing or coding)
random_coder 12y ago
It's also likely that these guys had already programmed before coming to the class. We all tend to get stuck when programming something for the first time. Second and third times are much smoother experiences.
- evincarofautumn 12y ago
  You’ve reminded me of a quote from Chuck Moore: > Before you can write your own subroutine, you have to know how. This means, to be practical, that you have written it before; which makes it difficult to get started. But give it a try. After writing the same subroutine a dozen times on as many computers and languages, you'll be pretty good at it. Or, as Fred Brooks put it, “plan one to throw away”.
vanderZwan 12y ago
> Hold it in your head! Or you won't have a clue what all your code, together, is doing. Do you mean to say: if you can't hold it all in your head, you're overcomplicating things? Because if so, I agree. I just finished giving an introduction to programming course, using Processing. I tried teaching my students to write structured code from the start, splitting functionality into smaller methods as much as sensible. That way they never have to juggle too much functionality in their head at once. On the first day I had them call built-in methods. Then I introduced the concept of variables and types, then I introduced defining your own methods, and only after that did I move on to booleans, if/else, for-loops and arrays. It seems to have worked well for getting them to structure their code from the start, using methods more or less the same way you would use chapters, headers, sub-headers and paragraphs to structure a paper. I know this isn't the best way to program, obviously, but this analogy is (relatively) easy for them to understand and loads better than the ball of muddy spaghetti you often see with people who just start out. It's also probably fairly easy to make the jump to other more advanced forms of code organisation. I also completely agree with your other two points by the way, I kept repeating similar statements throughout the course: - By splitting functionality into small methods, it's easy to test if something does what it is supposed to do (and I could easily correct them: "Hey, this method drawBall() is also doing hit detection. That's not what the method is supposed to do, restructure the code!"). - "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?"
- chiph 12y ago
  When I see students posting on /r/programming, what they usually seem to have a problem with is problem decomposition. i.e.: What are the steps needed to solve this part of the problem? Or algorithm design, critical thinking, or whatever you want to call it. How do I get from A to B, basically. Remembering back to my first year, it seems that some people "get this" and some do not. Which is nothing against them -- everyone is different.
- partition 12y ago
  > - "The computer only does what it is literally instructed to do. What do you literally want to do? What are you literally instructing the computer right now?" I feel like after you really know a system you almost start thinking only in terms of its components. "Hmm I want to do X...somehow I just automatically thought to modify function Y and call it in module Z."
humanarity 12y ago
Yeah I really reckon that "what is the simplest thing that could possibly work" is the vital question. If you stray from that, getting stuck wastes what could otherwise be productive effort. I do have to wonder tho -- why is programming so complicated? Silicon and neurons so fundamentally different that we're always going to be fighting ourselves -- or one day we're going to have interfaces make programming way more natural?

nerdy 12y ago

Begs the question: "Is there a 'right' way to think about programming problems?" My knee-jerk reaction would be "No! Everyone learns and thinks differently"-- but being able to put programming styles into buckets and correlate final grades from it suggests otherwise, doesn't it?

brudgers 12y ago
I think the emphasis is more about the way people think about solving programming problems than the way they think about the problem itself. An analogy is that two people may differ in their mental models at the level of syntactic sugar rather than Turing completeness.
atmosx 12y ago
It might suggest something, but it's just a hint. In order to take that approach as conclusive much more comprehensive studies in to highly diversified environments should take place. All that leaving aside that it's a social science we're talking so... It's not exactly science as in scientific method. It's just models that (under X circumstances) might or might not apply... Go figure ;-)

inglor 12y ago

From the actual article (and not the news article): "The results show that students’ change in programming patterns is only weakly predictive of course performance" Not as "juicy" as the website - but then again, it's the actual results :)

ynniv 12y ago

It sounds like they discovered that some people already knew how to program.

FinnDS 12y ago
It seems to me that there are way too many variables that aren't really measurable for this to be completely accurate -- such as this example, some people probably had programmed before.
kifler 12y ago
Having recently (last year) enrolled in a Computer Programming program, I can say this is likely the case. The class has a number of people who are simply enrolled to get a piece of paper that states they know how to code and then there are those that are breaking new ground in learning. I have the logic down quite well but my biggest issue is the syntax - it would appear that Stanford thinks I'm doomed to fail.
- spion 12y ago
  One of the most touted advantages of Scheme (and other Lisps) as a teaching language is precisely the (lack of) syntax which allows the student to focus on logic.
  - marssaxman 12y ago
    One might as well claim that C++ has no syntax, as its source code is nothing but a linear series of undifferentiated octets. Scheme and other lisps have plenty of syntax; it's just expressed at a different level of abstraction than the one most languages use.
    - spion 12y ago
      C++ is an abomination - it should not only be avoided when teaching, but we should make all the effort to avoid it whenever possible. Learning programming using C++ as a language is like learning to walk on a grass-covered minefield - almost every feature hides something that can blow your leg off. Anyway, the fact remains that you need much fewer syntax lessons to get started with Scheme compared to anything else.

kriro 12y ago

I think data mining editor/IDE use is a pretty useful idea overall and the "paradigm shift" of constant monitoring which allows immediate teacher feedback is much needed (not just for programming). The other interesting approach I read about a while back was using tracing quiz data to identify "blind spots". A typical example was identifying students that had problems understanding loop constructs (they would always assume one iteration for example). I talked to one of the authors of this paper and he had some interesting ideas fro CS education in general: http://dl.acm.org/citation.cfm?id=2526978 http://dl.acm.org/citation.cfm?id=2526978 My gut instinct and thinking back to my university days make me think that a blind spot for recursion may very well be a thing. I'd be pretty interested in identifying students that struggle with loops and/or recursion and investigating that further. For example...is there a correlation? If not what happens if you give a loop-struggler group only recursive tasks and a recursion-struggler group only loop tasks? Etc.

skywhopper 12y ago

It's not necessarily surprising that they might be able to detect students who will likely perform poorly in the course overall by monitoring their behavior early on. But the real challenge is not in determining which students are struggling, but rather in when and how you should intervene to improve these students' chances of success.

markbnj 12y ago

>> Alpha students, explained co-author Piech, moved efficiently from one point to another as they wrote code, creating a streamlined program. Beta and gamma students, on the other hand, wrote themselves into so-called “sink states” in which their programs slammed into dead ends from which they had to back out before they could resume progress. Very interesting, and intuitively right to me. The difference is likely in the ability to create and maintain a consistent mental model of the program's structure and behavior during the implementation.

AustinBGibbons 12y ago

Hey! I worked on the preliminary efforts of this project in 2012, and was also a grader for the course. (I'm "Gibbons, A." in the citation list) Many people are making great points - there's a wealth of uncontrolled variables (ie some students already know how to program), but it was a fascinating dataset and I could even see the paradigms play out over the quarter during discussions with my students, both experienced programmers and students getting their first exposure to programming. I would offer as a larger takeaway that work like this has the potential to vastly increase our understanding of learning and development with the rise of MOOCs, ipads-in-every-classroom, and of course IDEs that save every incremental compilation :-) Oh and as another mention - working with Marcelo and Paulo was awesome, both are super passionate, intelligent dudes.

the_watcher 12y ago

>> the authors did not find any correlation between the amount of “tinkering” — which is often thought to signify lack of programming expertise — and course performance. However, the authors found that students who changed their programming style — going from being a “planner” to a “tinkerer,” or vice versa — were the ones who performed best, suggesting that behavior change (rather than learning one single behavior) was determinant for course performance. Not being flippant: The conclusion here seems to be evidence of an ability to learn something new, regardless of what, is strongly indicative of performance.

GrinningFool 12y ago

> "The discovery of these “sink states,” and how students > got into them, offers opportunities beyond predicting > grades: They open the door for developing systems to > encourage students to go down more fruitful paths before > they become lost in the programming weeds." In the real world, my most valuable learning experiences have been those dead ends. Because I don't repeat those mistakes - better to make them and learn from them than to be steered away from them before they happen.

taeric 12y ago
Relatedly, I have seen plenty of very smart people that were able to bowl over these "sink states" completely unaware of just how unmaintainable their ultimate solution was.

convivialdingo 12y ago

Completely anecdotal, but I vividly remember my own experience learning to program. I learned in a vacuum, having nobody around with any software experience. I was given a PC and then saved up for an Amiga 500 (and later a $500 Lattice C compiler). My only link to software was through the bookstore and magazines. Being a nerd, my main motivation at the time (of course) was to make a D&D game. I remember my first code was in Basic (no subroutines!) so I eventually learned to create subroutines based on line numbers (100-900 was main, 1000 was map, 1100 was character A, etc) I managed to make about 50% of a single game before I ran out of memory. So I then set out to shrink (optimize) what I had made. After reading an article on dungeon map creation, I figured out that large arrays could replace the hand-coded drawing I had written. Check. Ok, so now I start doing everything in arrays and pretty soon my code is smaller but "fixing stuff" (maintaining code) was stupidly hard. Oh okay, so I learned then some that data and code should be together rather than global (encapsulation). I then needed to create random maps. After struggling with that, I read a paper on fractals, and implemented my first algorithm. You can probably guess by now where I'm going with all this... I had crudely recreated many things by knocking up against hardware and brain limitations and figuring out how to work around them. I progressed into Pascal, C, C++, assembly and eventually created my own assembler and graphical operating system at 16 years old. It was not a thing of beauty, but it did work. (Time was free, and compilers weren't cheap.) I think that most people learn programming by failing, hitting traps, finding limits, crashing, burning and trying again. The only real identifier of outcomes, in the long term, is the willingness to continue trying and learning from mistakes. So from my point of view, the outcomes of the paper are arbitrary in the long term. I would definitely had been a "Gamma" programmer. I hit every problem, and got terribly stuck. It took years for me to progress, learn good practices and become a proficient developer.

20edward 12y ago

Good Article, Thanks.