9 ms·

There's already two comments here about being "harsh" or "ungracious" towards Khan Academy which is ridiculous. The usual HN article that contains criticisms i

There's already two comments here about being "harsh" or "ungracious" towards Khan Academy which is ridiculous. The usual HN article that contains criticisms is usually limited to that. Some rant that took 10 minutes to write and contains nothing constructive. Bret Victor put an insane amount of time into this (I can only assume) and is truly advancing mindsets about programming tools and learning. We should all be thankful that he put this much effort into a "criticism" piece.

AdamFernandez 14y ago
I agree, and would add that Brett Victor is pointing out what he perceives to be valid flaws with the method Khan Academny uses to teach programming (live coding environments, javascript, and processing languages, which other teaching products happen to use as well). He is not criticizing Khan Academy as a whole directly, as the quality or lack thereof of Khan Academy itself is never brought into question or even discussed.
- adambratt 14y ago
  Totally. People are too quick to jump on the "he's ranting" band wagon.
- russelluresti 14y ago
  > Brett Victor is pointing out what he perceives to be valid flaws Where are the numbers? Where's the testing? Where's the proof that his proposed changes are more effective in teaching a new programmer? Validity comes through numbers, proof, and evidence.
  - kamaal 14y ago
    You could say say something similar about UI design of things like iPhone. Some times you have to show people what they want. And then when they see, they would know how awesome things are. Not all projects are started after research and data. That way many projects would never start.
    - russelluresti 14y ago
      This doesn't make sense, because the UI design of the iPhone WAS tested. They did a ton of research into their solutions. They did exactly the type of testing I'm advocating.
      - c141charlie 14y ago
        I often hear people talk up how the iPhone was never tested in usability labs with non-apple employees. Then I hear that the iPhone was heavily tested. What actually happened? Does anyone know definitively whether or not it was tested by non-Apple employees before launch?
  - ht_th 14y ago
    That's a positivist perspective on research and knowledge. While your perspective on validity might be fitting for quantitative research, for an exploratory expose and kinds of qualitative research some look for other measures of validity and quality. There is an enormous amount of research (papers) on this topic, just look for "qualitative research validity". As it is more a matter of perspective, there often seem some incommensurability creeping into these discussions. In this case, I believe that a professional/experienced educator who has tried teaching introductory programming can, given their own experiences, relate, understand, and build on the concepts in the article to improve their own teaching and understanding of teaching/learning introductory programming. This is sometimes called communicative generalization (see Smaling: http://www.ualberta.ca/~iiqm/backissues/2_1/html/smaling.html http://www.ualberta.ca/~iiqm/backissues/2_1/html/smaling.htm... ). For example, to me, as a computer science teacher in high school, I recognize a lot of the issues discussed in the article. Reading the article I am able to link these issues and solutions to my experiences in class and decide to what extent it is credible and trustworthy. Furthermore, and more importantly, I can more easily apply results this kind of research in my daily practice than I can results of (rigorous) quantitative research as that almost always is taken out of context and ' clinically' due to fixing variables that aren't fixed in real life situations (i.e. the classroom, the students (holistic), environment).
    - russelluresti 14y ago
      I'm well aware of qualitative research, but he doesn't even have that. And, while I agree that you can translate and interpret the major themes, all you have done at that point is recognize that you have a similar problem and have formed a hypothesis that needs to be tested. You cannot assume that your method of fixing the identified problem will actually work until it is tested.
      - ht_th 14y ago
        True. On the other hand, it allows the professional/expert to have a good idea if it might work. And, if so, if it is worth testing out or not. In my opinion the value of this kind of research and/or informed opinion pieces is local in scope and should be treated as such. Don't base your educational policy on this kind of research, for that you'd better look for quantitative and, preferably, longitudinal studies.
  - ArekDymalski 14y ago
    You can find proof in numerous publications from related fields like cognitive psychology and pedagogy. But don't expect the author to gather them for you. He already did amazing job.
    - russelluresti 14y ago
      I disagree with the idea that proof of those concepts translates perfectly across all situations. You have to test something before you know if it works or not.
  - russelluresti 14y ago
    It's amazing how many people have a problem with the idea of testing a solution. I'm simply saying that assumptions need to be challenged and assertions need data to back them up. I'm not saying any of this ideas are inherently wrong, I'm saying they're unproven. If you down-vote simply because I believe a person should offer proof when they make a claim, you're falling victim to your own biases.
    - wamatt 14y ago
      >"I'm simply saying that assumptions need to be challenged and assertions need data to back them up. " I hope you have a study to backup that assertion.. :p (Hint: not all knowledge is empirical)
    - laughinghan 14y ago
      You are not being downvoted because "people have a problem with the idea of testing a solution". You are being downvoted because your tone suggests you categorically dismissed a groundbreaking project because its introductory article wasn't a scholarly work bristling with citations. "Numbers, proof, evidence" tend to be marks of an excellent article. But an incredible amount of work clearly went into this article, and it presents powerful ideas in a novel and original light; it is highly valuable and fascinating in its own right, even if it is weak in certain areas that academic papers tend to be strong in. Yes, citations would help tease apart questionable assumptions from reliable ones. Yes, thorough testing would help lend more credence many of the ideas. Yes, the article is imperfect and could be better in a multitude more ways. Still, the article is highly valuable and fascinating. Have you heard of the term "strawman argument"? That's what you're doing when you say "It's amazing how many people [are completely unreasonable]". Most people are quite reasonable. If a large community is downvoting you, you should strongly consider the possibility that you, in fact, are the one being unreasonable.
      - russelluresti 14y ago
        First, this isn't Bret's introductory article on this topic. As far as I know, his first talk about this came in January of this year (watch the Inventing on Principle video on Vimeo). Granted, 7-8 months isn't a super long time, but he has never built/tested his ideas, yet asserts them as if they were proven facts. Also, I know why I'm being down-voted. Bret is an inspiring person. His Vimeo talk, specifically, is very inspiring. He acts on principle, he believes strongly in his ideals, and there are many people who agree with him. So, the people who like Bret are people who align themselves with his ideals. So, when I challenge his vision by asking for data, people who read it feel like I'm challenging them, personally - like I'm challenging their beliefs (because, well, I am - I'm asking them to prove their beliefs). People respond to that kind of challenge in a negative way. It'd be like if you tried to argue the ideas of Steve Jobs with an Apple fan - they're just not going to listen to you, no matter how solid your arguments are. I, myself, agree strongly with Bret's principles. He's right that developers need better tools and we need a better way to teach programming to people. However, I try to divorce myself from those emotions and still look for facts and evidence. If someone claims A is better than B, I will look for proof, even if my "gut instinct" tells me that they're right. I very well believe Bret could be right, but that doesn't mean we should all just assume he is without evidence. My point here is that asking someone to give evidence for their arguments is completely acceptable. But, when you challenge the ideas of someone feel emotionally invested in (because their beliefs and ideals are being challenged), you receive negative feedback. Thus, the down-votes. It's actually a very interesting study in how people can be loyal to someone even without evidence of that person being right.
everlost 14y ago
I agree. There is a lot of constructive criticism in the article, and I'm sure Khan Academy would love this kind of feedback to help improve the platform. Hell, even I learned a couple of syntax usages just watching videos of the learning interface!
- yaxu 14y ago
  He could have delivered the "this is something to build on" message without the "almost worthless" value judgement.
marcioaguiar 14y ago
It's important to point out that was Bret Victor that created this concept from which Kahn Academy took inspiration from. https://vimeo.com/36579366 https://vimeo.com/36579366 http://ejohn.org/blog/introducing-khan-cs/ http://ejohn.org/blog/introducing-khan-cs/
- yaxu 14y ago
  As Bret Victor says in his essay, live coding is not a new concept and he did not create it. The current trend in live coding seems to have spawned from a discussion between two computer musicians (Fabrice Mogini and Julian Rohrhuber) in 2001, with masses of related prior art going back to Lisp machines, Self, Smalltalk, corewar etc.
  - seanmcdirmid 14y ago
    Responsive live programming (not just code reload), which is closer to what Bret Victor is doing here, goes back to various visual languages of the 80s and 90s.
    - yaxu 14y ago
      Like the ones I mentioned.
      - seanmcdirmid 14y ago
        You didn't mention any VPLs. I would throw in a few...ThingLab, AgentSheets, LabView, Prograph, Fabrik, and so on. These were the original live languages.
        yaxu 14y ago
        Self is certainly one. http://selflanguage.org/documentation/published/programming-as-experience.html http://selflanguage.org/documentation/published/programming-... But why do you think liveness requires visual programming? I see them as pretty much orthogonal. Separately, when I see "visual programming language" I see "unusual programming language". When language features become normal (like 2D syntax in Python/haskell), then we stop calling them visual. A great deal of visual features are just alternative ways of structuring text, ways of constraining syntax, or ways of constructing high dimensional (and therefore non-visual) graphs. To a large extent, all programming languages are visual.
joelhooks 14y ago
Yes. His approach presents a proper critique with a path forward. I love what KA is doing, but it is going to be iterative (I presume), and I'd hope they are listening to well thought out ideas like this.
yaxu 14y ago
He's also being naive about live coding as a whole: http://toplap.org/?p=212 http://toplap.org/?p=212
mattmcknight 14y ago
I just found the analogy of a microwave with blank buttons turned me off as being an unnecessary rhetorical straw man. I know what fill means, and I know what ellipse means, and I know what numbers are. Discovering what those words mean in a different context is not analogous to blank buttons.
- chipsy 14y ago
  Okay, but what is ellipse(50,25,32,5)? "ellipse" and numbers are only hints, just as blank buttons and dials are only hints. Programmers rely a lot on IDE support now to see the parameters...which is exactly what Bret is talking about!
  - pacala 14y ago
    The comparison is still shallow. It is regrettable that most programming languages don't support named parameters, but it's fairly trivial to imagine a language and api: DrawEllipse(x=50, y=25, width=32, height=5) No real-time rendering and tracing of values required (I hope!). Furthermore, a decent IDE will bring in the method doc blurb, so the programer has a richer context to make an informed decision. I appreciate the work Brett put into the article and I'm half way pre-convinced myself. At the end I was somewhat disappointed because most of the stuff he talks about is already present in a decent IDE, perhaps in a less polished form, but present nonetheless. The deeper put-off is that the article seems to advocate not thinking, instead performing random walks using shiny tools. If someone can't think about their code, no amount of visualizations is going to help. That being said, the potential of enhancing one's thinking abilities by properly chosen visualizations is tantalizing, but I don't think the article makes a good case for it and I haven't yet seen a better approach than old school pen and paper.
    - jasomill 14y ago
      It'll be interesting to read his subsequent comments on graphics state, as extending this example to draw arbitrary ellipses without coordinate changes could make for a simple visual demonstration of function composition, assuming the goal is not "Learn PostScript in floor(π^e - 1) Days."
russelluresti 14y ago
Here's my issue with the "advancement" that Bret is doing. It's all conjecture. He has no product. He has no users. He has theories. Theories that sound good to you and me, but, ultimately, theories that are untested and unproven. There's no evidence that any of these changes would actually help anyone learn to program any better. Who is to say that a person new to coding wouldn't be overwhelmed by the amount of information Bret's system is asking them to process? There's a pretty strong case for cognitive overload when you have lines of code, different code hints that appear not just over every line, but every function and number, a time bar with many different symbols on it, etc. But, again, no benefits or faults can be found with Bret's proposed approach because it's untested and unproven. Conjecture, hypotheses, and assumptions. This is where Khan Academy exceeds. They have users. They have real people really trying to learn how to code. This means they can test their assumptions and modify their approach. An imperfect solution delivered today is better than a perfect solution delivered next week.
- chj 14y ago
  I am a user of his ideas. I read through his article without taking a break, which rarely happen these days.
- readymade 14y ago
  Khan Academy and Bret Victor are apples and oranges -- they're not even playing the same game. In order to make any major strides, we have to reconcile the need for PRODUCTS, NOW! with the conceptual, exploratory approach taken here. Real world innovation often makes very small tweaks on proven formulae, whereas bold new ideas can have a long incubation period with seemingly little payoff at first. It's roughly the difference between business and research. While it's not possible to "test" these bare assertions in the sense you imply, they can certainly be argued for and against. And he makes a rather compelling case for it, certainly a more nuanced one than much of the criticism here on HN. In any event, he's begun sharing some of his code (I believe) and products like Light Table are emerging that appear to draw from very similar inspiration, if not his work directly. It would appear these ideas are gaining some traction.
  - 14y ago
chj 14y ago
If I were a developer from KA, I would be very grateful for this kind of crticism, no matter how harsh it is. Bret basically did a lot of free research work for them. Some of the ideas are hard to implement, but there are quite a few things that can easily be added to their products. Inovative ideas are rare and fragile. We should be thankful someone sharing them.

AdamFernandez 14y ago

I agree, and would add that Brett Victor is pointing out what he perceives to be valid flaws with the method Khan Academny uses to teach programming (live coding environments, javascript, and processing languages, which other teaching products happen to use as well). He is not criticizing Khan Academy as a whole directly, as the quality or lack thereof of Khan Academy itself is never brought into question or even discussed.

adambratt 14y ago
Totally. People are too quick to jump on the "he's ranting" band wagon.
russelluresti 14y ago
> Brett Victor is pointing out what he perceives to be valid flaws Where are the numbers? Where's the testing? Where's the proof that his proposed changes are more effective in teaching a new programmer? Validity comes through numbers, proof, and evidence.
- kamaal 14y ago
  You could say say something similar about UI design of things like iPhone. Some times you have to show people what they want. And then when they see, they would know how awesome things are. Not all projects are started after research and data. That way many projects would never start.
  - russelluresti 14y ago
    This doesn't make sense, because the UI design of the iPhone WAS tested. They did a ton of research into their solutions. They did exactly the type of testing I'm advocating.
    - c141charlie 14y ago
      I often hear people talk up how the iPhone was never tested in usability labs with non-apple employees. Then I hear that the iPhone was heavily tested. What actually happened? Does anyone know definitively whether or not it was tested by non-Apple employees before launch?
- ht_th 14y ago
  That's a positivist perspective on research and knowledge. While your perspective on validity might be fitting for quantitative research, for an exploratory expose and kinds of qualitative research some look for other measures of validity and quality. There is an enormous amount of research (papers) on this topic, just look for "qualitative research validity". As it is more a matter of perspective, there often seem some incommensurability creeping into these discussions. In this case, I believe that a professional/experienced educator who has tried teaching introductory programming can, given their own experiences, relate, understand, and build on the concepts in the article to improve their own teaching and understanding of teaching/learning introductory programming. This is sometimes called communicative generalization (see Smaling: http://www.ualberta.ca/~iiqm/backissues/2_1/html/smaling.html http://www.ualberta.ca/~iiqm/backissues/2_1/html/smaling.htm... ). For example, to me, as a computer science teacher in high school, I recognize a lot of the issues discussed in the article. Reading the article I am able to link these issues and solutions to my experiences in class and decide to what extent it is credible and trustworthy. Furthermore, and more importantly, I can more easily apply results this kind of research in my daily practice than I can results of (rigorous) quantitative research as that almost always is taken out of context and ' clinically' due to fixing variables that aren't fixed in real life situations (i.e. the classroom, the students (holistic), environment).
  - russelluresti 14y ago
    I'm well aware of qualitative research, but he doesn't even have that. And, while I agree that you can translate and interpret the major themes, all you have done at that point is recognize that you have a similar problem and have formed a hypothesis that needs to be tested. You cannot assume that your method of fixing the identified problem will actually work until it is tested.
    - ht_th 14y ago
      True. On the other hand, it allows the professional/expert to have a good idea if it might work. And, if so, if it is worth testing out or not. In my opinion the value of this kind of research and/or informed opinion pieces is local in scope and should be treated as such. Don't base your educational policy on this kind of research, for that you'd better look for quantitative and, preferably, longitudinal studies.
- ArekDymalski 14y ago
  You can find proof in numerous publications from related fields like cognitive psychology and pedagogy. But don't expect the author to gather them for you. He already did amazing job.
  - russelluresti 14y ago
    I disagree with the idea that proof of those concepts translates perfectly across all situations. You have to test something before you know if it works or not.
- russelluresti 14y ago
  It's amazing how many people have a problem with the idea of testing a solution. I'm simply saying that assumptions need to be challenged and assertions need data to back them up. I'm not saying any of this ideas are inherently wrong, I'm saying they're unproven. If you down-vote simply because I believe a person should offer proof when they make a claim, you're falling victim to your own biases.
  - wamatt 14y ago
    >"I'm simply saying that assumptions need to be challenged and assertions need data to back them up. " I hope you have a study to backup that assertion.. :p (Hint: not all knowledge is empirical)
  - laughinghan 14y ago
    You are not being downvoted because "people have a problem with the idea of testing a solution". You are being downvoted because your tone suggests you categorically dismissed a groundbreaking project because its introductory article wasn't a scholarly work bristling with citations. "Numbers, proof, evidence" tend to be marks of an excellent article. But an incredible amount of work clearly went into this article, and it presents powerful ideas in a novel and original light; it is highly valuable and fascinating in its own right, even if it is weak in certain areas that academic papers tend to be strong in. Yes, citations would help tease apart questionable assumptions from reliable ones. Yes, thorough testing would help lend more credence many of the ideas. Yes, the article is imperfect and could be better in a multitude more ways. Still, the article is highly valuable and fascinating. Have you heard of the term "strawman argument"? That's what you're doing when you say "It's amazing how many people [are completely unreasonable]". Most people are quite reasonable. If a large community is downvoting you, you should strongly consider the possibility that you, in fact, are the one being unreasonable.
    - russelluresti 14y ago
      First, this isn't Bret's introductory article on this topic. As far as I know, his first talk about this came in January of this year (watch the Inventing on Principle video on Vimeo). Granted, 7-8 months isn't a super long time, but he has never built/tested his ideas, yet asserts them as if they were proven facts. Also, I know why I'm being down-voted. Bret is an inspiring person. His Vimeo talk, specifically, is very inspiring. He acts on principle, he believes strongly in his ideals, and there are many people who agree with him. So, the people who like Bret are people who align themselves with his ideals. So, when I challenge his vision by asking for data, people who read it feel like I'm challenging them, personally - like I'm challenging their beliefs (because, well, I am - I'm asking them to prove their beliefs). People respond to that kind of challenge in a negative way. It'd be like if you tried to argue the ideas of Steve Jobs with an Apple fan - they're just not going to listen to you, no matter how solid your arguments are. I, myself, agree strongly with Bret's principles. He's right that developers need better tools and we need a better way to teach programming to people. However, I try to divorce myself from those emotions and still look for facts and evidence. If someone claims A is better than B, I will look for proof, even if my "gut instinct" tells me that they're right. I very well believe Bret could be right, but that doesn't mean we should all just assume he is without evidence. My point here is that asking someone to give evidence for their arguments is completely acceptable. But, when you challenge the ideas of someone feel emotionally invested in (because their beliefs and ideals are being challenged), you receive negative feedback. Thus, the down-votes. It's actually a very interesting study in how people can be loyal to someone even without evidence of that person being right.

everlost 14y ago

I agree. There is a lot of constructive criticism in the article, and I'm sure Khan Academy would love this kind of feedback to help improve the platform. Hell, even I learned a couple of syntax usages just watching videos of the learning interface!

yaxu 14y ago
He could have delivered the "this is something to build on" message without the "almost worthless" value judgement.

marcioaguiar 14y ago

It's important to point out that was Bret Victor that created this concept from which Kahn Academy took inspiration from. https://vimeo.com/36579366 https://vimeo.com/36579366 http://ejohn.org/blog/introducing-khan-cs/ http://ejohn.org/blog/introducing-khan-cs/

yaxu 14y ago
As Bret Victor says in his essay, live coding is not a new concept and he did not create it. The current trend in live coding seems to have spawned from a discussion between two computer musicians (Fabrice Mogini and Julian Rohrhuber) in 2001, with masses of related prior art going back to Lisp machines, Self, Smalltalk, corewar etc.
- seanmcdirmid 14y ago
  Responsive live programming (not just code reload), which is closer to what Bret Victor is doing here, goes back to various visual languages of the 80s and 90s.
  - yaxu 14y ago
    Like the ones I mentioned.
    - seanmcdirmid 14y ago
      You didn't mention any VPLs. I would throw in a few...ThingLab, AgentSheets, LabView, Prograph, Fabrik, and so on. These were the original live languages.
      - yaxu 14y ago
        Self is certainly one. http://selflanguage.org/documentation/published/programming-as-experience.html http://selflanguage.org/documentation/published/programming-... But why do you think liveness requires visual programming? I see them as pretty much orthogonal. Separately, when I see "visual programming language" I see "unusual programming language". When language features become normal (like 2D syntax in Python/haskell), then we stop calling them visual. A great deal of visual features are just alternative ways of structuring text, ways of constraining syntax, or ways of constructing high dimensional (and therefore non-visual) graphs. To a large extent, all programming languages are visual.

joelhooks 14y ago

Yes. His approach presents a proper critique with a path forward. I love what KA is doing, but it is going to be iterative (I presume), and I'd hope they are listening to well thought out ideas like this.

yaxu 14y ago

He's also being naive about live coding as a whole: http://toplap.org/?p=212 http://toplap.org/?p=212

mattmcknight 14y ago

I just found the analogy of a microwave with blank buttons turned me off as being an unnecessary rhetorical straw man. I know what fill means, and I know what ellipse means, and I know what numbers are. Discovering what those words mean in a different context is not analogous to blank buttons.

chipsy 14y ago
Okay, but what is ellipse(50,25,32,5)? "ellipse" and numbers are only hints, just as blank buttons and dials are only hints. Programmers rely a lot on IDE support now to see the parameters...which is exactly what Bret is talking about!
- pacala 14y ago
  The comparison is still shallow. It is regrettable that most programming languages don't support named parameters, but it's fairly trivial to imagine a language and api: DrawEllipse(x=50, y=25, width=32, height=5) No real-time rendering and tracing of values required (I hope!). Furthermore, a decent IDE will bring in the method doc blurb, so the programer has a richer context to make an informed decision. I appreciate the work Brett put into the article and I'm half way pre-convinced myself. At the end I was somewhat disappointed because most of the stuff he talks about is already present in a decent IDE, perhaps in a less polished form, but present nonetheless. The deeper put-off is that the article seems to advocate not thinking, instead performing random walks using shiny tools. If someone can't think about their code, no amount of visualizations is going to help. That being said, the potential of enhancing one's thinking abilities by properly chosen visualizations is tantalizing, but I don't think the article makes a good case for it and I haven't yet seen a better approach than old school pen and paper.
  - jasomill 14y ago
    It'll be interesting to read his subsequent comments on graphics state, as extending this example to draw arbitrary ellipses without coordinate changes could make for a simple visual demonstration of function composition, assuming the goal is not "Learn PostScript in floor(π^e - 1) Days."

russelluresti 14y ago

Here's my issue with the "advancement" that Bret is doing. It's all conjecture. He has no product. He has no users. He has theories. Theories that sound good to you and me, but, ultimately, theories that are untested and unproven. There's no evidence that any of these changes would actually help anyone learn to program any better. Who is to say that a person new to coding wouldn't be overwhelmed by the amount of information Bret's system is asking them to process? There's a pretty strong case for cognitive overload when you have lines of code, different code hints that appear not just over every line, but every function and number, a time bar with many different symbols on it, etc. But, again, no benefits or faults can be found with Bret's proposed approach because it's untested and unproven. Conjecture, hypotheses, and assumptions. This is where Khan Academy exceeds. They have users. They have real people really trying to learn how to code. This means they can test their assumptions and modify their approach. An imperfect solution delivered today is better than a perfect solution delivered next week.

chj 14y ago
I am a user of his ideas. I read through his article without taking a break, which rarely happen these days.
readymade 14y ago
Khan Academy and Bret Victor are apples and oranges -- they're not even playing the same game. In order to make any major strides, we have to reconcile the need for PRODUCTS, NOW! with the conceptual, exploratory approach taken here. Real world innovation often makes very small tweaks on proven formulae, whereas bold new ideas can have a long incubation period with seemingly little payoff at first. It's roughly the difference between business and research. While it's not possible to "test" these bare assertions in the sense you imply, they can certainly be argued for and against. And he makes a rather compelling case for it, certainly a more nuanced one than much of the criticism here on HN. In any event, he's begun sharing some of his code (I believe) and products like Light Table are emerging that appear to draw from very similar inspiration, if not his work directly. It would appear these ideas are gaining some traction.
- 14y ago

chj 14y ago

If I were a developer from KA, I would be very grateful for this kind of crticism, no matter how harsh it is. Bret basically did a lot of free research work for them. Some of the ideas are hard to implement, but there are quite a few things that can easily be added to their products. Inovative ideas are rare and fragile. We should be thankful someone sharing them.