6 ms·

Modeling How Programmers Read Code

73 points by joeyespo 13y ago

silvio 13y ago
With more research along these lines, we may have a day when a programming job interview will consist on interacting with some codebase for 10 minutes, and the model will spit out our normalized scores on syntax comprehension, algorithmic thinking, library familiarity, etc.
- goldfeld 13y ago
  It does sound representative of the bulk of the daily work.
- keefe 13y ago
  has there been any work on this? it seems relatively straightforward to score a git branch in certain time scales EDIT should have RTFA before commenting, gaze tracking stuff is very cool
- seanmcdirmid 13y ago
  I wonder if when we get that far the computer is able to write the code for us anyways and programmers will be redundancies. Its just a matter of if, not when. Understanding human cognition well enough to evaluate it is a good indicator that the singularity is near.
  - gizmo686 13y ago
    Computers already write code for us, they just need to be told what we want them to write. The question is if we can design a programming environment where the computer can understand 'natural' human specifications and compile that, instead of needing to have a trained human compile the natural specifications into source code, which the computer can compile into a program.
    - seanmcdirmid 13y ago
      Sure we could. This is just like Siri, you tell it what you want, she asks for clarifications, a conversation you have with the computer until...bam, you get what you want. I would think we could do much of this already if we tried, but there are a lot of things on my list to do until I get to that.
      - encoderer 13y ago
        Your optimism on this seems unfounded in my opinion. When there are no domain restrictions, computers have not proven very capable of comprehension. Look even at the comparatively simple domain of handwriting recognition. More importantly, you entirely miss the problem that many times the human doesn't know exactly what he wants until he sees it. As a software engineer, I have no fear my job will be replaced by computers talking to product managers.
        seanmcdirmid 13y ago
        That's why the program should be expressed as a conversation! We don't know what we want so we should start vague, get feedback, provide clarifications, and so on. The conversation with the computer must be a two-way thing!
        lifeisstillgood 13y ago
        That's brilliant. Not in terms of some AI working out what a silver haired old grandmother meant but in terms of talking to clients - they could imagine talking to a computer and I am just providing that feedback. Daily or weekly, feedback nice and quick
        seanmcdirmid 13y ago
        The thing is, we probably don't need hard AI yet to do some of this. Yes, it must be a dialogue system, but we have those today. We already saw the movement in this direction with Hugo Liu's concept.net work, but for some reason no one has followed up yet. We are getting to the point with speech rec/understanding technology that someone is bound to try again soon.
  - kragen 13y ago
    That was the hype surrounding Fortran when it came out in 1959: it would eliminate the need for programmers, because you could just punch the formulas and it would translate them into a program automatically. And in a sense it did, and its successors did to an even greater extent. Today you have "nonprogrammers" who can build, in a day, in OpenOffice Calc, computations that would have taken weeks to get written by programmers in the 1950s (when they could have been done at all.) It's partly a matter of giving the objectives to the computer at progressively higher levels, and partly a matter of improving user interfaces so that you can tell what the thing is doing and what it's going to do.
  - eli_gottlieb 13y ago
    These predictions always fail to account for C++. That is, having computers write code depends on the code being in a form about which computers can formally reason. If you can't prove anything about your code because the language is that bad, good luck getting a computer to write code. /explain_the_joke.jpg
ansgri 13y ago
In light of recent mass surveillance uproars, the possibility of remote eye tracking technology is what really scares me. While the technological usage could be limited or masked, such attention detector and analyzer could give enourmous powers to those who have the behavioral models. Very interesting reserach though.
joeroot 13y ago
This highlights how "good" programmers are simply those who are most able to think like a computer. The article's language further reinforces this (perhaps intentionally), e.g. the reference to "compiling" functions. I find this contraint interesting - I don't know if computational thinking is a strength or a weakness. Increasingly, it seems that computational models occur naturally and therefore the ability to think in such a manner would have inter-disciplinary value. If we deem it a weakness, then programming becomes a UX problem rather than language one. The lack of change both within and across programming paradigms would suggest that many don't believe this to be a fundamental issue.
- jared314 13y ago
  This actually touches on Alan Kay's statements about programming with "what" instead of "how". It has made me seriously consider logic programming for the applications I write.
- MaulingMonkey 13y ago
  I'd place my skill bars higher: "thinking like a computer" seems fundamental: I don't think one can even be a mediocre programmer without it, and it's not enough to make you a good programmer. A good programmer also needs to be able to effectively communicate through code to both others and themselves, know how to design maintainable programs, how to preemptively avoid bugs by making their code harder to misuse, and too many other skills to list here -- and not all of those naturally flow from knowing how to think like a computer.
- eli_gottlieb 13y ago
  Computers don't think like computers. Computers reason formally. Thinking like a computer is just a matter of learning to reason formally and exactly instead of employing fuzzy mental abstractions. This is primarily hard because we're wired for fuzzy mental abstractions, since they make loads of tasks (natural language, making breakfast, manual labor) far easier compared to taking genuine formal approaches, and we have a lower layer of the brain that can be trained to follow exact procedures quite well anyway.
Chris_Newton 13y ago
Previous related discussion: https://news.ycombinator.com/item?id=4940952 https://news.ycombinator.com/item?id=4940952
oblique63 13y ago
I would love to see this kind of study done comparing different programming paradigms (i.e. functional vs. procedural vs. OO vs. logic). Some concrete cognitive data on how lisp-like languages may or may not actually make programmers more productive would be quite fascinating I think.
- Chris_Newton 13y ago
  I’m not aware of many eye-tracking studies specifically, but there has been a lot of research done on the general subject of how we read and understand code. Search for papers on “program comprehension”. To address (some of) your specific point, I suspect part of the appeal of functional programming languages is that they naturally promote describing the data flow of the code. In contrast, imperative languages inherently dwell on control flow, even though it’s often an unimportant implementation detail. It turns out that even when we’re reading imperative code, we’re trying to figure out the underlying data flow anyway, so why not cut out the middle step? I also suspect part of what holds back functional programming languages from wider acceptance is that when you’re modelling something where time/order matters, the control flow is not an unimportant implementation detail. You have to work harder to describe it in a functional language where you get a lot for free with an imperative programming language.
  - gizmo686 13y ago
    > You have to work harder to describe it in a functional language This isn't necessarily true. In Haskell you can use do blocks like: main = do action1 action2 ... and your actions will happen sequentially. This is just syntactic sugar behind "action1 >> action2". The real reason for the special syntax is when the actions also return a value you can do "x <- action1", and use the variable x. This is still just sugar, but the unsugared version gets messy pretty quickly. The big difficult in going from imperative to functional languages is thinking about data structures as immutable. You go from asking how do I change the value to asking giving this value, how do I construct a new value.
    - Chris_Newton 13y ago
buttproblem 13y ago
I see how this is interesting but I can't get over how the author says, "When contrasting this with Eric's video", but "Eric's video" is a different program than the novice programmer. Maybe this is me being too pedantic.
- NoodleIncident 13y ago
  Not just a different program, a better one, and easier to understand. It especially annoyed me when he praised Eric for going back to understand the function first, when that function isn't even present in the novice's program.
garysweaver 13y ago
> he "compiled" the between function in his head Really? How about "adequately understood the function and did not have to reread it". We are humans, not compilers. ;)

73 points by joeyespo 13y ago

With more research along these lines, we may have a day when a programming job interview will consist on interacting with some codebase for 10 minutes, and the model will spit out our normalized scores on syntax comprehension, algorithmic thinking, library familiarity, etc.

goldfeld 13y ago
It does sound representative of the bulk of the daily work.
keefe 13y ago
has there been any work on this? it seems relatively straightforward to score a git branch in certain time scales EDIT should have RTFA before commenting, gaze tracking stuff is very cool
seanmcdirmid 13y ago
I wonder if when we get that far the computer is able to write the code for us anyways and programmers will be redundancies. Its just a matter of if, not when. Understanding human cognition well enough to evaluate it is a good indicator that the singularity is near.
- gizmo686 13y ago
  Computers already write code for us, they just need to be told what we want them to write. The question is if we can design a programming environment where the computer can understand 'natural' human specifications and compile that, instead of needing to have a trained human compile the natural specifications into source code, which the computer can compile into a program.
  - seanmcdirmid 13y ago
    Sure we could. This is just like Siri, you tell it what you want, she asks for clarifications, a conversation you have with the computer until...bam, you get what you want. I would think we could do much of this already if we tried, but there are a lot of things on my list to do until I get to that.
    - encoderer 13y ago
      Your optimism on this seems unfounded in my opinion. When there are no domain restrictions, computers have not proven very capable of comprehension. Look even at the comparatively simple domain of handwriting recognition. More importantly, you entirely miss the problem that many times the human doesn't know exactly what he wants until he sees it. As a software engineer, I have no fear my job will be replaced by computers talking to product managers.
      - seanmcdirmid 13y ago
        That's why the program should be expressed as a conversation! We don't know what we want so we should start vague, get feedback, provide clarifications, and so on. The conversation with the computer must be a two-way thing!
        lifeisstillgood 13y ago
        That's brilliant. Not in terms of some AI working out what a silver haired old grandmother meant but in terms of talking to clients - they could imagine talking to a computer and I am just providing that feedback. Daily or weekly, feedback nice and quick
        seanmcdirmid 13y ago
        The thing is, we probably don't need hard AI yet to do some of this. Yes, it must be a dialogue system, but we have those today. We already saw the movement in this direction with Hugo Liu's concept.net work, but for some reason no one has followed up yet. We are getting to the point with speech rec/understanding technology that someone is bound to try again soon.
- kragen 13y ago
  That was the hype surrounding Fortran when it came out in 1959: it would eliminate the need for programmers, because you could just punch the formulas and it would translate them into a program automatically. And in a sense it did, and its successors did to an even greater extent. Today you have "nonprogrammers" who can build, in a day, in OpenOffice Calc, computations that would have taken weeks to get written by programmers in the 1950s (when they could have been done at all.) It's partly a matter of giving the objectives to the computer at progressively higher levels, and partly a matter of improving user interfaces so that you can tell what the thing is doing and what it's going to do.
- eli_gottlieb 13y ago
  These predictions always fail to account for C++. That is, having computers write code depends on the code being in a form about which computers can formally reason. If you can't prove anything about your code because the language is that bad, good luck getting a computer to write code. /explain_the_joke.jpg

ansgri 13y ago

In light of recent mass surveillance uproars, the possibility of remote eye tracking technology is what really scares me. While the technological usage could be limited or masked, such attention detector and analyzer could give enourmous powers to those who have the behavioral models. Very interesting reserach though.

joeroot 13y ago

This highlights how "good" programmers are simply those who are most able to think like a computer. The article's language further reinforces this (perhaps intentionally), e.g. the reference to "compiling" functions. I find this contraint interesting - I don't know if computational thinking is a strength or a weakness. Increasingly, it seems that computational models occur naturally and therefore the ability to think in such a manner would have inter-disciplinary value. If we deem it a weakness, then programming becomes a UX problem rather than language one. The lack of change both within and across programming paradigms would suggest that many don't believe this to be a fundamental issue.

jared314 13y ago
This actually touches on Alan Kay's statements about programming with "what" instead of "how". It has made me seriously consider logic programming for the applications I write.
MaulingMonkey 13y ago
I'd place my skill bars higher: "thinking like a computer" seems fundamental: I don't think one can even be a mediocre programmer without it, and it's not enough to make you a good programmer. A good programmer also needs to be able to effectively communicate through code to both others and themselves, know how to design maintainable programs, how to preemptively avoid bugs by making their code harder to misuse, and too many other skills to list here -- and not all of those naturally flow from knowing how to think like a computer.
eli_gottlieb 13y ago
Computers don't think like computers. Computers reason formally. Thinking like a computer is just a matter of learning to reason formally and exactly instead of employing fuzzy mental abstractions. This is primarily hard because we're wired for fuzzy mental abstractions, since they make loads of tasks (natural language, making breakfast, manual labor) far easier compared to taking genuine formal approaches, and we have a lower layer of the brain that can be trained to follow exact procedures quite well anyway.

Chris_Newton 13y ago

Previous related discussion: https://news.ycombinator.com/item?id=4940952 https://news.ycombinator.com/item?id=4940952

oblique63 13y ago

I would love to see this kind of study done comparing different programming paradigms (i.e. functional vs. procedural vs. OO vs. logic). Some concrete cognitive data on how lisp-like languages may or may not actually make programmers more productive would be quite fascinating I think.

Chris_Newton 13y ago
I’m not aware of many eye-tracking studies specifically, but there has been a lot of research done on the general subject of how we read and understand code. Search for papers on “program comprehension”. To address (some of) your specific point, I suspect part of the appeal of functional programming languages is that they naturally promote describing the data flow of the code. In contrast, imperative languages inherently dwell on control flow, even though it’s often an unimportant implementation detail. It turns out that even when we’re reading imperative code, we’re trying to figure out the underlying data flow anyway, so why not cut out the middle step? I also suspect part of what holds back functional programming languages from wider acceptance is that when you’re modelling something where time/order matters, the control flow is not an unimportant implementation detail. You have to work harder to describe it in a functional language where you get a lot for free with an imperative programming language.
- gizmo686 13y ago
  > You have to work harder to describe it in a functional language This isn't necessarily true. In Haskell you can use do blocks like: main = do action1 action2 ... and your actions will happen sequentially. This is just syntactic sugar behind "action1 >> action2". The real reason for the special syntax is when the actions also return a value you can do "x <- action1", and use the variable x. This is still just sugar, but the unsugared version gets messy pretty quickly. The big difficult in going from imperative to functional languages is thinking about data structures as immutable. You go from asking how do I change the value to asking giving this value, how do I construct a new value.
  - Chris_Newton 13y ago

buttproblem 13y ago

I see how this is interesting but I can't get over how the author says, "When contrasting this with Eric's video", but "Eric's video" is a different program than the novice programmer. Maybe this is me being too pedantic.

NoodleIncident 13y ago
Not just a different program, a better one, and easier to understand. It especially annoyed me when he praised Eric for going back to understand the function first, when that function isn't even present in the novice's program.

garysweaver 13y ago

> he "compiled" the between function in his head Really? How about "adequately understood the function and did not have to reread it". We are humans, not compilers. ;)