18 ms·

What Makes Code Hard to Understand?

97 points by denzil_correa 14y ago

dmlorenzetti 14y ago
Programmers often feel like the physical properties of notation have only a minor inﬂuence on their interpretation process. When in a hurry, they frequently dispense with recommended variable naming, indentation, and formatting as superficial and inconsequential. It's ironic that the authors of this statement can't be bothered to typeset their opening quote marks in the right direction-- a typical noobie mistake in TeX.
- denzil_correa 14y ago
  I am not sure if I understand you correctly. Is it ok if I ask you for an example?
  - ajanuary 14y ago
    At the beginning of the second paragraph on page 2: The second question is: ''How are programmers In LaTeX, `` is used for open quotes, and '' for close quotes. [0] [0] http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/QuotDash.html http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/QuotDash.html
    - denzil_correa 14y ago
      Thanks, I do know how open quotes are used in LaTex though. I couldn't understand the initial comment. Thanks for the clarification. Up-voted.
  - deleted 14y ago
    [deleted]
- synesthesiam 14y ago
  I'm not a TeX "noobie". This was a result of collaborating with multiple authors, one who preferred Word. Looks like I missed the quotes when pasting in his contributions.
groovy2shoes 14y ago
We need to have more research like this. Programmers spend a lot of time arguing about features, syntax, paradigm, methodology, etc., but no one really has any scientific proof. I laud the researchers who are coming up with new paradigms and extending current paradigms, but we really need researchers going behind and finding what's beneficial and what's not in terms of human factors.
- macspoofing 14y ago
  >We need to have more research like this. There is a lot of research done in this and other areas of Software Engineering. Problem is that it's usually locked away behind a huge paywall. If you're not affiliated with some university, you essentially have no access to it.
  - denzil_correa 14y ago
    > Problem (as with all academic papers) is that, it's usually locked away behind a huge paywall. This may be true for research in other domains but as a researcher in Computer Science I can assure you that the number of academic papers locked behind a paywall is negligible in CS. Only once was did I find a paper not available and I promptly received a copy by sending an e-mail to the authors of the paper.
    - keefe 14y ago
      almost everything I need I find in CiteSeer or ACM Digital Library (which does have a very small pay wall)
      - denzil_correa 14y ago
        Out of curiosity - which area do you work in? Can you also post a couple of examples (from your area) of academic research papers not available for free download?
        14y ago
acqq 14y ago
My "between" implementation would be like this: def between( a, low, high ): r = [] for x in a: if low < x < high: r.append( x ) return r or even def between( a, low, high ): return [ x for x in a if low < x < high ] instead of their: def between(numbers, low, high): winners = [] for num in numbers: if (low < num) and (num < high): winners.append(num) return winners The more variable is local the shorter it should be. Also by using mathematical notation you display "functionality" instead of distracting readers with the terms you coined. "Winners"?
- gee_totes 14y ago
  For me, the middle one is the most readable, but it's the hardest to figure out what data structure is being returned.
  - cecilpl 14y ago
    I agree with the middle one being the most readable (assuming you have python experience). But isn't it obviously returning a list? It's a list comprehension!
    - gee_totes 14y ago
      Ahh, that's what it's doing! Yeah, I have no python expirence.
- deleted 14y ago
  [deleted]
- jaredmcateer 14y ago
  > The more variable is local the shorter it should be. I disagree, as someone with low experience with python, I found their example much more understandable than both of your examples.
  - qu4z-2 14y ago
cecilpl 14y ago
"We were surprised to ﬁnd a signiﬁcant eﬀect of Python experience on the probability of making a very speciﬁc error (base = 0.13, OR = 1.44, p < 0.5). Instead of [8, 9, 0] for their last line of output, 22% of participants wrote [8] (for both versions). After the experiment, one participant reported they mistakenly performed the “common” operation on lists x btwn and y btwn instead of x and y because it seemed like the next logical step" I made this mistake too. In fact, even after reading that most people wrote [8] instead of [8,9,0], it took me a solid minute or so to figure out how anyone could possibly think the correct answer was [8,9,0]. This is something I've noticed happens very often while debugging. I make certain assumptions about the code and even when staring it right in the face it's hard to see where my assumptions differ from the code as written. My assumption in this case was "why would you calculate the between lists if you weren't going to use them right away".
- qu4z-2 14y ago
  I like the ff => ﬀ ligature that you've managed to paste from the .pdf. :) EDIT: I also made the mistake, but in their defence the between lists are used straight away: They're printed. But the code seems to do three unrelated things in a row, so I'd expect an implementation like their one to have three separate function calls if there's no dependency between them.
raverbashing 14y ago
I was thinking along those lines the other day. I guess the biggest hurdle is that people essentially think "functionally" not procedurally Because we fundamentally think x = x2 in terms of "the value is being doubled" without thinking of all the steps needed and everything that can go wrong there. The article is certainly interesting, from what I can quickly glance, naming is very important
- friendly_chap 14y ago
  I think functional programming and immutability goes hand in hand, so I wouldn't call x = x*2 an example of the functional thinking. With the same logic, C is functional because you don't care about the bit fiddling in x++...
  - raverbashing 14y ago
    Yes, x = x*2 is not (y = x*2 would be better) But expecting a while loop to exit as soon as the condition is met (as opposed as while the loop block finished) is a common case
- gizmo686 14y ago
  Interestingly, I remember reading an article saying that people who have never programmed before could learn haskell faster than they could learn java. But people who already learned java take longer than someone with no experience to learn haskell. I will see if I can find the link.
largesse 14y ago
Nice paper, but I wish the results were easier to understand.
keefe 14y ago
"expectation-congruent programs should take less time to understand" this statement feels tautological, I don't think there's a single correct set of expectations
- happimess 14y ago
  It's not a tautology. If the brain's benefit from "branch prediction" is low, then some other factor (e.g. length of code) could overwhelm the effects of predictability. So they're making a genuine claim, which is that expectation-congruence is the most important factor. My experience bears this out. In most of my professional life, I've abided by coding standards, even if I consider the standards imperfect. That's because of the very real improvements to the team's productivity when code does what it looks like it does.
  - keefe 14y ago
    not quite a tautology, I was being a bit facetious... imho the concept of understanding is tied up with the meaning of expectation, so much so that by definition the code I don't understand doesn't do what I expect - if it did, then I'd understand it!
Arjuna 14y ago
Interesting paper. The following code is probably my favorite, "What will this program output?" example. This is taken from the Quake III Arena code in "q_math.c" [1]. Note line 561 [2]. Non-obfuscated code, and one is left wondering... just what, exactly, is going on at that line? I understand that the point of the paper is to analyze code without having comments to help, but it serves as a reminder to me that commenting is important in helping not only other developers understand the code, but to help myself when revisiting code particulars that may have faded from memory. I find this to be a great example when I hear, "I don't comment; the code itself is self-documenting." 552 float Q_rsqrt( float number ) 553 { 554 long i; 555 float x2, y; 556 const float threehalfs = 1.5F; 557 558 x2 = number * 0.5F; 559 y = number; 560 i = * ( long * ) &y; // evil floating point bit level hacking 561 i = 0x5f3759df - ( i >> 1 ); // what the fuck? 562 y = * ( float * ) &i; 563 y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration 564 // y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed 565 566 #ifndef Q3_VM 567 #ifdef __linux__ 568 assert( !isnan(y) ); // bk010122 - FPE? 569 #endif 570 #endif 571 return y; 572 } [1] https://github.com/id-Software/Quake-III-Arena/blob/master/code/game/q_math.c#L552 https://github.com/id-Software/Quake-III-Arena/blob/master/c... [2] https://en.wikipedia.org/wiki/Fast_inverse_square_root https://en.wikipedia.org/wiki/Fast_inverse_square_root
- ryusage 14y ago
  That is a great example. Some code just absolutely needs extra documentation around it. IMO, this is mostly true with more complicated or non-obvious algorithms like your example. There aren't really much more intelligible names you can give those variables; the algorithm just needs them. Still, I find that with 90% of the code I write for work, it's not so difficult to come up with variable and method names that make it pretty clear what's going on. I take the approach that I'll add comments anywhere I think that naming might not be enough, but that very rarely happens. Granted, I'm not writing low-level, math heavy code. I think it really just depends.
- milkshakes 14y ago
  So beautiful. And yet, sadly, "for every 0x5f3759df we learn about, there are thousands we never see"[1] [1]: http://www.xkcd.com/664/ http://www.xkcd.com/664/
agentultra 14y ago
I've come to believe that it is the cognitive dissonance between the programmer and the machine. The language used often obscures what is actually executed. Often we find this useful but without proper cues we can write programs that make the divide between program and process really wide. Personally I find declarations about properties, relationships, and categories of things easier to understand than descriptions of the tedious processes that calculate them. The former give me the ability to reason about the program elements in a logical way. The latter requires me to become a stack machine and execute the program in my head... a process that is error-prone and full of false assumptions. eg: how many people believe that "all arrays are just pointers," in C? When is an array not like a pointer? Research like this is good. Are there any studies that go beyond trying to trick programmers with trivial programs in the Algol family and look at the difference in performance between categorically different styles? I mean languages that are declarative vs imperative vs functional vs concatenative in nature. I think that would be very interesting to read. updated for clarity
- jeremyjh 14y ago
  > eg: how many people believe that "all arrays are just pointers," in C? When is an array not like a pointer? Ok I know this wasn't your point but I got stuck on this and am just curious to know: what you were thinking of here? One thing that occurs to me is that in a recursive function you will run out of stack space a lot faster if you are using arrays rather than allocating from the heap. I don't think that is what you were referring to though, hence the question.
  - agentultra 14y ago
    Well my example was meant to support how programming languages can create a cognitive dissonance between what the programmer thinks will be executed vs. what is actually executed. One area where I think there is a high dissonance is in how C uses the same syntax for defining arrays, indexing into arrays, and referencing pointer offsets. Other examples are the array decomposition rules, array function parameters, etc. Even experienced programmers get tripped up by them: http://c-faq.com/~scs/cgi-bin/faqcat.cgi?sec=aryptr http://c-faq.com/~scs/cgi-bin/faqcat.cgi?sec=aryptr
wting 14y ago
When reading the paper's verbose versions, I have to act as a human compiler and keep track of state which leads to mental errors. Contrast this with a shorter version reducing the LOC count from 21 to 6: x = [..] y = [..] between = lambda ls, low, high: (n for n in ls if low < n < high) x_between = between(x, 2, 10) y_between = between(y, -2, 9) xy_common = set(x).intersection(y) Experienced users made mental stakes because they skim code and make assumptions from previous sections. This can be solved by taking advantage of a language's expressiveness to clarify intent.
incision 14y ago
Forgive my ignorance - IANAP. Why can't/don't we have more tools along the lines of App Inventor [1] that provide a visual structure for programming? The vast majority of time I've spent writing anything has been spent struggling with specific syntax, not any particular function of the code. Coding feels like trying to manipulate a clear model that exists in the machine by spelling my intentions out one letter at a time while peering through a straw. 1: http://appinventor.mit.edu/ http://appinventor.mit.edu/
- kbenson 14y ago
  Possibly because while we have a rich ecosystem of tools for dealing with text in myriad form and ways, and have conditioned ourselves over the years on how to read text in a general form, I don't think the same can be said for more graphical representations of programs. While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless. P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it.
  - incision 14y ago
    >Possibly because while we have a rich ecosystem of tools for dealing with text in myriad form and ways, and have conditioned ourselves over the years on how to read text in a general form, I don't think the same can be said for more graphical representations of programs. Yes, I completely agree. To go off on a bit of a tangent, I wonder if it's just a matter of the right technology and implementation? Is there an analog to the rise of touch interfaces waiting to happen in the way we record, manipulate and relay information? >While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless. That's a good point. I was thinking of this purely from the standpoint of creating as an individual, not maintaining or sharing. >P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it. Certainly, learning that word (idiomatic) was a boon to my understanding of code.
kruhft 14y ago
Code is hard to understand because most programmers only learn how to write it and never take the time to learn how to read it. If you can't read something with ease, it will always be hard to understand.
- nooneelse 14y ago
  Also, if you never learn to read it, you can't begin to learn to write for the reader.
jdlshore 14y ago
Interesting study. Here's a summary of their results: 1: "between". Experienced programmers were more likely to incorrectly assume that the results of earlier calculations would be used in a later calculation. 2: "counting". All participants, regardless of experience, were more likely to assume that a statement separated by vertical whitespace was outside of a loop, when it was actually still inside the loop. (Note that the programming language is Python, which doesn't have a loop termination token.) 3: "funcall". Whitespace between operators (e.g., 3+4+5 vs. 3 + 4 + 5) had no effect. 4: "initvar". No interesting result. (This one seems to have been mis-designed.) 5: "order". Respondents were slower when functions were defined in a different order than they were used. Experienced programmers didn't slow down as much. 6: "overload". Experienced programmers were more likely to be slower when faced with a "+" operator used for both string concatenation ("5" + "6") and addition (5 + 6), rather than used for just concatenation. 7: "partition". There was a result, but I don't think we can draw meaningful conclusions from it. 8: "rectangle". Calculating the area of a rectangle using a function that took tuples (e.g., "area(xy_1, xy_2)") took longer for programmers to understand than a function that took scalars (e.g., "area(x1, y1, x2, y2)"). Calculating the area using a class (e.g., "rect = Rectangle(0, 0, 10, 10); print rect.area()" took the same amount of time as using scalars, despite being a longer program. 9: "scope". No conclusive result. 10: "whitespace". Using horizontal whitespace to line up similar statements had no effect on response time. (There was another result relating to order of operations that deserves further study, but it wasn't thoroughly tested here, so I don't think it's conclusive.) Note that all programs were exceedingly simple (the longest was 24 lines), so be cautious applying these conclusions to real-world work.
randscreen 14y ago
From the paper: "This program prints the product f (1) ∗ f (0) ∗ f (−1) where f (x) = x + 5. ... Most people were able to get the correct answer of 60 in 30 seconds or less." Unless I'm losing my mind, the correct answer for the code as stated is 6 * 5 * 4=120, not 60. It would be 60 if f (x) = x + 4.
- synesthesiam 14y ago
  You're correct. This was a (very) unfortunate typo. We had changed from x + 5 to x + 4 during the design phase to try and lessen the mental burden without making it too easy. Thanks to you and a few others, I'm making corrections and will be uploading a fixed version of the paper soon.
synesthesiam 14y ago
Lots of great discussion here! If anyone is interested, I added a blog post about the paper (http://synesthesiam.com/posts/what-makes-code-hard-to-understand.html http://synesthesiam.com/posts/what-makes-code-hard-to-unders...) and put the data up on github (https://github.com/synesthesiam/eyecode-tools https://github.com/synesthesiam/eyecode-tools).

97 points by denzil_correa 14y ago

Programmers often feel like the physical properties of notation have only a minor inﬂuence on their interpretation process. When in a hurry, they frequently dispense with recommended variable naming, indentation, and formatting as superficial and inconsequential. It's ironic that the authors of this statement can't be bothered to typeset their opening quote marks in the right direction-- a typical noobie mistake in TeX.

denzil_correa 14y ago
I am not sure if I understand you correctly. Is it ok if I ask you for an example?
- ajanuary 14y ago
  At the beginning of the second paragraph on page 2: The second question is: ''How are programmers In LaTeX, `` is used for open quotes, and '' for close quotes. [0] [0] http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/QuotDash.html http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/QuotDash.html
  - denzil_correa 14y ago
    Thanks, I do know how open quotes are used in LaTex though. I couldn't understand the initial comment. Thanks for the clarification. Up-voted.
- deleted 14y ago
  [deleted]
synesthesiam 14y ago
I'm not a TeX "noobie". This was a result of collaborating with multiple authors, one who preferred Word. Looks like I missed the quotes when pasting in his contributions.

groovy2shoes 14y ago

We need to have more research like this. Programmers spend a lot of time arguing about features, syntax, paradigm, methodology, etc., but no one really has any scientific proof. I laud the researchers who are coming up with new paradigms and extending current paradigms, but we really need researchers going behind and finding what's beneficial and what's not in terms of human factors.

macspoofing 14y ago
>We need to have more research like this. There is a lot of research done in this and other areas of Software Engineering. Problem is that it's usually locked away behind a huge paywall. If you're not affiliated with some university, you essentially have no access to it.
- denzil_correa 14y ago
  > Problem (as with all academic papers) is that, it's usually locked away behind a huge paywall. This may be true for research in other domains but as a researcher in Computer Science I can assure you that the number of academic papers locked behind a paywall is negligible in CS. Only once was did I find a paper not available and I promptly received a copy by sending an e-mail to the authors of the paper.
  - keefe 14y ago
    almost everything I need I find in CiteSeer or ACM Digital Library (which does have a very small pay wall)
    - denzil_correa 14y ago
      Out of curiosity - which area do you work in? Can you also post a couple of examples (from your area) of academic research papers not available for free download?
      - 14y ago

acqq 14y ago

My "between" implementation would be like this: def between( a, low, high ): r = [] for x in a: if low < x < high: r.append( x ) return r or even def between( a, low, high ): return [ x for x in a if low < x < high ] instead of their: def between(numbers, low, high): winners = [] for num in numbers: if (low < num) and (num < high): winners.append(num) return winners The more variable is local the shorter it should be. Also by using mathematical notation you display "functionality" instead of distracting readers with the terms you coined. "Winners"?

gee_totes 14y ago
For me, the middle one is the most readable, but it's the hardest to figure out what data structure is being returned.
- cecilpl 14y ago
  I agree with the middle one being the most readable (assuming you have python experience). But isn't it obviously returning a list? It's a list comprehension!
  - gee_totes 14y ago
    Ahh, that's what it's doing! Yeah, I have no python expirence.
deleted 14y ago
[deleted]
jaredmcateer 14y ago
> The more variable is local the shorter it should be. I disagree, as someone with low experience with python, I found their example much more understandable than both of your examples.
- qu4z-2 14y ago

cecilpl 14y ago

"We were surprised to ﬁnd a signiﬁcant eﬀect of Python experience on the probability of making a very speciﬁc error (base = 0.13, OR = 1.44, p < 0.5). Instead of [8, 9, 0] for their last line of output, 22% of participants wrote [8] (for both versions). After the experiment, one participant reported they mistakenly performed the “common” operation on lists x btwn and y btwn instead of x and y because it seemed like the next logical step" I made this mistake too. In fact, even after reading that most people wrote [8] instead of [8,9,0], it took me a solid minute or so to figure out how anyone could possibly think the correct answer was [8,9,0]. This is something I've noticed happens very often while debugging. I make certain assumptions about the code and even when staring it right in the face it's hard to see where my assumptions differ from the code as written. My assumption in this case was "why would you calculate the between lists if you weren't going to use them right away".

qu4z-2 14y ago
I like the ff => ﬀ ligature that you've managed to paste from the .pdf. :) EDIT: I also made the mistake, but in their defence the between lists are used straight away: They're printed. But the code seems to do three unrelated things in a row, so I'd expect an implementation like their one to have three separate function calls if there's no dependency between them.

raverbashing 14y ago

I was thinking along those lines the other day. I guess the biggest hurdle is that people essentially think "functionally" not procedurally Because we fundamentally think x = x2 in terms of "the value is being doubled" without thinking of all the steps needed and everything that can go wrong there. The article is certainly interesting, from what I can quickly glance, naming is very important

friendly_chap 14y ago
I think functional programming and immutability goes hand in hand, so I wouldn't call x = x*2 an example of the functional thinking. With the same logic, C is functional because you don't care about the bit fiddling in x++...
- raverbashing 14y ago
  Yes, x = x*2 is not (y = x*2 would be better) But expecting a while loop to exit as soon as the condition is met (as opposed as while the loop block finished) is a common case
gizmo686 14y ago
Interestingly, I remember reading an article saying that people who have never programmed before could learn haskell faster than they could learn java. But people who already learned java take longer than someone with no experience to learn haskell. I will see if I can find the link.

largesse 14y ago

Nice paper, but I wish the results were easier to understand.

keefe 14y ago

"expectation-congruent programs should take less time to understand" this statement feels tautological, I don't think there's a single correct set of expectations

happimess 14y ago
It's not a tautology. If the brain's benefit from "branch prediction" is low, then some other factor (e.g. length of code) could overwhelm the effects of predictability. So they're making a genuine claim, which is that expectation-congruence is the most important factor. My experience bears this out. In most of my professional life, I've abided by coding standards, even if I consider the standards imperfect. That's because of the very real improvements to the team's productivity when code does what it looks like it does.
- keefe 14y ago
  not quite a tautology, I was being a bit facetious... imho the concept of understanding is tied up with the meaning of expectation, so much so that by definition the code I don't understand doesn't do what I expect - if it did, then I'd understand it!

Arjuna 14y ago

Interesting paper. The following code is probably my favorite, "What will this program output?" example. This is taken from the Quake III Arena code in "q_math.c" [1]. Note line 561 [2]. Non-obfuscated code, and one is left wondering... just what, exactly, is going on at that line? I understand that the point of the paper is to analyze code without having comments to help, but it serves as a reminder to me that commenting is important in helping not only other developers understand the code, but to help myself when revisiting code particulars that may have faded from memory. I find this to be a great example when I hear, "I don't comment; the code itself is self-documenting." 552 float Q_rsqrt( float number ) 553 { 554 long i; 555 float x2, y; 556 const float threehalfs = 1.5F; 557 558 x2 = number * 0.5F; 559 y = number; 560 i = * ( long * ) &y; // evil floating point bit level hacking 561 i = 0x5f3759df - ( i >> 1 ); // what the fuck? 562 y = * ( float * ) &i; 563 y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration 564 // y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed 565 566 #ifndef Q3_VM 567 #ifdef __linux__ 568 assert( !isnan(y) ); // bk010122 - FPE? 569 #endif 570 #endif 571 return y; 572 } [1] https://github.com/id-Software/Quake-III-Arena/blob/master/code/game/q_math.c#L552 https://github.com/id-Software/Quake-III-Arena/blob/master/c... [2] https://en.wikipedia.org/wiki/Fast_inverse_square_root https://en.wikipedia.org/wiki/Fast_inverse_square_root

ryusage 14y ago
That is a great example. Some code just absolutely needs extra documentation around it. IMO, this is mostly true with more complicated or non-obvious algorithms like your example. There aren't really much more intelligible names you can give those variables; the algorithm just needs them. Still, I find that with 90% of the code I write for work, it's not so difficult to come up with variable and method names that make it pretty clear what's going on. I take the approach that I'll add comments anywhere I think that naming might not be enough, but that very rarely happens. Granted, I'm not writing low-level, math heavy code. I think it really just depends.
milkshakes 14y ago
So beautiful. And yet, sadly, "for every 0x5f3759df we learn about, there are thousands we never see"[1] [1]: http://www.xkcd.com/664/ http://www.xkcd.com/664/

agentultra 14y ago

I've come to believe that it is the cognitive dissonance between the programmer and the machine. The language used often obscures what is actually executed. Often we find this useful but without proper cues we can write programs that make the divide between program and process really wide. Personally I find declarations about properties, relationships, and categories of things easier to understand than descriptions of the tedious processes that calculate them. The former give me the ability to reason about the program elements in a logical way. The latter requires me to become a stack machine and execute the program in my head... a process that is error-prone and full of false assumptions. eg: how many people believe that "all arrays are just pointers," in C? When is an array not like a pointer? Research like this is good. Are there any studies that go beyond trying to trick programmers with trivial programs in the Algol family and look at the difference in performance between categorically different styles? I mean languages that are declarative vs imperative vs functional vs concatenative in nature. I think that would be very interesting to read. updated for clarity

jeremyjh 14y ago
> eg: how many people believe that "all arrays are just pointers," in C? When is an array not like a pointer? Ok I know this wasn't your point but I got stuck on this and am just curious to know: what you were thinking of here? One thing that occurs to me is that in a recursive function you will run out of stack space a lot faster if you are using arrays rather than allocating from the heap. I don't think that is what you were referring to though, hence the question.
- agentultra 14y ago
  Well my example was meant to support how programming languages can create a cognitive dissonance between what the programmer thinks will be executed vs. what is actually executed. One area where I think there is a high dissonance is in how C uses the same syntax for defining arrays, indexing into arrays, and referencing pointer offsets. Other examples are the array decomposition rules, array function parameters, etc. Even experienced programmers get tripped up by them: http://c-faq.com/~scs/cgi-bin/faqcat.cgi?sec=aryptr http://c-faq.com/~scs/cgi-bin/faqcat.cgi?sec=aryptr

wting 14y ago

When reading the paper's verbose versions, I have to act as a human compiler and keep track of state which leads to mental errors. Contrast this with a shorter version reducing the LOC count from 21 to 6: x = [..] y = [..] between = lambda ls, low, high: (n for n in ls if low < n < high) x_between = between(x, 2, 10) y_between = between(y, -2, 9) xy_common = set(x).intersection(y) Experienced users made mental stakes because they skim code and make assumptions from previous sections. This can be solved by taking advantage of a language's expressiveness to clarify intent.

incision 14y ago

Forgive my ignorance - IANAP. Why can't/don't we have more tools along the lines of App Inventor [1] that provide a visual structure for programming? The vast majority of time I've spent writing anything has been spent struggling with specific syntax, not any particular function of the code. Coding feels like trying to manipulate a clear model that exists in the machine by spelling my intentions out one letter at a time while peering through a straw. 1: http://appinventor.mit.edu/ http://appinventor.mit.edu/

kbenson 14y ago
Possibly because while we have a rich ecosystem of tools for dealing with text in myriad form and ways, and have conditioned ourselves over the years on how to read text in a general form, I don't think the same can be said for more graphical representations of programs. While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless. P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it.
- incision 14y ago
  >Possibly because while we have a rich ecosystem of tools for dealing with text in myriad form and ways, and have conditioned ourselves over the years on how to read text in a general form, I don't think the same can be said for more graphical representations of programs. Yes, I completely agree. To go off on a bit of a tangent, I wonder if it's just a matter of the right technology and implementation? Is there an analog to the rise of touch interfaces waiting to happen in the way we record, manipulate and relay information? >While a representation that uses graphical cues in lieu of textual ones may be easier to learn and understand initially, the disadvantage of not being able to use the thousands of utilities for searching, refactoring examining code are then less useful, if not entirely useless. That's a good point. I was thinking of this purely from the standpoint of creating as an individual, not maintaining or sharing. >P.S. In a way, idiomatic structure to code puts a graphical representation on the textual form. Programmers learn to recognize this. Python goes as far as to enforce it. Certainly, learning that word (idiomatic) was a boon to my understanding of code.

kruhft 14y ago

Code is hard to understand because most programmers only learn how to write it and never take the time to learn how to read it. If you can't read something with ease, it will always be hard to understand.

nooneelse 14y ago
Also, if you never learn to read it, you can't begin to learn to write for the reader.

jdlshore 14y ago

Interesting study. Here's a summary of their results: 1: "between". Experienced programmers were more likely to incorrectly assume that the results of earlier calculations would be used in a later calculation. 2: "counting". All participants, regardless of experience, were more likely to assume that a statement separated by vertical whitespace was outside of a loop, when it was actually still inside the loop. (Note that the programming language is Python, which doesn't have a loop termination token.) 3: "funcall". Whitespace between operators (e.g., 3+4+5 vs. 3 + 4 + 5) had no effect. 4: "initvar". No interesting result. (This one seems to have been mis-designed.) 5: "order". Respondents were slower when functions were defined in a different order than they were used. Experienced programmers didn't slow down as much. 6: "overload". Experienced programmers were more likely to be slower when faced with a "+" operator used for both string concatenation ("5" + "6") and addition (5 + 6), rather than used for just concatenation. 7: "partition". There was a result, but I don't think we can draw meaningful conclusions from it. 8: "rectangle". Calculating the area of a rectangle using a function that took tuples (e.g., "area(xy_1, xy_2)") took longer for programmers to understand than a function that took scalars (e.g., "area(x1, y1, x2, y2)"). Calculating the area using a class (e.g., "rect = Rectangle(0, 0, 10, 10); print rect.area()" took the same amount of time as using scalars, despite being a longer program. 9: "scope". No conclusive result. 10: "whitespace". Using horizontal whitespace to line up similar statements had no effect on response time. (There was another result relating to order of operations that deserves further study, but it wasn't thoroughly tested here, so I don't think it's conclusive.) Note that all programs were exceedingly simple (the longest was 24 lines), so be cautious applying these conclusions to real-world work.

randscreen 14y ago

From the paper: "This program prints the product f (1) ∗ f (0) ∗ f (−1) where f (x) = x + 5. ... Most people were able to get the correct answer of 60 in 30 seconds or less." Unless I'm losing my mind, the correct answer for the code as stated is 6 * 5 * 4=120, not 60. It would be 60 if f (x) = x + 4.

synesthesiam 14y ago
You're correct. This was a (very) unfortunate typo. We had changed from x + 5 to x + 4 during the design phase to try and lessen the mental burden without making it too easy. Thanks to you and a few others, I'm making corrections and will be uploading a fixed version of the paper soon.

synesthesiam 14y ago

Lots of great discussion here! If anyone is interested, I added a blog post about the paper (http://synesthesiam.com/posts/what-makes-code-hard-to-understand.html http://synesthesiam.com/posts/what-makes-code-hard-to-unders...) and put the data up on github (https://github.com/synesthesiam/eyecode-tools https://github.com/synesthesiam/eyecode-tools).