8 ms·

How do we read code?

214 points by erjiang 14y ago

3pt14159 14y ago
Great post. Very relevant to this audience and very interesting research may come from it. I look forward to seeing follow ups.
columbo 14y ago
This is very interesting. Have they considered using multiple languages (functional, oo) as well as including 'garbage' languages (brainfuck) to track differences? I'd also be curious to see how it compares to a standard word problem (two trains leaving chicago etc).
- synesthesiam 14y ago
  Here's an interesting paper where they compare their language (Quorum), Perl, and a language with random ASCII tokens: http://ecs.victoria.ac.nz/twiki/pub/Events/PLATEAU/Program/plateau2011-stefik.pdf http://ecs.victoria.ac.nz/twiki/pub/Events/PLATEAU/Program/p... (Spoiler alert: Perl doesn't fair much better than the random language!)
akshaykarthik 14y ago
One of the things I noticed from the Eye Tracking was that this is very similar to how we were trained to work through programming problems in high school level computer science competitions. Some competition problems involve tracing through code and determining output (of usually recursive functions) but the 'proper' method we were taught is almost exactly how he describes his thought process.
afhof 14y ago
Makes me wonder if future languages will be designed to be read in serial. Perhaps that will decrease the amount of time we need to look at code to understand what it does.
- kscaldef 14y ago
  Probably not. I don't think that in the large that helps with readability / maintainability. Even in the small, I find languages which don't allow forward references (Ruby, sh) rather annoying. It forces relatively insignificant implementation details to be read first, rather than starting with the big picture of what the program does.
- saurabh 14y ago
  I would be interesting to do a similar study on some Factor code.
  - evincarofautumn 14y ago
    I would expect similar results. Factor and other stack-based languages are not particularly linear, thanks to, well, factoring. Sure, data flow generally runs from left to right and top to bottom, but word and tuple definitions are just as scattered about as in any procedural language.
darrennix 14y ago
I'll be very interested in findings on the terseness of code and its effects on readability by experienced coders. For example, the ternary operator is consider by many to be an elegant solution to simple if statements but from a comprehension (and therefore bug-finding) standpoint, is it superior? Also, how does this effect change as the size of the codebase grows from a single page (as depicted in the video) to a more complex class file. value = test ? (some_value * multiple) : false_value # vs if (test) { value = some_value * multiple } else { value = false_value }
- birken 14y ago
  I would say it is a matter of personal preference. However, first and foremost whichever thing you prefer should be codified in your project's style guide. Consistency trumps everything else by a large margin. My personal opinion is that this is an excellent use of the ternary operator. When writing software, you want your code to be as simple and short as possible provided it is readable[1]. I find both of them to be equally readable, therefore I prefer the shorter one by a lot. We are talking about 5 lines of code vs 1, which can really add up if this appears commonly all over your codebase. One limited asset every engineer has is monitor real estate, and the more code you can fit on your screen, the better (provided all of that code is the readable variety). I also find in this particular example, the ternary could get a slight nod for readability as well. This is because with the ternary operator, you start with: value = Ok, value is getting assigned to something... then you read the ternary expression. If you are grepping through this code and looking to see what value might be set to, you get to this line and you know you have found it, then you parse the rest of it to figure out what it is being set to. With the other example, your grepping will lead you to a "value =" that is within a conditional, so now you have to look up and down and explore a little more to see what it might be set to. This is because the ternary operator can only do one small thing, whereas the if statement can do lots of things. Since you are only doing the simple thing, using the simplest possible operator to do that in some sense helps future people reading the code [2]. [1] Which might seem like a spectrum, but I think of it as much more binary. Code is either readable or it isn't. This might seem strange, but my fellow engineers and I spend quite a bit of time grading code submissions to our programming challenges, and of course the "readability" of the code is a key thing we grade for. I think we probably match up our independent up or down votes on readability at least 90% of the time. [2] If you want to frustrate experienced engineers, have them look through a bunch of code that hasn't been factored down to its simplest form. This is extremely common among inexperienced engineers who refactor code and don't delete a bunch of cruft that now exists due to code or logic changes because "who cares? the code works!", and you get stuff like this: if ($has_phone_number) { return TRUE; } else if ($has_phone_number || $has_email_address) { return TRUE; } else if (!$has_phone_number) { return FALSE; } else { return FALSE; } return FALSE; ahhhh!
tejaswiy 14y ago
Would it be possible for you to open source the eye tracking piece of this ? I'm interested in doing this for photographs or paintings ...
- eholk 14y ago
  It looked like a commercial eye tracker to me, so we wouldn't be able to open source it. However, a quick Google search revealed some things that might be promising.
- eholk 14y ago
  I forgot to add, I would also be interested in seeing this for photographs and paintings!
- synesthesiam 14y ago
  Do you mean the videos themselves, the software, or the hardware?
- lomendil 14y ago
  Eye tracker hardware is tricky and therefore usually expensive, but on the software side there are good tools. I've used VisionEgg[0] with an Eyelink II system and found it quite easy to deal with. [0] http://www.visionegg.org/ http://www.visionegg.org/
  - dchichkov 14y ago
    Not that expensive. For a couple of hundred dollars you can build very nice 60-90fps IR eye tracking system. You'll need IR camera, IR tele lens (very important), couple of IR sources and open source tracking software. See http://www.gazegroup.org/develop/ http://www.gazegroup.org/develop/ There is also alternative approach. If you are just recording and don't need interactivity - all you need, is a regular camera with zoom and a small tripod. You can record on an SD card and synchronize the stream stream time in post-processing. Just flash the monitor screen couple of times at the start and the end of the recording, and play tracking alignment sequence. Here's an attempt to do just that: http://www.youtube.com/watch?v=zJqKRL9D2qY http://www.youtube.com/watch?v=zJqKRL9D2qY
akennberg 14y ago
"One of the things that stood out to me in watching the video was how much my mind seems to work like a computer." A more accurate way of thinking about this is: how much the computer works like our mind. Not surprising considering that people build things based on how we understand everything else. Whether it was done consciously or subconsciously.
- eholk 14y ago
  I read somewhere once that we tend to think of our mind in terms of whatever the current most complex technology is. At one point we thought of the brain as a fantastically complicated plumbing system, and later as a telephone switchboard. I wonder if in another decade or so will the idea that our brains work like computers seem as wrong as the idea that they work like plumbing systems.
  - lomendil 14y ago
    I think that decade is now. Brains work nothing like computers. Well, as much as they do like plumbing systems ;)
Chris_Newton 14y ago
Program comprehension is a fascinating (and IMHO very enlightening) field. For anyone curious, I wrote up some of what I learned while exploring the research a few years ago: http://www.clarityincode.com/readability/ http://www.clarityincode.com/readability/ (I apologise for the less than stellar formatting; I haven’t updated that page for some time. I also apologise to the actual researchers I cited, if I’ve dumbed down their work too much in aiming for a non-expert audience.) For the eye tracking reported here, I wonder whether the early emphasis on the top part was a combination of trying to figure out the data flow in the between() function and then its significance to the wider program. I think it would be interesting to compare the results with a similar eye track of a program written with more emphasis on data flow rather than control flow, e.g., def between(numbers, low, high): return [n for n in numbers if low < n < high] def common(list1, list2): return [i for i in list1 if i in list2] x = [2, 8, 7, 9, -5, 0, 2] x_btwn = between(x, 2, 10) print x_btwn y = [1, -3, 10, 0, 8, 9, 1] y_btwn = between(y, -2, 9) print y_btwn xy_common = common(x, y) print xy_common It might also be interesting to compare the results with a functional programming language that expresses those ideas more concisely and/or with tools like between() and common() as part of the standard library that programmers would probably be familiar with. Final thought: How much does the absence of a clearly marked starting point (like a main() function in C) affect how a reader approaches unknown code in Python? If this had been a C program, would the reader have aimed straight for main() and then worked down from there to functions like between() and common()?
- dchichkov 14y ago
  I think it is very important to use code from real-life applications, rather then synthetic examples like that, or like ones presented in the original article. Otherwise the results of that research would be biased towards comprehension of meaningless chunks of code, rather that of a real thing.
  - Chris_Newton 14y ago
    I agree, but isn’t this the curse of most experimental research in programming techniques? Ideally, we’d compare like-against-like using industrial scale applications, implemented by professional practitioners, controlling for everything except what we’re trying to investigate. Finding opportunities to do that in practice is rather harder, because obviously most real software development projects don’t get implemented twice by identical teams making exactly one significant change in their approach. What I thought was interesting here was that even though it is only a toy program, there were still some patterns to how the reader explored it that might suggest more general trends. As long as we understand the limitations of the experiment and don’t overgeneralise any conclusions, isn’t some data still better than random conjecture?
suyash 14y ago
This video seems incorrect to me, I read code line by line one at a time most of the time. The video seems to be showing that the eye is jumping all around.
- Hemospectrum 14y ago
  Your eyes jump around a lot more than you think they do, no matter what you're looking at. Your brain compensates for the effect somewhat, so it's impossible to notice the full extent of the movement on a conscious level. Some quick reading on the phenomenon: http://en.wikipedia.org/wiki/Saccade http://en.wikipedia.org/wiki/Saccade
- evincarofautumn 14y ago
  Consider that you can conceptualise reading code as a linear narrative even if your eyes tell a different story. Our perceptions of ourselves are often radically different from our actual behaviour. One fine example of this is in phonetics: the two /t/ sounds in “tomato” are quite different, yet we think of them as the same sound.
synesthesiam 14y ago
It's awesome to see that people are interested in my research! I've made a blog post with another video and a few more details: http://synesthesiam.com/?p=218 http://synesthesiam.com/?p=218
evincarofautumn 14y ago
“One of the things that stood out to me in watching the video was how much my mind seems to work like a computer.” One of the things that stood out to me in watching the video was how much his mind seemed emphatically not to work like a computer at all. His process gave the appearance of a network self-training for a little while, then simultaneously training and producing output. The more times an area of the program was visited, the better the training, and consequently the longer it could be retained. Notice how the results of calculations are “picked up” from the source and “dropped” almost immediately into the output, as though they’re heavy and difficult to hold on to!
route66 14y ago
From http://infoscience.epfl.ch/record/138586 http://infoscience.epfl.ch/record/138586 : ... that modern languages, such as Scala, offer advantages as human communication mediums. I describe an experiment, using an eye-tracking device, that measures the performance of code comprehension.
Raphael_Amiard 14y ago
> In programming language terms, I seem to be doing some kind of just-in-time compilation Seems a lot more like abstract interpretation to me ! Would be a lot more logical too :)
mattmanser 14y ago
That code is jarring to my eye because of the variable name, my eye tracking would keep going back to 'winners'. Why winners? What a bizarre variable name, to me anyway. Does anyone else use that? I always go with `matches`, `retVals` or `returnValues` depending on the language/IDE I'm using.
- eholk 14y ago
  The experiment has several variants of each program. In some cases the names make sense, and in other cases they are just random words. One of the goals is to measure how much choosing good names matters.
- synesthesiam 14y ago
  I went back and forth over what to name this variable. I wanted it to convey some meaning (winners are the items that passed or "won" the test), but not give away too much. Several people have commented on this, however, so I may just change the name to "matches" or something. Thanks for the suggestion :)
graue 14y ago
I'm surprised no one pointed out that the answer given in the video is wrong. The second line should be "10 0 8 1" and the third "8 9 0". The author writes "10 0 9 1" and "9". Perhaps it's not relevant to the experiment, but it seems worthy of mention at least. Following along, I was second-guessing myself because my answer didn't match...
dschiptsov 14y ago
I don't think eye tracking and do any good, because recognizing of familiar shapes and then mapping them to a familiar constructs and "structures" comes first. So, reading a code with familiar shape is one thing (that why Lisp has such emphasis on the form of an expression, and Python put that into extreme), while reading long.chains.of.unfamiliar.methods.is.another.) Then comes recognition of a familiar zones (areas) of an expression, expecting particular kind of sub-expressions here and there. Then match what you have seen with known whole things. Lets say it is a recursive process of reduction to something already known by examining shapes, forms, and details. So, shape matters. Small procedures, around ten lines matters. Naming matters, and, especially, using one-letter, non-confusing (no meaning) names for just a placeholders matters. Let say that this solved in Lambda Calculus (by a naming strategy), and then in Lisp (by shapping strategy) by accident.)
- mcherm 14y ago
  I find it rather amusing, given the context, that I spent a long time (more than 30 seconds) trying to figure out the match for your unbalanced parenthesis. I don't think I would have done that if it hadn't been a paragraph about Lisp: I kept going back thinking "Wait, I didn't realize I was in a parenthetical expression... does this change the interpretation of what is being said?".
  - kolektiv 14y ago
    Seconded. Surprising how distracting something so trivial is given recent (unrelated) context. My mind immediately sprang to the conclusion that perhaps this sentence is partially applied - a legacy of an evening spent writing some interesting functional code for treating MIDI as observable sequences.
JoeAltmaier 14y ago
100 times more frequently I read code I already read (or wrote) before. It works differently. I page through, scanning the code, noting its "shape" as I go without actually reading any lines. Once I've found the place the problem might be, I start hand-executing (or eye-executing?) for problems. Does it really matter much how we read new code? We read many tims faster than we write; re-reading is the norm.
gtani 14y ago
Funny, i mentioned this a few days ago. I was going to do some googling on other interesting code quality/dev productivity/language metrics, but, uh, never got around to it. It seems to me that in the first few passes on-sighting or sight reading code (as climbers and pianists call it), you're looking for easy to comprehend structures, and blocking off difficult to decipher, simultaneously, so maybe a bimodal distributions at work here https://news.ycombinator.com/item?id=4926313 https://news.ycombinator.com/item?id=4926313
robomartin 14y ago
Language has to be an important factor here. If I re-write this in APL --and you know APL-- reading the solution is pretty linear: R ← data BETWEEN limits R ← ((data>limits[1])∧(data<limits[2]))/data R ← a COMMON b R ← (∨/a ∘.= b)/a x ← 2 8 7 9 ¯5 0 2 y ← 1 ¯3 10 0 8 9 1 x_btwn ← x BETWEEN 2 10 y_btwn ← y BETWEEN ¯2 9 xy_common ← x_btwn COMMON y_btwn If you know APL the above pretty much reads like the palm of your hand. Since APL is unknown to most, here's a quick explanation. R ← data BETWEEN limits Dyadic function declaration. Takes two arguments. R ← ((data>limits[1])∧(data<limits[2]))/data Let's break this up: (data>limits[1]) Takes the "data" vector and compares it to the first element in "limits", which happens to be the "low" limit. You get a binary vector as the result with a "1" anywhere the comparison is true and "0" otherwise. If "limits" is 2 10: 0 1 1 1 0 0 0 0 Now: (data<limits[2]) Does the same thing with the upper limit: 1 1 1 1 1 1 1 1 Then: 0 1 1 1 0 0 0 0 ∧ 1 1 1 1 1 1 1 1 Performs a logical AND of the two binary vectors, resulting in a new vector: 0 1 1 1 0 0 0 0 Finally: R ← 0 1 1 1 0 0 0 0/data Selects elements from the "data" vector based on the values in the binary vector and returns the result vector: 8 7 9 Anyhow, that's why I think that language is important. If I wrote this in Forth the pattern would be very different and the thought process required to understand it more convoluted. Probably true as well for assembly.
pilgrim689 14y ago
Interesting how he starts from the components and then rolls up to the main logic, whereas I would start at the main logic then drill down into the routines it uses.
bryceneal 14y ago
Didn't this guy get it wrong? I think the answer should be, right ? 8 7 9 1 0 8 1 8

214 points by erjiang 14y ago

Great post. Very relevant to this audience and very interesting research may come from it. I look forward to seeing follow ups.

columbo 14y ago

This is very interesting. Have they considered using multiple languages (functional, oo) as well as including 'garbage' languages (brainfuck) to track differences? I'd also be curious to see how it compares to a standard word problem (two trains leaving chicago etc).

synesthesiam 14y ago
Here's an interesting paper where they compare their language (Quorum), Perl, and a language with random ASCII tokens: http://ecs.victoria.ac.nz/twiki/pub/Events/PLATEAU/Program/plateau2011-stefik.pdf http://ecs.victoria.ac.nz/twiki/pub/Events/PLATEAU/Program/p... (Spoiler alert: Perl doesn't fair much better than the random language!)

akshaykarthik 14y ago

One of the things I noticed from the Eye Tracking was that this is very similar to how we were trained to work through programming problems in high school level computer science competitions. Some competition problems involve tracing through code and determining output (of usually recursive functions) but the 'proper' method we were taught is almost exactly how he describes his thought process.

afhof 14y ago

Makes me wonder if future languages will be designed to be read in serial. Perhaps that will decrease the amount of time we need to look at code to understand what it does.

kscaldef 14y ago
Probably not. I don't think that in the large that helps with readability / maintainability. Even in the small, I find languages which don't allow forward references (Ruby, sh) rather annoying. It forces relatively insignificant implementation details to be read first, rather than starting with the big picture of what the program does.
saurabh 14y ago
I would be interesting to do a similar study on some Factor code.
- evincarofautumn 14y ago
  I would expect similar results. Factor and other stack-based languages are not particularly linear, thanks to, well, factoring. Sure, data flow generally runs from left to right and top to bottom, but word and tuple definitions are just as scattered about as in any procedural language.

darrennix 14y ago

I'll be very interested in findings on the terseness of code and its effects on readability by experienced coders. For example, the ternary operator is consider by many to be an elegant solution to simple if statements but from a comprehension (and therefore bug-finding) standpoint, is it superior? Also, how does this effect change as the size of the codebase grows from a single page (as depicted in the video) to a more complex class file. value = test ? (some_value * multiple) : false_value # vs if (test) { value = some_value * multiple } else { value = false_value }

birken 14y ago
I would say it is a matter of personal preference. However, first and foremost whichever thing you prefer should be codified in your project's style guide. Consistency trumps everything else by a large margin. My personal opinion is that this is an excellent use of the ternary operator. When writing software, you want your code to be as simple and short as possible provided it is readable[1]. I find both of them to be equally readable, therefore I prefer the shorter one by a lot. We are talking about 5 lines of code vs 1, which can really add up if this appears commonly all over your codebase. One limited asset every engineer has is monitor real estate, and the more code you can fit on your screen, the better (provided all of that code is the readable variety). I also find in this particular example, the ternary could get a slight nod for readability as well. This is because with the ternary operator, you start with: value = Ok, value is getting assigned to something... then you read the ternary expression. If you are grepping through this code and looking to see what value might be set to, you get to this line and you know you have found it, then you parse the rest of it to figure out what it is being set to. With the other example, your grepping will lead you to a "value =" that is within a conditional, so now you have to look up and down and explore a little more to see what it might be set to. This is because the ternary operator can only do one small thing, whereas the if statement can do lots of things. Since you are only doing the simple thing, using the simplest possible operator to do that in some sense helps future people reading the code [2]. [1] Which might seem like a spectrum, but I think of it as much more binary. Code is either readable or it isn't. This might seem strange, but my fellow engineers and I spend quite a bit of time grading code submissions to our programming challenges, and of course the "readability" of the code is a key thing we grade for. I think we probably match up our independent up or down votes on readability at least 90% of the time. [2] If you want to frustrate experienced engineers, have them look through a bunch of code that hasn't been factored down to its simplest form. This is extremely common among inexperienced engineers who refactor code and don't delete a bunch of cruft that now exists due to code or logic changes because "who cares? the code works!", and you get stuff like this: if ($has_phone_number) { return TRUE; } else if ($has_phone_number || $has_email_address) { return TRUE; } else if (!$has_phone_number) { return FALSE; } else { return FALSE; } return FALSE; ahhhh!

tejaswiy 14y ago

Would it be possible for you to open source the eye tracking piece of this ? I'm interested in doing this for photographs or paintings ...

eholk 14y ago
It looked like a commercial eye tracker to me, so we wouldn't be able to open source it. However, a quick Google search revealed some things that might be promising.
eholk 14y ago
I forgot to add, I would also be interested in seeing this for photographs and paintings!
synesthesiam 14y ago
Do you mean the videos themselves, the software, or the hardware?
lomendil 14y ago
Eye tracker hardware is tricky and therefore usually expensive, but on the software side there are good tools. I've used VisionEgg[0] with an Eyelink II system and found it quite easy to deal with. [0] http://www.visionegg.org/ http://www.visionegg.org/
- dchichkov 14y ago
  Not that expensive. For a couple of hundred dollars you can build very nice 60-90fps IR eye tracking system. You'll need IR camera, IR tele lens (very important), couple of IR sources and open source tracking software. See http://www.gazegroup.org/develop/ http://www.gazegroup.org/develop/ There is also alternative approach. If you are just recording and don't need interactivity - all you need, is a regular camera with zoom and a small tripod. You can record on an SD card and synchronize the stream stream time in post-processing. Just flash the monitor screen couple of times at the start and the end of the recording, and play tracking alignment sequence. Here's an attempt to do just that: http://www.youtube.com/watch?v=zJqKRL9D2qY http://www.youtube.com/watch?v=zJqKRL9D2qY

akennberg 14y ago

"One of the things that stood out to me in watching the video was how much my mind seems to work like a computer." A more accurate way of thinking about this is: how much the computer works like our mind. Not surprising considering that people build things based on how we understand everything else. Whether it was done consciously or subconsciously.

eholk 14y ago
I read somewhere once that we tend to think of our mind in terms of whatever the current most complex technology is. At one point we thought of the brain as a fantastically complicated plumbing system, and later as a telephone switchboard. I wonder if in another decade or so will the idea that our brains work like computers seem as wrong as the idea that they work like plumbing systems.
- lomendil 14y ago
  I think that decade is now. Brains work nothing like computers. Well, as much as they do like plumbing systems ;)

Chris_Newton 14y ago

Program comprehension is a fascinating (and IMHO very enlightening) field. For anyone curious, I wrote up some of what I learned while exploring the research a few years ago: http://www.clarityincode.com/readability/ http://www.clarityincode.com/readability/ (I apologise for the less than stellar formatting; I haven’t updated that page for some time. I also apologise to the actual researchers I cited, if I’ve dumbed down their work too much in aiming for a non-expert audience.) For the eye tracking reported here, I wonder whether the early emphasis on the top part was a combination of trying to figure out the data flow in the between() function and then its significance to the wider program. I think it would be interesting to compare the results with a similar eye track of a program written with more emphasis on data flow rather than control flow, e.g., def between(numbers, low, high): return [n for n in numbers if low < n < high] def common(list1, list2): return [i for i in list1 if i in list2] x = [2, 8, 7, 9, -5, 0, 2] x_btwn = between(x, 2, 10) print x_btwn y = [1, -3, 10, 0, 8, 9, 1] y_btwn = between(y, -2, 9) print y_btwn xy_common = common(x, y) print xy_common It might also be interesting to compare the results with a functional programming language that expresses those ideas more concisely and/or with tools like between() and common() as part of the standard library that programmers would probably be familiar with. Final thought: How much does the absence of a clearly marked starting point (like a main() function in C) affect how a reader approaches unknown code in Python? If this had been a C program, would the reader have aimed straight for main() and then worked down from there to functions like between() and common()?

dchichkov 14y ago
I think it is very important to use code from real-life applications, rather then synthetic examples like that, or like ones presented in the original article. Otherwise the results of that research would be biased towards comprehension of meaningless chunks of code, rather that of a real thing.
- Chris_Newton 14y ago
  I agree, but isn’t this the curse of most experimental research in programming techniques? Ideally, we’d compare like-against-like using industrial scale applications, implemented by professional practitioners, controlling for everything except what we’re trying to investigate. Finding opportunities to do that in practice is rather harder, because obviously most real software development projects don’t get implemented twice by identical teams making exactly one significant change in their approach. What I thought was interesting here was that even though it is only a toy program, there were still some patterns to how the reader explored it that might suggest more general trends. As long as we understand the limitations of the experiment and don’t overgeneralise any conclusions, isn’t some data still better than random conjecture?

suyash 14y ago

This video seems incorrect to me, I read code line by line one at a time most of the time. The video seems to be showing that the eye is jumping all around.

Hemospectrum 14y ago
Your eyes jump around a lot more than you think they do, no matter what you're looking at. Your brain compensates for the effect somewhat, so it's impossible to notice the full extent of the movement on a conscious level. Some quick reading on the phenomenon: http://en.wikipedia.org/wiki/Saccade http://en.wikipedia.org/wiki/Saccade
evincarofautumn 14y ago
Consider that you can conceptualise reading code as a linear narrative even if your eyes tell a different story. Our perceptions of ourselves are often radically different from our actual behaviour. One fine example of this is in phonetics: the two /t/ sounds in “tomato” are quite different, yet we think of them as the same sound.

synesthesiam 14y ago

It's awesome to see that people are interested in my research! I've made a blog post with another video and a few more details: http://synesthesiam.com/?p=218 http://synesthesiam.com/?p=218

evincarofautumn 14y ago

“One of the things that stood out to me in watching the video was how much my mind seems to work like a computer.” One of the things that stood out to me in watching the video was how much his mind seemed emphatically not to work like a computer at all. His process gave the appearance of a network self-training for a little while, then simultaneously training and producing output. The more times an area of the program was visited, the better the training, and consequently the longer it could be retained. Notice how the results of calculations are “picked up” from the source and “dropped” almost immediately into the output, as though they’re heavy and difficult to hold on to!

route66 14y ago

From http://infoscience.epfl.ch/record/138586 http://infoscience.epfl.ch/record/138586 : ... that modern languages, such as Scala, offer advantages as human communication mediums. I describe an experiment, using an eye-tracking device, that measures the performance of code comprehension.

Raphael_Amiard 14y ago

> In programming language terms, I seem to be doing some kind of just-in-time compilation Seems a lot more like abstract interpretation to me ! Would be a lot more logical too :)

mattmanser 14y ago

That code is jarring to my eye because of the variable name, my eye tracking would keep going back to 'winners'. Why winners? What a bizarre variable name, to me anyway. Does anyone else use that? I always go with `matches`, `retVals` or `returnValues` depending on the language/IDE I'm using.

eholk 14y ago
The experiment has several variants of each program. In some cases the names make sense, and in other cases they are just random words. One of the goals is to measure how much choosing good names matters.
synesthesiam 14y ago
I went back and forth over what to name this variable. I wanted it to convey some meaning (winners are the items that passed or "won" the test), but not give away too much. Several people have commented on this, however, so I may just change the name to "matches" or something. Thanks for the suggestion :)

graue 14y ago

I'm surprised no one pointed out that the answer given in the video is wrong. The second line should be "10 0 8 1" and the third "8 9 0". The author writes "10 0 9 1" and "9". Perhaps it's not relevant to the experiment, but it seems worthy of mention at least. Following along, I was second-guessing myself because my answer didn't match...

dschiptsov 14y ago

I don't think eye tracking and do any good, because recognizing of familiar shapes and then mapping them to a familiar constructs and "structures" comes first. So, reading a code with familiar shape is one thing (that why Lisp has such emphasis on the form of an expression, and Python put that into extreme), while reading long.chains.of.unfamiliar.methods.is.another.) Then comes recognition of a familiar zones (areas) of an expression, expecting particular kind of sub-expressions here and there. Then match what you have seen with known whole things. Lets say it is a recursive process of reduction to something already known by examining shapes, forms, and details. So, shape matters. Small procedures, around ten lines matters. Naming matters, and, especially, using one-letter, non-confusing (no meaning) names for just a placeholders matters. Let say that this solved in Lambda Calculus (by a naming strategy), and then in Lisp (by shapping strategy) by accident.)

mcherm 14y ago
I find it rather amusing, given the context, that I spent a long time (more than 30 seconds) trying to figure out the match for your unbalanced parenthesis. I don't think I would have done that if it hadn't been a paragraph about Lisp: I kept going back thinking "Wait, I didn't realize I was in a parenthetical expression... does this change the interpretation of what is being said?".
- kolektiv 14y ago
  Seconded. Surprising how distracting something so trivial is given recent (unrelated) context. My mind immediately sprang to the conclusion that perhaps this sentence is partially applied - a legacy of an evening spent writing some interesting functional code for treating MIDI as observable sequences.

JoeAltmaier 14y ago

100 times more frequently I read code I already read (or wrote) before. It works differently. I page through, scanning the code, noting its "shape" as I go without actually reading any lines. Once I've found the place the problem might be, I start hand-executing (or eye-executing?) for problems. Does it really matter much how we read new code? We read many tims faster than we write; re-reading is the norm.

gtani 14y ago

Funny, i mentioned this a few days ago. I was going to do some googling on other interesting code quality/dev productivity/language metrics, but, uh, never got around to it. It seems to me that in the first few passes on-sighting or sight reading code (as climbers and pianists call it), you're looking for easy to comprehend structures, and blocking off difficult to decipher, simultaneously, so maybe a bimodal distributions at work here https://news.ycombinator.com/item?id=4926313 https://news.ycombinator.com/item?id=4926313

robomartin 14y ago

Language has to be an important factor here. If I re-write this in APL --and you know APL-- reading the solution is pretty linear: R ← data BETWEEN limits R ← ((data>limits[1])∧(data<limits[2]))/data R ← a COMMON b R ← (∨/a ∘.= b)/a x ← 2 8 7 9 ¯5 0 2 y ← 1 ¯3 10 0 8 9 1 x_btwn ← x BETWEEN 2 10 y_btwn ← y BETWEEN ¯2 9 xy_common ← x_btwn COMMON y_btwn If you know APL the above pretty much reads like the palm of your hand. Since APL is unknown to most, here's a quick explanation. R ← data BETWEEN limits Dyadic function declaration. Takes two arguments. R ← ((data>limits[1])∧(data<limits[2]))/data Let's break this up: (data>limits[1]) Takes the "data" vector and compares it to the first element in "limits", which happens to be the "low" limit. You get a binary vector as the result with a "1" anywhere the comparison is true and "0" otherwise. If "limits" is 2 10: 0 1 1 1 0 0 0 0 Now: (data<limits[2]) Does the same thing with the upper limit: 1 1 1 1 1 1 1 1 Then: 0 1 1 1 0 0 0 0 ∧ 1 1 1 1 1 1 1 1 Performs a logical AND of the two binary vectors, resulting in a new vector: 0 1 1 1 0 0 0 0 Finally: R ← 0 1 1 1 0 0 0 0/data Selects elements from the "data" vector based on the values in the binary vector and returns the result vector: 8 7 9 Anyhow, that's why I think that language is important. If I wrote this in Forth the pattern would be very different and the thought process required to understand it more convoluted. Probably true as well for assembly.

pilgrim689 14y ago

Interesting how he starts from the components and then rolls up to the main logic, whereas I would start at the main logic then drill down into the routines it uses.

bryceneal 14y ago

Didn't this guy get it wrong? I think the answer should be, right ? 8 7 9 1 0 8 1 8