7 ms·

IMO, this is blind spot in the research community right now. The programming language community doesn't care about the ergonomics of language use and is only i

by munin 9y ago

IMO, this is blind spot in the research community right now. The programming language community doesn't care about the ergonomics of language use and is only interested in theoretical evaluation of their work with rigorous proofs. The Human Computer Interaction (HCI) community should care about this, but is beset by two problems: people in PL don't take people in HCI seriously because the people in HCI don't do proofs, and, the people in HCI generally don't have the background to start tackling the mathematical principles of programming language design. This seems like a natural time for some kind of cross discipline collaboration, but the two forces above keep potential collaborators apart. PL people talk about judgements and sub-structural typing and HCI people's eyes glaze over, HCI people talk about human subjects, experimental methodology and statistics and PL people's eyes glaze over. I've advocated for the creation of a new field of study, call it Programmer Computer Interaction. No one takes it that seriously though, and the intersection of researchers that care simultaneously about things like good experimental design, statistical significance, type theory and constructive logic seems to be just me.

zzzcpan 9y ago
I don't see how good experimental design could benefit from type theory and such. There is no need for an intersection, just the first group of researchers working on programming language design.
- yorwba 9y ago
  Well thought-out type systems allow decidable inference. Inference lets you omit details about the program that are already obvious from its structure. Omitting obvious details lets you focus on the important parts. Focus helps creating better programs quicker.
vanderZwan 9y ago
There are people doing research into this, for example comparing how quickly various styles of loops are understood by beginning users (I wish I could find the talk on that right now). Like you said, it's not taken very seriously though. The problem I have with that is that it seems to be limited to verifying existing structures and solutions. That kind of quantitative research has its use, but is not really helping with exploring novel ideas in human friendly interfaces. For example, I stumbled across Céu a few years back and I have never seen concurrency and timing done in such an intuitive style before[0]. Other things that have blown my mind in the last few years include Halide's decoupling of algorithm and scheduling[1], Jonathan Edwards' experiments with schematic tables as an alternative to if-statements[2], vega-lite's grammar of interactive graphics to declaratively construct interactive plots[3], and aprt.us' hybrid graphics/text programming environment[4]. These are all novel ideas that you cannot find through quantitative measurements about which syntax is optimal. Not that the latter is without use, since it can make people shut up about pointless disagreements (although I think the better solution to ending a holy war on bracket style is to get rid of it altogether and use something one default style like elm-format[5]). And maybe we're also not looking outside of our own field enough. For example, I recently read Steven Pinker's "The Stuff of Thought". There was a chapter discussing all kinds of (human) language paradoxes and hidden rules based on how humans have different ways of thinking about aggregates, and how we use those subconsciously in our daily language. As I was reading it, it made me think "this makes so much sense of how different languages use collections differently, they're just applying different styles of intuition described here!" The funny thing about most language paradoxes is that they require very specific ways of framing a question, and that they disappear when framed differently. Which sounds a lot like how some problems are easier in one style of programming or the other. And that makes me wonder if we can't learn a lot from these branches of linguistics about how we might set up our computer languages in such a way that humans are less likely to make errors of thinking in them. [0] http://www.ceu-lang.org/ http://www.ceu-lang.org/ [1] http://halide-lang.org/ http://halide-lang.org/ [2] https://vimeo.com/177767802 https://vimeo.com/177767802 [3] https://vimeo.com/177767802 https://vimeo.com/177767802 [4] https://www.youtube.com/watch?v=i3Xack9ufYk https://www.youtube.com/watch?v=i3Xack9ufYk and http://aprt.us/ http://aprt.us/ [5] https://github.com/avh4/elm-format https://github.com/avh4/elm-format
- lgas 9y ago
  > There are people doing research into this, for example comparing how quickly various styles of loops are understood by beginning users (I wish I could find the talk on that right now). Like you said, it's not taken very seriously though. I don't know other research exists, but I wouldn't personally value any research on beginning users comprehension because you're typically a beginner in a new language for a few days to a few weeks but an experienced user for years or decades, so beginner experience is just not something I care about in the big picture. (Obviously there are reasons other people might care about it... it affects initial adoption of the language and so on, but I just don't care about them personally).
  - seanmcdirmid 9y ago
    The problem with studying humans is that you need relatively blank slates so you can have proper control groups. Experts or non beginners are hardly blank slates, so are much more difficult to study in the lab.
    - deleted 9y ago
      [deleted]
    - lgas 9y ago
      I get that, and I don't have any great ideas for solutions, but I hope we can find an effective way to study the people of interest rather than to study a group of decidedly different people and try to extrapolate the results to the original group.
      - seanmcdirmid 9y ago
        There are other ways, and they are used, but they aren't scientific enough (even if useful) to be published.
tikhonj 9y ago
My only worry with the HCI community is that they seem to over-focus on beginners and prioritizing systems that are intuitive rather than powerful. I am absolutely not worried about whether they write proofs or not! This might just be an issue of perception on my end, although it seems consistent with the HCI-conscious PL work I've seen. Personally, I have two priorities in PL design: expressiveness and aesthetics. Both of these seem perfectly suited to a design-oriented approach. But are they a good fit with the sort of work regularly done in HCI? I genuinely don't know.
- kd0amg 9y ago
  HCI's methods in particular are too expensive to test the conjectures we normally offer about our designs though. A sizeable team of experienced programmers designing, building, or maintaining a significant software product can't be paid for on anyone's research budget, so everyone uses small, short-term projects written by undergrads (or maybe MS students) instead. The results may be real enough, but they won't address our actual questions. Looking in the other direction, I would like to think that even the most theory-obsessed PL people can appreciate that "intuitive" and "powerful" can both arise from clean design, but I've also been told, "your proofs are boring" as though that's a bad thing.
  - munin 9y ago
    It might be my cynicism, but in the PL literature I find "intuitive" to be a very low calorie word. Obviously you would describe your own system as "intuitive" what are you going to do, write a paper that says "we have a complicated and difficult to understand system S that..." "intuitive" is very subjective, I think. I know lots of PL people that think that functional programming, for example, is intuitive. After many years of study and practice, I too find functional programming intuitive, but I can remember a time when I did not, but I could still program. From this I conclude that something being "intuitive" is very subjective. Expressive power, on the other hand, I think we can put an objective definition on. My favorite take of this is Felleisen "On the Expressive Power of Programming Languages." Of course, what use is something being powerful?
naasking 9y ago
> The Human Computer Interaction (HCI) community should care about this, but is beset by two problems: people in PL don't take people in HCI seriously because the people in HCI don't do proofs Well that's not true. If the HCI people actually had empirical data supporting their positions, you'd be a fool to ignore them. But what are PLT researchers to do with HCI claims that are completely unsupported, or whose supported positions are extremely limited in scope? We can't halt work on PLT to wait for them to catch up. Some collaboration on empirical studies on programming ergonomics are needed. Given how huge much money is invested in this industry, I'm surprised more effort isn't put into this.
seanmcdirmid 9y ago
Exactly. We have a working group tackling these problems: http://program-transformation.org/WGLD/ http://program-transformation.org/WGLD/ However, it seems to be drifting back to traditional SIGPLAN topics lately. The problem is the conferences: there was a concerted effort to make them more sciency and less designy in the last 15 years and....well, design no longer sells to acedmics. The HCI community has similar problems actually. You can't really equate empirical evaluation of user behavior (a topic in itself) to design, where many aspects are quite difficult to validate empirically (things like Fitts' law can evaluate simple behaviors, but don't really scale to complex behaviors).
kazinator 9y ago
I'm not sure that the article is about caring about ergonomics. There is this: > I also believe that interesting programming language research is not about finding a keyword that will make writing for loops 3x easier If we go out on a limb here a bit, "a keyword that will make writing for loops easier" seems to be a metaphor for everyday ergonomic issues in coding. (I don't have a hypothesis about what the article is about, though).

by munin 9y ago

zzzcpan 9y ago

I don't see how good experimental design could benefit from type theory and such. There is no need for an intersection, just the first group of researchers working on programming language design.

yorwba 9y ago
Well thought-out type systems allow decidable inference. Inference lets you omit details about the program that are already obvious from its structure. Omitting obvious details lets you focus on the important parts. Focus helps creating better programs quicker.

vanderZwan 9y ago

There are people doing research into this, for example comparing how quickly various styles of loops are understood by beginning users (I wish I could find the talk on that right now). Like you said, it's not taken very seriously though. The problem I have with that is that it seems to be limited to verifying existing structures and solutions. That kind of quantitative research has its use, but is not really helping with exploring novel ideas in human friendly interfaces. For example, I stumbled across Céu a few years back and I have never seen concurrency and timing done in such an intuitive style before[0]. Other things that have blown my mind in the last few years include Halide's decoupling of algorithm and scheduling[1], Jonathan Edwards' experiments with schematic tables as an alternative to if-statements[2], vega-lite's grammar of interactive graphics to declaratively construct interactive plots[3], and aprt.us' hybrid graphics/text programming environment[4]. These are all novel ideas that you cannot find through quantitative measurements about which syntax is optimal. Not that the latter is without use, since it can make people shut up about pointless disagreements (although I think the better solution to ending a holy war on bracket style is to get rid of it altogether and use something one default style like elm-format[5]). And maybe we're also not looking outside of our own field enough. For example, I recently read Steven Pinker's "The Stuff of Thought". There was a chapter discussing all kinds of (human) language paradoxes and hidden rules based on how humans have different ways of thinking about aggregates, and how we use those subconsciously in our daily language. As I was reading it, it made me think "this makes so much sense of how different languages use collections differently, they're just applying different styles of intuition described here!" The funny thing about most language paradoxes is that they require very specific ways of framing a question, and that they disappear when framed differently. Which sounds a lot like how some problems are easier in one style of programming or the other. And that makes me wonder if we can't learn a lot from these branches of linguistics about how we might set up our computer languages in such a way that humans are less likely to make errors of thinking in them. [0] http://www.ceu-lang.org/ http://www.ceu-lang.org/ [1] http://halide-lang.org/ http://halide-lang.org/ [2] https://vimeo.com/177767802 https://vimeo.com/177767802 [3] https://vimeo.com/177767802 https://vimeo.com/177767802 [4] https://www.youtube.com/watch?v=i3Xack9ufYk https://www.youtube.com/watch?v=i3Xack9ufYk and http://aprt.us/ http://aprt.us/ [5] https://github.com/avh4/elm-format https://github.com/avh4/elm-format

lgas 9y ago
> There are people doing research into this, for example comparing how quickly various styles of loops are understood by beginning users (I wish I could find the talk on that right now). Like you said, it's not taken very seriously though. I don't know other research exists, but I wouldn't personally value any research on beginning users comprehension because you're typically a beginner in a new language for a few days to a few weeks but an experienced user for years or decades, so beginner experience is just not something I care about in the big picture. (Obviously there are reasons other people might care about it... it affects initial adoption of the language and so on, but I just don't care about them personally).
- seanmcdirmid 9y ago
  The problem with studying humans is that you need relatively blank slates so you can have proper control groups. Experts or non beginners are hardly blank slates, so are much more difficult to study in the lab.
  - deleted 9y ago
    [deleted]
  - lgas 9y ago
    I get that, and I don't have any great ideas for solutions, but I hope we can find an effective way to study the people of interest rather than to study a group of decidedly different people and try to extrapolate the results to the original group.
    - seanmcdirmid 9y ago
      There are other ways, and they are used, but they aren't scientific enough (even if useful) to be published.

tikhonj 9y ago

My only worry with the HCI community is that they seem to over-focus on beginners and prioritizing systems that are intuitive rather than powerful. I am absolutely not worried about whether they write proofs or not! This might just be an issue of perception on my end, although it seems consistent with the HCI-conscious PL work I've seen. Personally, I have two priorities in PL design: expressiveness and aesthetics. Both of these seem perfectly suited to a design-oriented approach. But are they a good fit with the sort of work regularly done in HCI? I genuinely don't know.

kd0amg 9y ago
HCI's methods in particular are too expensive to test the conjectures we normally offer about our designs though. A sizeable team of experienced programmers designing, building, or maintaining a significant software product can't be paid for on anyone's research budget, so everyone uses small, short-term projects written by undergrads (or maybe MS students) instead. The results may be real enough, but they won't address our actual questions. Looking in the other direction, I would like to think that even the most theory-obsessed PL people can appreciate that "intuitive" and "powerful" can both arise from clean design, but I've also been told, "your proofs are boring" as though that's a bad thing.
- munin 9y ago
  It might be my cynicism, but in the PL literature I find "intuitive" to be a very low calorie word. Obviously you would describe your own system as "intuitive" what are you going to do, write a paper that says "we have a complicated and difficult to understand system S that..." "intuitive" is very subjective, I think. I know lots of PL people that think that functional programming, for example, is intuitive. After many years of study and practice, I too find functional programming intuitive, but I can remember a time when I did not, but I could still program. From this I conclude that something being "intuitive" is very subjective. Expressive power, on the other hand, I think we can put an objective definition on. My favorite take of this is Felleisen "On the Expressive Power of Programming Languages." Of course, what use is something being powerful?

naasking 9y ago

> The Human Computer Interaction (HCI) community should care about this, but is beset by two problems: people in PL don't take people in HCI seriously because the people in HCI don't do proofs Well that's not true. If the HCI people actually had empirical data supporting their positions, you'd be a fool to ignore them. But what are PLT researchers to do with HCI claims that are completely unsupported, or whose supported positions are extremely limited in scope? We can't halt work on PLT to wait for them to catch up. Some collaboration on empirical studies on programming ergonomics are needed. Given how huge much money is invested in this industry, I'm surprised more effort isn't put into this.

seanmcdirmid 9y ago

Exactly. We have a working group tackling these problems: http://program-transformation.org/WGLD/ http://program-transformation.org/WGLD/ However, it seems to be drifting back to traditional SIGPLAN topics lately. The problem is the conferences: there was a concerted effort to make them more sciency and less designy in the last 15 years and....well, design no longer sells to acedmics. The HCI community has similar problems actually. You can't really equate empirical evaluation of user behavior (a topic in itself) to design, where many aspects are quite difficult to validate empirically (things like Fitts' law can evaluate simple behaviors, but don't really scale to complex behaviors).

kazinator 9y ago

I'm not sure that the article is about caring about ergonomics. There is this: > I also believe that interesting programming language research is not about finding a keyword that will make writing for loops 3x easier If we go out on a limb here a bit, "a keyword that will make writing for loops easier" seems to be a metaphor for everyday ergonomic issues in coding. (I don't have a hypothesis about what the article is about, though).