5 ms·

We need to have more research like this. Programmers spend a lot of time arguing about features, syntax, paradigm, methodology, etc., but no one really has any

We need to have more research like this. Programmers spend a lot of time arguing about features, syntax, paradigm, methodology, etc., but no one really has any scientific proof. I laud the researchers who are coming up with new paradigms and extending current paradigms, but we really need researchers going behind and finding what's beneficial and what's not in terms of human factors.

macspoofing 14y ago
>We need to have more research like this. There is a lot of research done in this and other areas of Software Engineering. Problem is that it's usually locked away behind a huge paywall. If you're not affiliated with some university, you essentially have no access to it.
- denzil_correa 14y ago
  > Problem (as with all academic papers) is that, it's usually locked away behind a huge paywall. This may be true for research in other domains but as a researcher in Computer Science I can assure you that the number of academic papers locked behind a paywall is negligible in CS. Only once was did I find a paper not available and I promptly received a copy by sending an e-mail to the authors of the paper.
  - keefe 14y ago
    almost everything I need I find in CiteSeer or ACM Digital Library (which does have a very small pay wall)
    - denzil_correa 14y ago
      Out of curiosity - which area do you work in? Can you also post a couple of examples (from your area) of academic research papers not available for free download?
      - keefe 14y ago
        I've been out in the wilds of industry for a while now, so it's been a while since I needed to pay for something professionally... back in the halcyon days of academia, I worked in computational medicine and had (still have) interest in lingustics/neuroecon/neurosci and I still run into walls there occasionally. I can't cite specific examples for you, if I can't find it somebody I know can get me a copy.
        denzil_correa 14y ago
        > I worked in computational medicine and had (still have) interest in lingustics/neuroecon/neurosci and I still run into walls there occasionally. Like I said earlier areas like computational Medicine, Biology and other such areas run into pay walls. CS research areas like theory, data mining, NLP etc. will rarely run into such problems.
    - macspoofing 14y ago
      ACM charges $200/year or $15/paper. A paywall is a paywall.
  - jweese 14y ago
    My experience is the same. I work in NLP, and I've found that at least everything administered by ACL (probably our largest professional organization) is pretty much always available.
    - arnsholt 14y ago
      I love that about ACL. Not only is open-access good and nice and all that, but it also means I don't have to screw around with VPNs and such when I need to get an article while I'm at home.
    - denzil_correa 14y ago
      The only thing I do not like about ACL is the fact that is not indexed in DBLP. I have a paper in one of the ACL affiliated conference and it does not show up. I have since then shifted from core NLP and it doesn't really matter but it would be nice to be consistent with other indexing mechanisms.
- groovy2shoes 14y ago
  Another problem is that the research hasn't been collected into a manager-friendly tome like the Gang of Four book. The software industry is overflowing with snake oil, and we need to replace it with science if we want to see this turn into a real engineering discipline. I'm tired of bikeshedding and holy wars -- show me the science!
  - zwegner 14y ago
    If it involves studying human behaviors, it's probably not going to be science. I would prefer the holy wars over some manager telling me it's been "proven" that one side is correct.
    - groovy2shoes 14y ago
      Good point. The thought of a manager saying that sent a chill down my spine.
  - mikebike 14y ago
    There's "Making Software" by Oram and Wilson: http://shop.oreilly.com/product/9780596808303.do http://shop.oreilly.com/product/9780596808303.do
ibdknox 14y ago
I agree, and I was astounded when I found out they had never done an end to end study of how people use Visual Studio while I was at MSFT. It surprises me how little we really know about the human aspects of our own field. The practical/pragmatic side of understanding how we really program, what's hard, where we stumble, what we do easily, etc is something that we're trying to address with Light Table. LT happens to be the perfect playground to do this sort of experimentation, so at the very least we'll be making it easier for researchers already in the space. But as we get further along and have a few more hands to spare, we'll start picking off research questions ourselves. The more the merrier, right? :)
- Finster 14y ago
  > I agree, and I was astounded when I found out they had never done an end to end study of how people use Visual Studio while I was at MSFT. As someone who uses Visual Studio everyday, I am not surprised by this at all. Don't get me wrong, Visual Studio is one of the best IDE's out there, but sometimes it's inspite of itself. Like removing functionality from the Text Explorer in VS 2012, which they have finally been patching back in. Dumb, dumb, dumb. If they HAD end-to-end user testing, they never would've remove the functionality in the first place.
  - ibdknox 14y ago
    FWIW, while I was there I ran the first end-to-end usability study. It was... enlightening :)
    - frozenport 14y ago
      Wow, what did you learn?
synesthesiam 14y ago
Thanks! We'll be following up soon with an analysis of our eye-tracking data. I'm getting permission to post the data, so I hope to have the complete data set and videos up when the paper comes out.
Darmani 14y ago
There is tons of research like this. There are multiple conferences dedicated to research like this (e.g.: http://www.program-comprehension.org/ http://www.program-comprehension.org/ ). You just need to go out and find it instead of waiting for it to appear on Hacker News.
- groovy2shoes 14y ago
  It's cool to see this research, but it really needs to become more accessible and widely applied. I'll start digging in, thanks for the pointer.
greggman 14y ago
Research is great except when its results are misinterpreted, its tests are poorly designed, etc. I could easily redesign the horizontal whitespace test to show that horizontal whitespace matters. Compare: spot the error class FruitCollection { pubic: FruitCollection() : num_apple(0), num_apricot(0), num_avocado(0), num_banana(0), num_breadfruit(0), num_bilberry(0), num_blackberry(0), num_blackcurrant(0), num_blueberry(0), num_currant(0), num_cherry(0), num_cherimoya(0), num_clementine(0), num_cloudberry(0), num_coconut(0), num_damson(0), num_date(0), num_dragonfruit(0), num_durian(0), num_elderberry(0), num_feijoa(0), num_fig(0), num_gooseberry(0), num_grape(0), num_grapefruit(0), num_guava(0), num_huckleberry(0), num_honeydew(0) num_jackfruit(0), num_jettamelon(0), num_jambul(0), num_jujube(0), num_kiwifruit(0), num_kumquat(0), num_legume(0), num_lemon(0), num_lime(0), num_loquat(0), num_lychee(0), num_mandarine(0), num_mango(0), num_melon(0) { }; }; Now spot the error here class FruitCollection { public: FruitCollection() : num_apple(0) , num_apricot(0) , num_avocado(0) , num_banana(0) , num_breadfruit(0) , num_bilberry(0) , num_blackberry(0) , num_blackcurrant(0) , num_blueberry(0) , num_currant(0) , num_cherry(0) , num_cherimoya(0) , num_clementine(0) , num_cloudberry(0) , num_coconut(0) , num_damson(0) , num_date(0) , num_dragonfruit(0) , num_durian(0) , num_elderberry(0) , num_feijoa(0) , num_fig(0) , num_gooseberry(0) , num_grape(0) , num_grapefruit(0) , num_guava(0) , num_huckleberry(0) , num_honeydew(0) num_jackfruit(0) , num_jettamelon(0) , num_jambul(0) , num_jujube(0) , num_kiwifruit(0) , num_kumquat(0) , num_legume(0) , num_lemon(0) , num_lime(0) , num_loquat(0) , num_lychee(0) , num_mandarine(0) , num_mango(0) , num_melon(0) { }; }; Similarly the rectangle with tuples. I'm only guessing on this one but I suspect if you change it to vectors then a = Vector(1, 2, 3, 4) b = Vector(3, 4, 5, 6) c = mul(a, b); Is far more understandable than a = Vector(1, 2, 3, 4) b = Vector(3, 4, 5, 6) c = mul(a.x, a.y, a.z, a.w, b.x, b.y, b.z, b.w); And that the more complex the math the worse it would get. Even for rectangles a larger example that did various intersections and unions I suspect the results would change.

macspoofing 14y ago

>We need to have more research like this. There is a lot of research done in this and other areas of Software Engineering. Problem is that it's usually locked away behind a huge paywall. If you're not affiliated with some university, you essentially have no access to it.

denzil_correa 14y ago
> Problem (as with all academic papers) is that, it's usually locked away behind a huge paywall. This may be true for research in other domains but as a researcher in Computer Science I can assure you that the number of academic papers locked behind a paywall is negligible in CS. Only once was did I find a paper not available and I promptly received a copy by sending an e-mail to the authors of the paper.
- keefe 14y ago
  almost everything I need I find in CiteSeer or ACM Digital Library (which does have a very small pay wall)
  - denzil_correa 14y ago
    Out of curiosity - which area do you work in? Can you also post a couple of examples (from your area) of academic research papers not available for free download?
    - keefe 14y ago
      I've been out in the wilds of industry for a while now, so it's been a while since I needed to pay for something professionally... back in the halcyon days of academia, I worked in computational medicine and had (still have) interest in lingustics/neuroecon/neurosci and I still run into walls there occasionally. I can't cite specific examples for you, if I can't find it somebody I know can get me a copy.
      - denzil_correa 14y ago
        > I worked in computational medicine and had (still have) interest in lingustics/neuroecon/neurosci and I still run into walls there occasionally. Like I said earlier areas like computational Medicine, Biology and other such areas run into pay walls. CS research areas like theory, data mining, NLP etc. will rarely run into such problems.
  - macspoofing 14y ago
    ACM charges $200/year or $15/paper. A paywall is a paywall.
- jweese 14y ago
  My experience is the same. I work in NLP, and I've found that at least everything administered by ACL (probably our largest professional organization) is pretty much always available.
  - arnsholt 14y ago
    I love that about ACL. Not only is open-access good and nice and all that, but it also means I don't have to screw around with VPNs and such when I need to get an article while I'm at home.
  - denzil_correa 14y ago
    The only thing I do not like about ACL is the fact that is not indexed in DBLP. I have a paper in one of the ACL affiliated conference and it does not show up. I have since then shifted from core NLP and it doesn't really matter but it would be nice to be consistent with other indexing mechanisms.
groovy2shoes 14y ago
Another problem is that the research hasn't been collected into a manager-friendly tome like the Gang of Four book. The software industry is overflowing with snake oil, and we need to replace it with science if we want to see this turn into a real engineering discipline. I'm tired of bikeshedding and holy wars -- show me the science!
- zwegner 14y ago
  If it involves studying human behaviors, it's probably not going to be science. I would prefer the holy wars over some manager telling me it's been "proven" that one side is correct.
  - groovy2shoes 14y ago
    Good point. The thought of a manager saying that sent a chill down my spine.
- mikebike 14y ago
  There's "Making Software" by Oram and Wilson: http://shop.oreilly.com/product/9780596808303.do http://shop.oreilly.com/product/9780596808303.do

ibdknox 14y ago

I agree, and I was astounded when I found out they had never done an end to end study of how people use Visual Studio while I was at MSFT. It surprises me how little we really know about the human aspects of our own field. The practical/pragmatic side of understanding how we really program, what's hard, where we stumble, what we do easily, etc is something that we're trying to address with Light Table. LT happens to be the perfect playground to do this sort of experimentation, so at the very least we'll be making it easier for researchers already in the space. But as we get further along and have a few more hands to spare, we'll start picking off research questions ourselves. The more the merrier, right? :)

Finster 14y ago
> I agree, and I was astounded when I found out they had never done an end to end study of how people use Visual Studio while I was at MSFT. As someone who uses Visual Studio everyday, I am not surprised by this at all. Don't get me wrong, Visual Studio is one of the best IDE's out there, but sometimes it's inspite of itself. Like removing functionality from the Text Explorer in VS 2012, which they have finally been patching back in. Dumb, dumb, dumb. If they HAD end-to-end user testing, they never would've remove the functionality in the first place.
- ibdknox 14y ago
  FWIW, while I was there I ran the first end-to-end usability study. It was... enlightening :)
  - frozenport 14y ago
    Wow, what did you learn?

synesthesiam 14y ago

Thanks! We'll be following up soon with an analysis of our eye-tracking data. I'm getting permission to post the data, so I hope to have the complete data set and videos up when the paper comes out.

Darmani 14y ago

There is tons of research like this. There are multiple conferences dedicated to research like this (e.g.: http://www.program-comprehension.org/ http://www.program-comprehension.org/ ). You just need to go out and find it instead of waiting for it to appear on Hacker News.

groovy2shoes 14y ago
It's cool to see this research, but it really needs to become more accessible and widely applied. I'll start digging in, thanks for the pointer.

greggman 14y ago

Research is great except when its results are misinterpreted, its tests are poorly designed, etc. I could easily redesign the horizontal whitespace test to show that horizontal whitespace matters. Compare: spot the error class FruitCollection { pubic: FruitCollection() : num_apple(0), num_apricot(0), num_avocado(0), num_banana(0), num_breadfruit(0), num_bilberry(0), num_blackberry(0), num_blackcurrant(0), num_blueberry(0), num_currant(0), num_cherry(0), num_cherimoya(0), num_clementine(0), num_cloudberry(0), num_coconut(0), num_damson(0), num_date(0), num_dragonfruit(0), num_durian(0), num_elderberry(0), num_feijoa(0), num_fig(0), num_gooseberry(0), num_grape(0), num_grapefruit(0), num_guava(0), num_huckleberry(0), num_honeydew(0) num_jackfruit(0), num_jettamelon(0), num_jambul(0), num_jujube(0), num_kiwifruit(0), num_kumquat(0), num_legume(0), num_lemon(0), num_lime(0), num_loquat(0), num_lychee(0), num_mandarine(0), num_mango(0), num_melon(0) { }; }; Now spot the error here class FruitCollection { public: FruitCollection() : num_apple(0) , num_apricot(0) , num_avocado(0) , num_banana(0) , num_breadfruit(0) , num_bilberry(0) , num_blackberry(0) , num_blackcurrant(0) , num_blueberry(0) , num_currant(0) , num_cherry(0) , num_cherimoya(0) , num_clementine(0) , num_cloudberry(0) , num_coconut(0) , num_damson(0) , num_date(0) , num_dragonfruit(0) , num_durian(0) , num_elderberry(0) , num_feijoa(0) , num_fig(0) , num_gooseberry(0) , num_grape(0) , num_grapefruit(0) , num_guava(0) , num_huckleberry(0) , num_honeydew(0) num_jackfruit(0) , num_jettamelon(0) , num_jambul(0) , num_jujube(0) , num_kiwifruit(0) , num_kumquat(0) , num_legume(0) , num_lemon(0) , num_lime(0) , num_loquat(0) , num_lychee(0) , num_mandarine(0) , num_mango(0) , num_melon(0) { }; }; Similarly the rectangle with tuples. I'm only guessing on this one but I suspect if you change it to vectors then a = Vector(1, 2, 3, 4) b = Vector(3, 4, 5, 6) c = mul(a, b); Is far more understandable than a = Vector(1, 2, 3, 4) b = Vector(3, 4, 5, 6) c = mul(a.x, a.y, a.z, a.w, b.x, b.y, b.z, b.w); And that the more complex the math the worse it would get. Even for rectangles a larger example that did various intersections and unions I suspect the results would change.