4 ms·

You would probably see a bunch of blog posts about 'how we saved 3000% of our SRE time by switching from Go to NodeJS'. Have you seen many of those? Aside fro

You would probably see a bunch of blog posts about 'how we saved 3000% of our SRE time by switching from Go to NodeJS'. Have you seen many of those? Aside from this it's probably quite difficult to measure! Sure there are a bunch of metrics you can capture, but unless you're really doing a randomised controlled trial, you will have an absolute dog's breakfast of confounded observational data - good luck getting much useful out of that.

CharlieDigital 2y ago
Hey, it would make for an excellent doctoral thesis for anyone in a CS program interested in programming languages and productivity...and a great gateway to get hired by Google! I don't think it would be hard to measure at all. Randomize some CS students into Go and non-Go and see which teams produce projects with lower rates of defect and faster time to submission. Try it with teams in enterprises; I'm 100% certain that this kind of statistical evidence is meaningful to some CTO/CIO somewhere. Prove to the entire world that everyone should be using Go because it will lead to cost savings not just in time to market, but also less downtime and defects. The financial benefits would be enormous! Possibly billions if not trillions of dollars of productivity gained, system downtime avoided, and runtime defects eliminated ("what runtime defects?") because of the inherent superiority of: varFoo, err := GetFoo() if err != nil { return err } Billions!
- MattPalmer1086 2y ago
  Errors do not only happen because of code defects, they are often a response to bad data. And even if there were fewer defects, that could be because of other features of Go, not the error handling. So that doesn't seem like a good metric to determine if Go's error handling is "better" (in some dimension) than other techniques.
  - CharlieDigital 2y ago
    Bad data isn't unique to Go; let's assume bad data is an external constant. If Go is a superior programming language -- error handling or not -- it should be easy to find the statistical evidence. Otherwise, you just prefer green crayons; please, by all means, go enjoy your green crayons. Go's error handling isn't any better or worse; it's just objectively way more verbose. With what benefit? If you wish to claim one beyond your subjective vibes, please by all means prove it with evidence.
    - MattPalmer1086 2y ago
      The topic was not determining which language has fewer defects. It was specifically about whether the error handling aspect is better. I'm not claiming any benefit by the way of Go's error handling. I've never even used Go. Just pointing out that measuring defects is not a good way to determine whether it is or isn't.
      - CharlieDigital 2y ago
        > It was specifically about whether the error handling aspect is better. If it's better, how? Can you prove it? Does it lead to lower defect rates? Lower time to debug? Less runtime errors? How is it better other than your preference of green crayons?
        MattPalmer1086 2y ago
        As I said before, I've never used Go. I have no opinions about its error handling, or green crayons for that matter. Measuring defect rates against another language will not tell you whether the error handling is better or worse than the other language either.
- akdor1154 2y ago
  > Randomize some CS students into Go and non-Go and see which teams produce projects with lower rates of defect and faster time to submission. Using students still has problems - have they coded before? Have they worked in a team with industry-style review processes before? The population you're sampling is quite different to experienced software engineers. It's not quite "...in mice" levels of qualification needed, but it's fundamentally the same problem. > Try it with teams in enterprises; I'm 100% certain that this kind of statistical evidence is meaningful to some CTO/CIO somewhere Difficult again - how do you do a fair comparison? You'd need to be doing fair randomization - in particular as soon as anyone gets any say in which team uses what language for which project, you have a good chance of stuffing things up. Is one of the projects more important to the business, so is likely to get a "better" team allocated to it? Does one of the teams have a language preference? Does the business have a language preference? Is one of the projects well-suited for Go (say a web server) and the other terrible? (say a desktop app) One approach might be to get two teams to build the same thing and one gets thrown away, but what CTO is going to pay for that once? Let alone finding enough to pay for a big enough sample size to make anything of. I'm not trying to get into fights about languages, I'm trying to make the point that actually making an objective measurement like you suggest is really, really difficult.
  - CharlieDigital 2y ago
    > Using students still has problems - have they coded before? Why would it matter if we're doing randomized experiments with students placed in groups of different languages? That's how randomized experiments work: it doesn't matter if they've coded before because the selection process will randomize a statistically relevant number of students.

CharlieDigital 2y ago

Hey, it would make for an excellent doctoral thesis for anyone in a CS program interested in programming languages and productivity...and a great gateway to get hired by Google! I don't think it would be hard to measure at all. Randomize some CS students into Go and non-Go and see which teams produce projects with lower rates of defect and faster time to submission. Try it with teams in enterprises; I'm 100% certain that this kind of statistical evidence is meaningful to some CTO/CIO somewhere. Prove to the entire world that everyone should be using Go because it will lead to cost savings not just in time to market, but also less downtime and defects. The financial benefits would be enormous! Possibly billions if not trillions of dollars of productivity gained, system downtime avoided, and runtime defects eliminated ("what runtime defects?") because of the inherent superiority of: varFoo, err := GetFoo() if err != nil { return err } Billions!

MattPalmer1086 2y ago
Errors do not only happen because of code defects, they are often a response to bad data. And even if there were fewer defects, that could be because of other features of Go, not the error handling. So that doesn't seem like a good metric to determine if Go's error handling is "better" (in some dimension) than other techniques.
- CharlieDigital 2y ago
  Bad data isn't unique to Go; let's assume bad data is an external constant. If Go is a superior programming language -- error handling or not -- it should be easy to find the statistical evidence. Otherwise, you just prefer green crayons; please, by all means, go enjoy your green crayons. Go's error handling isn't any better or worse; it's just objectively way more verbose. With what benefit? If you wish to claim one beyond your subjective vibes, please by all means prove it with evidence.
  - MattPalmer1086 2y ago
    The topic was not determining which language has fewer defects. It was specifically about whether the error handling aspect is better. I'm not claiming any benefit by the way of Go's error handling. I've never even used Go. Just pointing out that measuring defects is not a good way to determine whether it is or isn't.
    - CharlieDigital 2y ago
      > It was specifically about whether the error handling aspect is better. If it's better, how? Can you prove it? Does it lead to lower defect rates? Lower time to debug? Less runtime errors? How is it better other than your preference of green crayons?
      - MattPalmer1086 2y ago
        As I said before, I've never used Go. I have no opinions about its error handling, or green crayons for that matter. Measuring defect rates against another language will not tell you whether the error handling is better or worse than the other language either.
akdor1154 2y ago
> Randomize some CS students into Go and non-Go and see which teams produce projects with lower rates of defect and faster time to submission. Using students still has problems - have they coded before? Have they worked in a team with industry-style review processes before? The population you're sampling is quite different to experienced software engineers. It's not quite "...in mice" levels of qualification needed, but it's fundamentally the same problem. > Try it with teams in enterprises; I'm 100% certain that this kind of statistical evidence is meaningful to some CTO/CIO somewhere Difficult again - how do you do a fair comparison? You'd need to be doing fair randomization - in particular as soon as anyone gets any say in which team uses what language for which project, you have a good chance of stuffing things up. Is one of the projects more important to the business, so is likely to get a "better" team allocated to it? Does one of the teams have a language preference? Does the business have a language preference? Is one of the projects well-suited for Go (say a web server) and the other terrible? (say a desktop app) One approach might be to get two teams to build the same thing and one gets thrown away, but what CTO is going to pay for that once? Let alone finding enough to pay for a big enough sample size to make anything of. I'm not trying to get into fights about languages, I'm trying to make the point that actually making an objective measurement like you suggest is really, really difficult.
- CharlieDigital 2y ago
  > Using students still has problems - have they coded before? Why would it matter if we're doing randomized experiments with students placed in groups of different languages? That's how randomized experiments work: it doesn't matter if they've coded before because the selection process will randomize a statistically relevant number of students.