3 ms·

Goodness is fundamentally a subjective attribute. Trying to measure it objectively is attempting alchemy. You cannot make the subjective out of the objective

by Dove 2y ago

Goodness is fundamentally a subjective attribute. Trying to measure it objectively is attempting alchemy. You cannot make the subjective out of the objective - you will always have smuggled subjectivity in somehow. Suppose I say that good code should have the lowest number of curly braces, or the lowest number of subroutines, or the flattest or deepest object hierarchy. A measure being objective doesn't make it good. So why is test coverage good? In fact, every single one of the metrics proposed in the parent post is something I have seen gamed to the point of maladaptivness. * execution performance time - So frequently overly focused on that you can find google hits for "Premature optimization is the root of all evil." I have seen developers spend hours or days saving (sometimes imagined!) run time, where all the time they saved over the life of the product wouldn't add up to the time spent. Extreme pursuit of performance leads to code that is hard to work on - who cares about those milliseconds when critical new features take months to write, or are even impossible? * build time - It is very easy to diminish build time by destroying critical architecture. I have done it myself. :) * regression frequency - Insisting on only making very safe changes is how you wind up spending six weeks lobbying change control boards for one line of code. *defect quantity - In environments that actually track this, people merge issues into a single "fix what's wrong" ticket, degrading the utility of the ticket system. Defect granularity is not actually obvious! * code size - Obfuscated X contest entries are often very compact, and people who obsess on saving lines and characters can wind up leaning towards that style. * dependency quantity - Leads to attempts to homebrew encryption. * test automation coverage - Automated testing ossifies design and architecture, which can paralyze, e.g., an experimental prototype. Full coverage is also costly - time and energy spent maintaining pointless tests can come at the expense of mitigating more realistic risks. I realize I depart from prevailing wisdom in this, but there are times and places when automated testing is simply inappropriate. * test automation execution duration - Sometimes the right way to write a test is slow. I'm not disagreeing that these are generally good things to strive for. They are! I'm saying that if you think these things define goodness, each one can lead you to a cursed place. (I hasten to add that there are times when a metric really does define goodness - sometimes you need speed or reliability or whatever, at any cost. Recognizing that circumstance and its limits - "any cost" does not generally mean any cost - is subjective.) Goodness is subjective, and while objective measures can help you assess it, such measures cannot define it - when and how you use which measures, and when you think they've gone off the rails, is itself a judgement call. I once inherited a system that was both essential for business operations and a thorn in everyone's side. The guy I inherited it from (and the guy he inherited it from) had taken over a year to learn how to use it. I set about reorganizing, rewriting, documenting, abstracting - all those soft changes in pursuit of clarity and obviousness. They aren't objective, but they do pay off: when I handed the system off to the next guy (and three more after him!), he was off and building on it in a day. That was how I knew I had succeeded! When doing that sort of thing, you do always wonder if what you're writing is clearer for everybody or just you. But surely even the most hardheaded bean counter can see the value of training developers in a day rather than a year. That's good. :) Goodness is contextual and subjective. I can agree that your goals are generally right, and I can point to circumstances where they're overemphasized or even outright wrong. Sometimes, when the sky is falling, a nasty little bash script that meets none of the usual criteria for "quality" is the best possible thing. There are people who use subjectivity as a haven for vanity, and build mountains of pointless code in pursuit of some idea of goodness that serves no practical purpose or is even harmful. It is important that we retain our on ability to criticize on subjective grounds, precisely to counter that sort of activity - because you will find it in the land of the objective advocates as well, building mountains of metrics that don't serve any practical purpose either. To recognize a bad abstraction and a bad metric is the same skill, and requires the same confidence in your own good judgement. Objectivity is no refuge from the necessity of good taste.

austin-cheney 2y ago
> A measure being objective doesn't make it good That completely misses the point. It’s not about what’s good. It’s not even about what’s better. It’s only about how much better, the distance between theirs and ours. Better is objective only when like aspects of competing items are compared within accepted bounds of precision using evidence. The interesting thing about measuring stuff isn’t that people are otherwise entirely wrong in their assumptions more than 80% of the time, but that they are typically wrong by one or more orders of magnitude.
- Dove 2y ago
  >> A measure being objective doesn't make it good >That completely misses the point. It’s not about what’s good The loss of context here makes me wonder if I am talking to an AI. The comment you originally replied to was, We know what a good batting looks like but we still can’t say what good code is in any reasonably objective way. You replied with, "Sure we do" and proposed a list of metrics. Are we tracking? The original comment claimed that we do not know how to objectively measure goodness in code. You are (apparently) claiming to know how to do it. I am claiming it is impossible even in principle. In this context, I find your response ("It’s not about what’s good", and the claim that "better" is easier fo measure) bizarre and nonsensical. Like an AI, you seem to have lost track of what we are talking about. We are talking about whether we can objectively measure code being good. It is exactly "about what's good". Of course it is possible to measure things about code, but equating those measures to goodness relies on artificially constrained circumstances -- like a code golf contest or a PO declaring test coverage a metric to maximize. Athletes find themselves in such constrained circumstances all the time because it is a pursuit dominated by competition and games! It is most obvious in sports like sprinting or powerlifting, which are analogous to something like code golf, but even sports like basketball in which "goodness" is harder to define are heavily artificially constrained such that goodness in a player is about maximizing an objective measure - team score. This might be analogous to a programmer who sees his mission in terms of ticket closed per week. By contrast, programmers in general are usually working in a context in which the quality of what they produce is measured by a lot of complex impacts - on users, on business, on other programmers. Some of what code needs to accomplish - conceptualizing a problem well, communicating clearly - is inherently subjective, having to do with how it is received by another mind. Programmers (myself included) are generally focusing on these sorts of characteristics when talking about code in isolation, partly because we feel the impacts to ourselves most keenly. But I would contend that there is something deeper and less obvious here - that maximizing this subjective goodness profoundly improves the situation in more objective areas. Well architected code, well communicated code, clear code is resistant to defects in a way that mere test coverage can't accomplish. This is not obvious, but it is deep wisdom arising from experience, and is part of what drives programmers to emphasize the ineffable in their understanding of goodness. In fact, I was originally going to draw a parallel between being a good basketball player, maximizing team score, and a programmer maximizing business revenue. But I stopped myself, because this is an element of the deep wisdom of good code: focusing on ineffable, subjective excellence is profoundly positive for revenue. It's not something you can measure directly so much as it defines the circumstances the business finds itself in. This observation isn't unique to me - here's a pg essay making a similar point: https://paulgraham.com/avg.html https://paulgraham.com/avg.html Programmers talking about code being good in this sense may only be building sandcastles in the air - but they also may not be. You must possess the wisdom yourself to tell the difference. But there is certainly more to it than than selfish vanity. But even leaving that aside, because code must meet a broad array of conflicting demands, optimizing among those depends both on the circumstance and the values held by the people in it. Hence, goodness in code will always have a subjective element, and (in the athletic sphere) is most like saying you are "healthy and fit" or "your best self". You can certainly bring measurements to bear, and we are certainly talking about something real, but there is an inescapable subjective dependence on the value judgement of the judge. This actually touches on a broader philosophical debate: Are value judgements mere meaningless personal preferences, or are they (often imperfect) attempts to articulate something real? In code, and in life, I believe the subjective is pursuing the real, and moreover that anyone who thinks it's worth arguing about intuitively agrees with this assessment. By contrast, the view that only the objective is real, popular as it has been for the last couple of centuries, and attractive as its promises are, has been increasingly producing absurd results.

by Dove 2y ago

austin-cheney 2y ago

> A measure being objective doesn't make it good That completely misses the point. It’s not about what’s good. It’s not even about what’s better. It’s only about how much better, the distance between theirs and ours. Better is objective only when like aspects of competing items are compared within accepted bounds of precision using evidence. The interesting thing about measuring stuff isn’t that people are otherwise entirely wrong in their assumptions more than 80% of the time, but that they are typically wrong by one or more orders of magnitude.

Dove 2y ago
>> A measure being objective doesn't make it good >That completely misses the point. It’s not about what’s good The loss of context here makes me wonder if I am talking to an AI. The comment you originally replied to was, We know what a good batting looks like but we still can’t say what good code is in any reasonably objective way. You replied with, "Sure we do" and proposed a list of metrics. Are we tracking? The original comment claimed that we do not know how to objectively measure goodness in code. You are (apparently) claiming to know how to do it. I am claiming it is impossible even in principle. In this context, I find your response ("It’s not about what’s good", and the claim that "better" is easier fo measure) bizarre and nonsensical. Like an AI, you seem to have lost track of what we are talking about. We are talking about whether we can objectively measure code being good. It is exactly "about what's good". Of course it is possible to measure things about code, but equating those measures to goodness relies on artificially constrained circumstances -- like a code golf contest or a PO declaring test coverage a metric to maximize. Athletes find themselves in such constrained circumstances all the time because it is a pursuit dominated by competition and games! It is most obvious in sports like sprinting or powerlifting, which are analogous to something like code golf, but even sports like basketball in which "goodness" is harder to define are heavily artificially constrained such that goodness in a player is about maximizing an objective measure - team score. This might be analogous to a programmer who sees his mission in terms of ticket closed per week. By contrast, programmers in general are usually working in a context in which the quality of what they produce is measured by a lot of complex impacts - on users, on business, on other programmers. Some of what code needs to accomplish - conceptualizing a problem well, communicating clearly - is inherently subjective, having to do with how it is received by another mind. Programmers (myself included) are generally focusing on these sorts of characteristics when talking about code in isolation, partly because we feel the impacts to ourselves most keenly. But I would contend that there is something deeper and less obvious here - that maximizing this subjective goodness profoundly improves the situation in more objective areas. Well architected code, well communicated code, clear code is resistant to defects in a way that mere test coverage can't accomplish. This is not obvious, but it is deep wisdom arising from experience, and is part of what drives programmers to emphasize the ineffable in their understanding of goodness. In fact, I was originally going to draw a parallel between being a good basketball player, maximizing team score, and a programmer maximizing business revenue. But I stopped myself, because this is an element of the deep wisdom of good code: focusing on ineffable, subjective excellence is profoundly positive for revenue. It's not something you can measure directly so much as it defines the circumstances the business finds itself in. This observation isn't unique to me - here's a pg essay making a similar point: https://paulgraham.com/avg.html https://paulgraham.com/avg.html Programmers talking about code being good in this sense may only be building sandcastles in the air - but they also may not be. You must possess the wisdom yourself to tell the difference. But there is certainly more to it than than selfish vanity. But even leaving that aside, because code must meet a broad array of conflicting demands, optimizing among those depends both on the circumstance and the values held by the people in it. Hence, goodness in code will always have a subjective element, and (in the athletic sphere) is most like saying you are "healthy and fit" or "your best self". You can certainly bring measurements to bear, and we are certainly talking about something real, but there is an inescapable subjective dependence on the value judgement of the judge. This actually touches on a broader philosophical debate: Are value judgements mere meaningless personal preferences, or are they (often imperfect) attempts to articulate something real? In code, and in life, I believe the subjective is pursuing the real, and moreover that anyone who thinks it's worth arguing about intuitively agrees with this assessment. By contrast, the view that only the objective is real, popular as it has been for the last couple of centuries, and attractive as its promises are, has been increasingly producing absurd results.