5 ms·
Hacker Laws: Theories, principles and patterns that developers will find useful
- NineBlam 7y agoThis is a nice list to read through. The concept quoted, Goodhart’s law, came up in a project review today, though I didn’t know it was named. It feels nearly unavoidable, but at least recognition helps.
- jedbrown 7y agoThis paper attempts to classify and evaluate mitigation techniques for flawed metrics. https://mpra.ub.uni-muenchen.de/90649/1/MPRA_paper_90649.pdf https://mpra.ub.uni-muenchen.de/90649/1/MPRA_paper_90649.pdf
- cpeterso 7y agoThanks! I had hoped the original article might provide approaches for balancing the positives and negatives of metrics, but didn't find any. The "Building Less Flawed Metrics: Dodging Goodhart and Campbell’s Laws" paper you linked is exactly what I was looking for. :)
- csours 7y agoKeep your eye on the prize, and the prize isn't measured by metrics. Any prize in the real world is something complex and complicated, hard to describe and prone to squishiness. Metrics are things that are "easy" to measure and compare across systems. Setting goals in terms of metrics is appealing and even necessary because it makes things measurable and comparable in the real world, but you need to have some cross-check where you can come back and verify that you are moving closer to your real world goal. --- A real world example: Consumer products are often distinguished by appearance, and surface quality is an important facet of appearance. You purchase a surface quality tester and improve the measurements produced by the surface tester to the point where you beat a competitor's surface measurements. Then you have a customer clinic and have real potential customers evaluate the two products again. Customers prefer the competitor product because you were not actually measuring attractiveness or appearance of quality, you were measuring something that the surface tester measures.
- vanniv 7y agoIf the impact of your work can be losslessly reduced to something that can be directly and cheaply measured, you're probably solving an easy problem or a theoretical one.
- tobinfricke 7y agoReminds me of this story: https://www.bostonmagazine.com/news/2014/08/26/how-northeastern-gamed-the-college-rankings/ https://www.bostonmagazine.com/news/2014/08/26/how-northeast... "How to Game the College Rankings - Northeastern University executed one of the most dramatic turnarounds in higher education. Its recipe for success? A single-minded focus on just one list."
- aaron695 7y agoBullshit. When a measure becomes a target it will be gamed. Nothing says it will not work overall unless you have real examples that it STOPs becoming useful? Every fucking person goes on about the story of Bill Atkinson and the -2000 Lines Of Code. He was not fired for not competing the KPI, nor did anyone expect him to be. In fact he 'beat' the managers in the story and they stopped asking him for submitting lines of code because they are not stupid. No one talks about if it was a good measure for other staff, which at the time(early 80's) might have been good at a personal level or at a corporation level. People can see gaming and people can see when it's not fair. Just because you have a friend of a friend who read a story about gaming doesn't mean this 'it ceases to be a good measure' is true.
- d-d 7y ago> Goodhart's Law: When a measure becomes a target, it ceases to be a good measure. This makes me think of the first song I posted to YouTube. I couldn't help but curiously flip through the viewer statistics. Automatically I began wondering how I could prevent the 50% drop-off at X seconds, and what type of person was sticking around beyond the drop-off. And before I looked much into it I realized this would be a great way to suck all the life out of something that's supposed to be fun.
- saagarjha 7y agoThis is one of the many reasons I don't run analytics on my website (in addition to the more obvious ones, such as privacy). In my case, I still get useful "second-order" information from the frequency of people writing emails to me about the content, and it feels a lot nicer to read that someone found something I wrote useful than to try to micro-optimize for what people are willing to click on.
- hliyan 7y agoI came across this law many years ago, and since then lost track of it's origins: McIntye's Law: Under the right circumstances, anything I tell you could be wrong. I preface much of my advice to young developers with this.
- james_s_tayler 7y agoWow. I love it. Never heard this one before. Would be better if it were: Under the right circumstances, anything I tell you could be wrong. Including this.
- Peckingjay 7y agoMy immediate thought went to benchmarks. Some of them get stale pretty fast. Nice to have a name to put on that concept.
- dang 7y agoWe changed the submitted title to that of the page. The submitted title was Hacker Laws: “When a measure becomes a target, it ceases to be a good measure.”, which of course is Goodhart's Law. Some of the comments in this thread are referencing that one.
- matt4077 7y agoIt's commendable that the author counteracts the tendency towards cynicism with the occasional rebuttal, although to fully embrace the spirit of kindness, I would maybe skip the Dilbert reference. Remember: If everyone really were as stupid and/or evil as some of these maxims postulate, it'd be highly unlikely that you just happen to be the exception. Also: management, design, law, marketing, journalism, social sciences, and politics are all disciplines just as challenging as programming or engineering. And if you can't think of any reason why one of those fields should be difficult, you're just as likely to better at it as that field's practitioners being better at your job.
- BerislavLopac 7y agoPrevious discussion: https://news.ycombinator.com/item?id=21442330 https://news.ycombinator.com/item?id=21442330
- dsr_ 7y agoThe First Law of Consulting: The customer knows their goal, not how to achieve it. The Second Law of Consulting: The customer's list of constraints is incomplete, because fish don't think about water.
- jroesner 7y agoValuable collection Dave! Brilliant as always. Thx for putting this together.
- Chris2048 7y agoI absolutely disagree with Hanlon's Razor; malicious intent often hides behind the appearance of incompetance.
- 0x445442 7y agoYour assertion could be correct 49.9% of the time and still not invalidate the proposition.
- hderms 7y agoWould anyone care about discussing some 'rule' with barely greater explanatory power than flipping a coin? People aren't going to adjust their thought patterns, in practice, for such an improvement.
- Chris2048 7y agoWhat is the propostition? I read it as "That which is adequately explained by stupidity is not (or is rarely) malice". Otherwise it is a not a statement that can be objectively "factual" and is as such not a proposition.
- Pete_D 7y agoGoodhart's Corollary to Hanlon's Razor: When an exculpatory heuristic becomes well known, it ceases to be a good heuristic.
- AnimalMuppet 7y agoQuoting philipov here on HN: "Any sufficiently advanced incompetence is indistinguishable from malice."
- Chris2048 7y agoDRY is dangerous if applied to pedantically; sometimes similar code should be allowed to exist to make overall architecture simpler, or allow similar components to evolve (possibly Divergantly) over time.
- k_sze 7y agoThe Pragmatic Programmer also warns against overzealous deduplication. Stuff that are coincidentally the same should be left untouched. Only remove duplication of knowledge.
- fit2rule 7y agoRule #1: Its only about the user. Rule #2: The meaning of 'use' is, even itself, subject to the whims of the user - i.e. See Rule #1. Everything else is culture and can thus be discarded.