4 ms·

While premature optimization is still evil, I think it's no longer the root of it. Average quality of developer is way lower than it used to be (people aren't

by hbrn 4y ago

While premature optimization is still evil, I think it's no longer the root of it. Average quality of developer is way lower than it used to be (people aren't even capable of optimization). The need for optimization is also not as high: hardware is fast and cheap, abundance of Electron apps illustrates that. I have a feeling that today is the age of premature abstractions and premature scalability. The main point is that there's the right and the wrong time to do anything: optimization, scaling, testing, monitoring, etc. When a concept is deemed absolutely positive, naturally it will be applied prematurely.

firebaze 4y ago
The root of all evil is someone claiming to know the root of all evil. Snarky, but in my experience 6 sigma true.
mjlawson 4y ago
I agree with this, mostly. I've worked at many a company where I've inherited the work of developers who built towards an expected view of the future. That expected future, of course, never quite lined up with the future that did come to pass. So a lot of extra work building out the wrong abstraction lead to a lot more work to undo it. I would disagree with you only that I'm not familiar with a time when premature monitoring was added that caused me a lot of pain. If anything, the opposite has been true. Can you explain what you mean there?
- hbrn 4y ago
  It all comes down to cost. Most of the time you can get kind monitoring for free or at a very low cost. AWS gives you a bunch of metrics out of the box for every product. Wrap your webapp in newrelic-agent and get a bunch of nice dashboards. But the more you want to monitor, the higher the costs are. There's a lot of examples where you can catch something with monitoring, but it doesn't necessarily mean that you should. A recent one from my memory: in a SaaS product a team shipped a bug that went unnoticed for a few days. It was feature flagged, so it only affected a small fraction of customers and didn't trigger any global alerts. Now, since it didn't trigger alerts, the natural post-mortem action plan was "better monitoring". That would mean monitoring and alerting on "rate of errors by customer" (or "rate of errors by endpoint by customer", I don't remember). Given the usage pattern of the product, it was impossible to create a global monitor like that, we'd have manually configure it for each customer (and we had thousands of those). And even then, we'd inevitably be dealing with false positives every week. The right action plan was to learn from failure, but do nothing. We got extremely unlucky during infrastructure update, shit happens. We don't need to build a complex monitoring system that catches one bug every 5 years.
halayli 4y ago
> I have a feeling that today is the age of premature abstractions and premature scalability which are optimizations.
dahart 4y ago
I feel like what you’re referring to is mainly web dev? There I agree a lot about premature scaling - a lot of people build out for the size of web site they hope to have, and not for the size they have or end up with. That might not be entirely the wrong choice for a startup, there absolutely is a certain amount of guessing what you’ll need if everything goes well. > Average quality of developer is way lower than it used to be I think this is an illusion based on the fact that the number of software developers is growing quickly, which means that most of them are new & young at all times, giving the misleading impression that things are trending down from the perspective of someone experienced. In absolute numbers, there might also seem to be a lot more unserious developers today than 10 or 20 or 30 years ago, but there’s also many many more serious and extremely good devs too, and that can be harder to see from where you sit. Schools have gotten better in many (but not all) ways too. > The need for optimization is also not as high With Moore’s law slowing down, and with competition in software greater than ever before, and with improvements to compute continuing to outpace improvements in memory bandwidth, I think the complete opposite is true, that the need for optimization has never been higher and is increasing as we speak. And I think in practice more people are optimizing systems like crazy compared to when I was younger. In school we learned about the occasional writing of a small loop in assembly language. Today we have cloud caches, and GPUs running CUDA code, and much better compilers. You might just not be getting exposure to optimization where you work, and there are places where people don’t see a lot of it, but there are tons of groups and companies where people are always optimizing. I see a lot of it, and I don’t think Electron apps demonstrate it’s not happening. Electron exists and is popular because it’s easy, and that’s a valid development choice for a lot of people. It might be slow, but there have always been frameworks that are easy to use that are slow. And then there have always been the people who care about performance and avoid slow frameworks, they’re still here and growing in number.
jandrewrogers 4y ago
I am ambivalent about your assertion. Developers can do much more while knowing much less than in the past, a lot of complicated details that could not be ignored in the past are out of sight and out of mind. However, from a systems engineering view, our software is deeply and vastly more complicated than it used to be. We routinely use software today that had no complexity equivalent in the 1990s. No one wrote software in the 1990s like the advanced software we write today, it is qualitatively different. I think that perspective is lost on a lot of people that are not as old as I am. That increase in complexity is a serious barrier to understanding. The state-of-the-art today requires cognitive abilities that are much higher than they used to be. Hell, it is why C used to be an adequate language for most tasks.

by hbrn 4y ago

firebaze 4y ago

The root of all evil is someone claiming to know the root of all evil. Snarky, but in my experience 6 sigma true.

mjlawson 4y ago

I agree with this, mostly. I've worked at many a company where I've inherited the work of developers who built towards an expected view of the future. That expected future, of course, never quite lined up with the future that did come to pass. So a lot of extra work building out the wrong abstraction lead to a lot more work to undo it. I would disagree with you only that I'm not familiar with a time when premature monitoring was added that caused me a lot of pain. If anything, the opposite has been true. Can you explain what you mean there?

hbrn 4y ago
It all comes down to cost. Most of the time you can get kind monitoring for free or at a very low cost. AWS gives you a bunch of metrics out of the box for every product. Wrap your webapp in newrelic-agent and get a bunch of nice dashboards. But the more you want to monitor, the higher the costs are. There's a lot of examples where you can catch something with monitoring, but it doesn't necessarily mean that you should. A recent one from my memory: in a SaaS product a team shipped a bug that went unnoticed for a few days. It was feature flagged, so it only affected a small fraction of customers and didn't trigger any global alerts. Now, since it didn't trigger alerts, the natural post-mortem action plan was "better monitoring". That would mean monitoring and alerting on "rate of errors by customer" (or "rate of errors by endpoint by customer", I don't remember). Given the usage pattern of the product, it was impossible to create a global monitor like that, we'd have manually configure it for each customer (and we had thousands of those). And even then, we'd inevitably be dealing with false positives every week. The right action plan was to learn from failure, but do nothing. We got extremely unlucky during infrastructure update, shit happens. We don't need to build a complex monitoring system that catches one bug every 5 years.

halayli 4y ago

> I have a feeling that today is the age of premature abstractions and premature scalability which are optimizations.

dahart 4y ago

I feel like what you’re referring to is mainly web dev? There I agree a lot about premature scaling - a lot of people build out for the size of web site they hope to have, and not for the size they have or end up with. That might not be entirely the wrong choice for a startup, there absolutely is a certain amount of guessing what you’ll need if everything goes well. > Average quality of developer is way lower than it used to be I think this is an illusion based on the fact that the number of software developers is growing quickly, which means that most of them are new & young at all times, giving the misleading impression that things are trending down from the perspective of someone experienced. In absolute numbers, there might also seem to be a lot more unserious developers today than 10 or 20 or 30 years ago, but there’s also many many more serious and extremely good devs too, and that can be harder to see from where you sit. Schools have gotten better in many (but not all) ways too. > The need for optimization is also not as high With Moore’s law slowing down, and with competition in software greater than ever before, and with improvements to compute continuing to outpace improvements in memory bandwidth, I think the complete opposite is true, that the need for optimization has never been higher and is increasing as we speak. And I think in practice more people are optimizing systems like crazy compared to when I was younger. In school we learned about the occasional writing of a small loop in assembly language. Today we have cloud caches, and GPUs running CUDA code, and much better compilers. You might just not be getting exposure to optimization where you work, and there are places where people don’t see a lot of it, but there are tons of groups and companies where people are always optimizing. I see a lot of it, and I don’t think Electron apps demonstrate it’s not happening. Electron exists and is popular because it’s easy, and that’s a valid development choice for a lot of people. It might be slow, but there have always been frameworks that are easy to use that are slow. And then there have always been the people who care about performance and avoid slow frameworks, they’re still here and growing in number.

jandrewrogers 4y ago

I am ambivalent about your assertion. Developers can do much more while knowing much less than in the past, a lot of complicated details that could not be ignored in the past are out of sight and out of mind. However, from a systems engineering view, our software is deeply and vastly more complicated than it used to be. We routinely use software today that had no complexity equivalent in the 1990s. No one wrote software in the 1990s like the advanced software we write today, it is qualitatively different. I think that perspective is lost on a lot of people that are not as old as I am. That increase in complexity is a serious barrier to understanding. The state-of-the-art today requires cognitive abilities that are much higher than they used to be. Hell, it is why C used to be an adequate language for most tasks.