5 ms·

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enter

What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There's this bizarre misconception popular among bigtech managers that there's some tunable tradeoff between quality and development speed. But it doesn't actually work that way at all. I can't even count anymore how many times I've had to explain how taking this or that locally optimal shortcut will make it take longer overall to complete the project. In other words, it's a skill issue. LLMs can only make this worse. Hiring unskilled programmers and giving them a machine for generating garbage isn't the way. Instead, train them, and reject low quality work.

sethammons 2y ago
"Slow is smooth and smooth is fast"
- jcgrillo 2y ago
  It's true every single time.
giantg2 2y ago
"What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti." In my opinion the reason we get enterprise spaghetti is largely due to requirement issues and scope creep. It's nearly impossible to create a streamlined system without knowing what it should look like. And once the system gets to a certain size, it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary. Plus the full requirements are usually poorly documented and long forgotten by that time.
- jcgrillo 2y ago
  When scopes creep and requirements change, simply refactor. Where is it written in The Law that you have to accrue technical debt? EDIT: I'm gonna double down on this one. The fact that your organization thinks they can demand of you that you can magically weathervane your codebase to their changeable whims is evidence that you have failed to realistically communicate to them what is actually possible to do well. The fact that they think it's a move you can make to creep the scope, or change the requirements, is the problem. Every time that happens it should be studied within the organization as a major, costly failure--like an outage or similar. > it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary That's a choice. There are some other options: - Simply don't get business buy-in. Do without. Form a terrorist cell within your organization. You'll likely outpace them. Or you'll get fired, which means you'll get severance, unemployment, a vacation, and the opportunity to apply to a job at a better company. - Fight viciously for engineering independence. You business people can do the businessing, but us engineers are going to do the engineering. We'll tell you how we'll do it, not the other way. - Build companies around a culture of doing good, consistent work instead of taking expedient shortcuts. They're rare, but they exist!
  - aleph_minus_one 2y ago
    > Fight viciously for engineering independence. Or simply find a position in an industry or department where you commonly have more independence. In my opinion this fight is not worth it - look for another position instead is typically easier.
  - llm_trw 2y ago
    >When scopes creep and requirements change, simply refactor. Congratulations, you just refactored out a use case which was documented in a knowledge base which has been replaced by 3 newer ones since then, happens once every 18 months and makes the company go bankrupt if it isn't carried out promptly. The type of junior devs who think that making code tidy is fixing the application are the type of dev who you don't let near the heart of the code base, and incidentally the type who are best replaced with code gen AI.
    - 2y ago
jihadjihad 2y ago
> The problem is finding people to do it with who care enough to think rigorously > ... > train them, and reject low quality work. I agree very strongly with both of these points. But I've observed a truth about each of them over the last decade-plus of building software. 1) very few people approach the field of software engineering with anything remotely resembling rigor, and 2) there is often little incentive to train juniors and reject subpar output (move fast and break things, etc.) I don't know where this takes us as an industry? But I feel your comment on a deep level.
- steve_adams_86 2y ago
  I agree as well. These are actually things that bother me a lot about the industry. I’d love to write software that should run problem-free in 2035, but the reality is almost no one cares. I’ve had the good fortune of getting to write some firmware that will likely work well for a long time to come, but I find most things being written on computers are written with (or very close to) the minimum care possible in order to get the product out. Clean up is intended but rarely occurs. I think we’d see real benefits from doing a better job, but like many things, we fail to invest early and crave immediate gratification.
- karolinepauls 2y ago
  > very few people approach the field of software engineering with anything remotely resembling rigor, and I have this one opinion which I would not say at work: In software development it's easy to feel smart because what you made "works" and you can show "effects". - Does it wrap every failable condition in `except Exception`? Uhh, but look, it works. - Does it define a class hierarchy for what should be a dictionary lookup? It works great tho! - Does it create a cyclic graph of objects calling each other's methods to create more objects holding references to the objects that created them? And for what, to produce a flat dictionary of data at end of the day? But see, it works. this is getting boring, maybe just skip past the list - Does it stuff what should be local variables and parameters in self, creating a big stateful blob of an object where every attribute is optional and methods need to be called in the right order, otherwise you get an exception? Yes, but it works. - Does it embed a browser engine? But it works! The programmer, positively affirmed, continues spewing out crap, while the senior keep fighting fires to keep things running, while insulating the programmer from the taste of their own medicine. But more generally, it's hard to expect people to learn how to solve problems simply if they're given gigantic OO languages with all the features and no apparent cost to any of them. People learn how to write classes and then never learn get good at writing code with a clear data flow. Even very bright people can get fall for this trap because engineering isn't just about being smart but about using intelligence and experience to solve a problem while minmaxing correctly chosen properties. Those properties should generally be: dev time, complexity (state/flow), correctness, test coverage, ease of change, performance (anything else?). Anyway, "Affirming one's opinions about how things should be done" isn't one of them.
A4ET8a8uTh0 2y ago
<< Instead, train them, and reject low quality work. Ahh, well, in order to save money, training is done via an online class with multiple choice questions, or, if your company is like mine and really committed to making sure that you know they take your training seriously, they put portions of a generic book on 'tech Z' in pdf spread spread over a drm ridden web pages. As for code, that is reviewed, commented and rejected by llms as well. It is used to be turtles. Now it truly is llms all the way down. That said, in a sane world, this is what should be happening for a company that actually wants to get good results over time .
c0balt 2y ago
It's relatively easy to find a programmer(s) who can realize enterprise project X, it's hard to find a programmer(s) who cares about X. Throwing an increased requirement like speed at it makes this worse because it usually ends up burning out both ends of the equation.
aleph_minus_one 2y ago
> What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. I don't think finding such programmers is really difficult. What is difficult is finding such people if you expect them to be docile to incompetent managers and other incompetent people involved in the project who, for example, got their position not by merit and competence, but by playing political games.
wpietri 2y ago
Agreed. The way I explain this to managers is that software development is unlike most work. If I'm making widgets and I fuck up, that widget goes out the door never to be seen again. But in software, today's outputs are tomorrow's raw materials. You can trade quality for speed in the very short term at the cost of future productivity, so you're really trading speed for speed. I should add, though, that one can do the rigorous thinking before or after the doing, and ideally one should do both. That was the key insight behind Martin Fowler's "Refactoring: Improving the Design of Existing Code". Think up front if you can, but the best designs are based on the most information, and there's a lot of information that is not available until later in a project. So you'll want to think as information comes in and adjust designs as you go. That's something an LLM absolutely can't do, because it doesn't have access to that flow of information and it can't think about where the system should be going.
- jcgrillo 2y ago
  > the best designs are based on the most information, and there's a lot of information that is not available until later in a project This is an important point. I don't remember where I read it, but someone said something similar about taking a loss on your first few customers as an early stage startup--basically, the idea is you're buying information about how well or poorly your product meets a need. Where it goes wrong is if you choose not to act on that information.
  - wpietri 2y ago
    For sure. Or, worse, choose to run a company in such a way that anybody making choices is insulated from that information.
galdosdi 2y ago
It's a market for lemons. Without redoing their work or finding a way to have deep trust (which is possible, but uncommon at a bigcorp) it's hard enough to tell who is earnest and who is faking it (or buying their own baloney) when it comes to propositions like "investing in this piece of tech debt will pay off big time" As a result, if managers tend to believe such plans, bad ideas drive out good and you end up investing in a tech debt proposal that just wastes time. Burned managers therefore cope by undervaluing any such proposals and preferring the crappy car that at least you know is crappy over the car that allegedly has a brand new 0 mile motor on it but you have no way of distinguishing from a car with a rolled back odometer. They take the locally optimal path because it's the best they can do. It's taken me 15 years of working in the field and thinking about this to figure it out. The only way out is an organization where everyone is trusted and competent and is worthy of trust, which again, hard to do at most random bigcorps. This is my current theory anyway. It's sad, but I think it kind of makes sense.
- jcgrillo 2y ago
  Soviet vs NATO. The Soviet management style is micromanaging exactly how to do everything from the rear. The NATO style is delegating to the front line ranks. Being good at the NATO style of management means focusing on the big picture--what, when, why--and leaving how to the people actually doing it.
noisy_boy 2y ago
> The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There is no incentive to do it. I worked that way, focused on quality and testing and none of my changes blew up in production. My manager opined that this approach is too slow and that it was ok to have minor breakages as long as they are fixed soon. When things break though, it's blame game all around. Loads of hypocrisy.

sethammons 2y ago

"Slow is smooth and smooth is fast"

jcgrillo 2y ago
It's true every single time.

giantg2 2y ago

"What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti." In my opinion the reason we get enterprise spaghetti is largely due to requirement issues and scope creep. It's nearly impossible to create a streamlined system without knowing what it should look like. And once the system gets to a certain size, it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary. Plus the full requirements are usually poorly documented and long forgotten by that time.

jcgrillo 2y ago
When scopes creep and requirements change, simply refactor. Where is it written in The Law that you have to accrue technical debt? EDIT: I'm gonna double down on this one. The fact that your organization thinks they can demand of you that you can magically weathervane your codebase to their changeable whims is evidence that you have failed to realistically communicate to them what is actually possible to do well. The fact that they think it's a move you can make to creep the scope, or change the requirements, is the problem. Every time that happens it should be studied within the organization as a major, costly failure--like an outage or similar. > it's impossible to get business buy-in to rearchitect or refactor to the degree that is necessary That's a choice. There are some other options: - Simply don't get business buy-in. Do without. Form a terrorist cell within your organization. You'll likely outpace them. Or you'll get fired, which means you'll get severance, unemployment, a vacation, and the opportunity to apply to a job at a better company. - Fight viciously for engineering independence. You business people can do the businessing, but us engineers are going to do the engineering. We'll tell you how we'll do it, not the other way. - Build companies around a culture of doing good, consistent work instead of taking expedient shortcuts. They're rare, but they exist!
- aleph_minus_one 2y ago
  > Fight viciously for engineering independence. Or simply find a position in an industry or department where you commonly have more independence. In my opinion this fight is not worth it - look for another position instead is typically easier.
- llm_trw 2y ago
  >When scopes creep and requirements change, simply refactor. Congratulations, you just refactored out a use case which was documented in a knowledge base which has been replaced by 3 newer ones since then, happens once every 18 months and makes the company go bankrupt if it isn't carried out promptly. The type of junior devs who think that making code tidy is fixing the application are the type of dev who you don't let near the heart of the code base, and incidentally the type who are best replaced with code gen AI.
  - 2y ago

jihadjihad 2y ago

> The problem is finding people to do it with who care enough to think rigorously > ... > train them, and reject low quality work. I agree very strongly with both of these points. But I've observed a truth about each of them over the last decade-plus of building software. 1) very few people approach the field of software engineering with anything remotely resembling rigor, and 2) there is often little incentive to train juniors and reject subpar output (move fast and break things, etc.) I don't know where this takes us as an industry? But I feel your comment on a deep level.

steve_adams_86 2y ago
I agree as well. These are actually things that bother me a lot about the industry. I’d love to write software that should run problem-free in 2035, but the reality is almost no one cares. I’ve had the good fortune of getting to write some firmware that will likely work well for a long time to come, but I find most things being written on computers are written with (or very close to) the minimum care possible in order to get the product out. Clean up is intended but rarely occurs. I think we’d see real benefits from doing a better job, but like many things, we fail to invest early and crave immediate gratification.
karolinepauls 2y ago
> very few people approach the field of software engineering with anything remotely resembling rigor, and I have this one opinion which I would not say at work: In software development it's easy to feel smart because what you made "works" and you can show "effects". - Does it wrap every failable condition in `except Exception`? Uhh, but look, it works. - Does it define a class hierarchy for what should be a dictionary lookup? It works great tho! - Does it create a cyclic graph of objects calling each other's methods to create more objects holding references to the objects that created them? And for what, to produce a flat dictionary of data at end of the day? But see, it works. this is getting boring, maybe just skip past the list - Does it stuff what should be local variables and parameters in self, creating a big stateful blob of an object where every attribute is optional and methods need to be called in the right order, otherwise you get an exception? Yes, but it works. - Does it embed a browser engine? But it works! The programmer, positively affirmed, continues spewing out crap, while the senior keep fighting fires to keep things running, while insulating the programmer from the taste of their own medicine. But more generally, it's hard to expect people to learn how to solve problems simply if they're given gigantic OO languages with all the features and no apparent cost to any of them. People learn how to write classes and then never learn get good at writing code with a clear data flow. Even very bright people can get fall for this trap because engineering isn't just about being smart but about using intelligence and experience to solve a problem while minmaxing correctly chosen properties. Those properties should generally be: dev time, complexity (state/flow), correctness, test coverage, ease of change, performance (anything else?). Anyway, "Affirming one's opinions about how things should be done" isn't one of them.

A4ET8a8uTh0 2y ago

<< Instead, train them, and reject low quality work. Ahh, well, in order to save money, training is done via an online class with multiple choice questions, or, if your company is like mine and really committed to making sure that you know they take your training seriously, they put portions of a generic book on 'tech Z' in pdf spread spread over a drm ridden web pages. As for code, that is reviewed, commented and rejected by llms as well. It is used to be turtles. Now it truly is llms all the way down. That said, in a sane world, this is what should be happening for a company that actually wants to get good results over time .

c0balt 2y ago

It's relatively easy to find a programmer(s) who can realize enterprise project X, it's hard to find a programmer(s) who cares about X. Throwing an increased requirement like speed at it makes this worse because it usually ends up burning out both ends of the equation.

aleph_minus_one 2y ago

> What I can't wrap my head around is that making good, efficient software doesn't (by and large) take significantly longer than making bloated, inefficient enterprise spaghetti. The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. I don't think finding such programmers is really difficult. What is difficult is finding such people if you expect them to be docile to incompetent managers and other incompetent people involved in the project who, for example, got their position not by merit and competence, but by playing political games.

wpietri 2y ago

Agreed. The way I explain this to managers is that software development is unlike most work. If I'm making widgets and I fuck up, that widget goes out the door never to be seen again. But in software, today's outputs are tomorrow's raw materials. You can trade quality for speed in the very short term at the cost of future productivity, so you're really trading speed for speed. I should add, though, that one can do the rigorous thinking before or after the doing, and ideally one should do both. That was the key insight behind Martin Fowler's "Refactoring: Improving the Design of Existing Code". Think up front if you can, but the best designs are based on the most information, and there's a lot of information that is not available until later in a project. So you'll want to think as information comes in and adjust designs as you go. That's something an LLM absolutely can't do, because it doesn't have access to that flow of information and it can't think about where the system should be going.

jcgrillo 2y ago
> the best designs are based on the most information, and there's a lot of information that is not available until later in a project This is an important point. I don't remember where I read it, but someone said something similar about taking a loss on your first few customers as an early stage startup--basically, the idea is you're buying information about how well or poorly your product meets a need. Where it goes wrong is if you choose not to act on that information.
- wpietri 2y ago
  For sure. Or, worse, choose to run a company in such a way that anybody making choices is insulated from that information.

galdosdi 2y ago

It's a market for lemons. Without redoing their work or finding a way to have deep trust (which is possible, but uncommon at a bigcorp) it's hard enough to tell who is earnest and who is faking it (or buying their own baloney) when it comes to propositions like "investing in this piece of tech debt will pay off big time" As a result, if managers tend to believe such plans, bad ideas drive out good and you end up investing in a tech debt proposal that just wastes time. Burned managers therefore cope by undervaluing any such proposals and preferring the crappy car that at least you know is crappy over the car that allegedly has a brand new 0 mile motor on it but you have no way of distinguishing from a car with a rolled back odometer. They take the locally optimal path because it's the best they can do. It's taken me 15 years of working in the field and thinking about this to figure it out. The only way out is an organization where everyone is trusted and competent and is worthy of trust, which again, hard to do at most random bigcorps. This is my current theory anyway. It's sad, but I think it kind of makes sense.

jcgrillo 2y ago
Soviet vs NATO. The Soviet management style is micromanaging exactly how to do everything from the rear. The NATO style is delegating to the front line ranks. Being good at the NATO style of management means focusing on the big picture--what, when, why--and leaving how to the people actually doing it.

noisy_boy 2y ago

> The problem is finding people to do it with who care enough to think rigorously about what they're going to do before they start doing it. There is no incentive to do it. I worked that way, focused on quality and testing and none of my changes blew up in production. My manager opined that this approach is too slow and that it was ok to have minor breakages as long as they are fixed soon. When things break though, it's blame game all around. Loads of hypocrisy.