4 ms·

> as much as possible, do not allow custom patterns/structures This is key. Most engineers want to build frameworks instead of applications. But leave the fram

> as much as possible, do not allow custom patterns/structures This is key. Most engineers want to build frameworks instead of applications. But leave the frameworks to the framework builders unless you were hired to build a framework. I can jump into a 10 year old rails code base and be productive right away if they stuck to the framework defaults. But invariably someone read some blog posts about how "rails doesn't scale" and built some rats nest of service objects/decorators/portsAndAdapters wtfs and it takes some serious code spelunking to figure out what's going on (hint - the end result is always creating or updating a database record, or making a single api call). I would argue microservices also encourage bitrot. You can leave some service running on some ancient libraries for years it doesn't need changes. But if you have a monolith where you constantly keep the dependencies up to date - everything stays up to date.

twawaaay 4y ago
I am sorry I have quickly deleted that part with advices while you must have been responding to it. But this in my experience is truly important -- from the very start plan how a new member of the team is going to get really proficient at the system. Ideally, they should be able to immediately recognise/reuse patterns/idioms they already know. People bitch and moan about boilerplate of Java applications. But when I join the project and I see controllers, models, repositories, entities, services, etc. I am pretty much at home as I know what all those things are doing and how they are connected to each other. Whenever developers decide to run with their intellectual desires it ends badly. I try to explain to people that typical business backends applications do not have custom problems. Most of those REST services are just supposed to receive query, run a piece of business logic, translate it into database call and then map response into specified format.
- dan-robertson 4y ago
  Maybe we work on wildly different things, but I feel like this is looking at things from too low a level. That is, I think uncommon or unfamiliar patterns are much less of an issue than no one knowing the whole critical path of a system or undocumented invariants, especially between systems. Eg maybe you must restart certain things in a certain order after making Config changes. Maybe there are times when restarts are/aren’t safe. Maybe you need to have a certain OS config parameter set to get good performance and new boxes won’t have it set automatically. Maybe there is an accidental circular or nearly-circular dependency between two systems and you’ll have a terrible time if you accidentally create downtime for both at the same time. Maybe you have a binary that’s been lying around and running as some corn job but no one has compiled/deployed it since 2015 and you don’t have the source code or really know what it does. Mostly I’m thinking of more generic services rather than pure ‘business logic’ systems. These are the sorts of problems that Twitter might have in the future. Some of these are problems that Facebook might have gotten to discover in their last big outage.
  - twawaaay 4y ago
    I noticed that when people do not understand the system, things start to deteriorate until they can't do the simplest thing correctly or doing it takes way more time than stakeholders can accept. There is then pressure to get done faster, people start compromising on quality, etc. I hope you can see how there are many ways for things to start spiraling down. Many probably noticed that very old, relatively simple machines, cars, tractors, printing presses, etc. can be maintained almost indefinitely. Many businesses do prefer old machines even when there are superior new alternatives and that is because the new versions are just too complex. To keep maintaining something indefinitely you need to understand the machine enough to know every single piece that needs to be maintained and how to do it properly. Everywhere you need to lube it or clean it or to look from time to time. Make something complex enough and nobody will have this knowledge. Things start deteriorating. Maybe the data in the database will start accumulating defects, maybe those defects start breaking processes, maybe the processes will not be fixed properly but the errors just get ignored possibly causing other data inconsistencies, and so on. When you have people who wrote it and understand it fully they can very likely maintain it properly because they understand how to work with it, where to look, how to address problems. Loose them and if the system is complex enough the new crew may not be able to acquire this ability. And then it is just the question of how fast the system is going to be deteriorating.
j2kun 4y ago
> But if you have a monolith where you constantly keep the dependencies up to date - everything stays up to date. At the expense of dependencies constantly breaking you, which itself contributes to bit rot by pissing off developers who find workarounds to pin their dependencies and/or switch to work on different products.
- jacobsenscott 4y ago
  This is not my experience, but we update our dependencies multiple times per month. Of course if you let them go for six months or a year you'll be in the situation you are talking about.

twawaaay 4y ago

I am sorry I have quickly deleted that part with advices while you must have been responding to it. But this in my experience is truly important -- from the very start plan how a new member of the team is going to get really proficient at the system. Ideally, they should be able to immediately recognise/reuse patterns/idioms they already know. People bitch and moan about boilerplate of Java applications. But when I join the project and I see controllers, models, repositories, entities, services, etc. I am pretty much at home as I know what all those things are doing and how they are connected to each other. Whenever developers decide to run with their intellectual desires it ends badly. I try to explain to people that typical business backends applications do not have custom problems. Most of those REST services are just supposed to receive query, run a piece of business logic, translate it into database call and then map response into specified format.

dan-robertson 4y ago
Maybe we work on wildly different things, but I feel like this is looking at things from too low a level. That is, I think uncommon or unfamiliar patterns are much less of an issue than no one knowing the whole critical path of a system or undocumented invariants, especially between systems. Eg maybe you must restart certain things in a certain order after making Config changes. Maybe there are times when restarts are/aren’t safe. Maybe you need to have a certain OS config parameter set to get good performance and new boxes won’t have it set automatically. Maybe there is an accidental circular or nearly-circular dependency between two systems and you’ll have a terrible time if you accidentally create downtime for both at the same time. Maybe you have a binary that’s been lying around and running as some corn job but no one has compiled/deployed it since 2015 and you don’t have the source code or really know what it does. Mostly I’m thinking of more generic services rather than pure ‘business logic’ systems. These are the sorts of problems that Twitter might have in the future. Some of these are problems that Facebook might have gotten to discover in their last big outage.
- twawaaay 4y ago
  I noticed that when people do not understand the system, things start to deteriorate until they can't do the simplest thing correctly or doing it takes way more time than stakeholders can accept. There is then pressure to get done faster, people start compromising on quality, etc. I hope you can see how there are many ways for things to start spiraling down. Many probably noticed that very old, relatively simple machines, cars, tractors, printing presses, etc. can be maintained almost indefinitely. Many businesses do prefer old machines even when there are superior new alternatives and that is because the new versions are just too complex. To keep maintaining something indefinitely you need to understand the machine enough to know every single piece that needs to be maintained and how to do it properly. Everywhere you need to lube it or clean it or to look from time to time. Make something complex enough and nobody will have this knowledge. Things start deteriorating. Maybe the data in the database will start accumulating defects, maybe those defects start breaking processes, maybe the processes will not be fixed properly but the errors just get ignored possibly causing other data inconsistencies, and so on. When you have people who wrote it and understand it fully they can very likely maintain it properly because they understand how to work with it, where to look, how to address problems. Loose them and if the system is complex enough the new crew may not be able to acquire this ability. And then it is just the question of how fast the system is going to be deteriorating.

j2kun 4y ago

> But if you have a monolith where you constantly keep the dependencies up to date - everything stays up to date. At the expense of dependencies constantly breaking you, which itself contributes to bit rot by pissing off developers who find workarounds to pin their dependencies and/or switch to work on different products.

jacobsenscott 4y ago
This is not my experience, but we update our dependencies multiple times per month. Of course if you let them go for six months or a year you'll be in the situation you are talking about.