3 ms·

This is how SOLID, or rather its most zealous proponents, go from reasonable but limited capabilities to claims that cannot be sustained. While small classes a

This is how SOLID, or rather its most zealous proponents, go from reasonable but limited capabilities to claims that cannot be sustained. While small classes and methods can be (though are not necessarily) at least superficially easy to understand, this does not guarantee that the system as a whole is understandable. This is because a lot of the complexity is in how the classes interact with one another. There are contracts and constraints that have to be observed, that are inherently about more than one thing, and so cannot be localized in one method. This approach, applied to writing, would be to allow only kernel sentences using common words. This is not going to be enough to make a complex topic simple to understand, and, if applied zealously, is likely to have the opposite effect. Unnecessarily complex code is not a consequence of not following simple rules; it is usually a consequence of trying to write simple code but not understanding all the issues. There is nothing about SOLID that guarantees that unnecessary complexity will be avoided. Your idea of turning programming into something like painting-by-the-numbers will not work.

SergeAx 7y ago
In SOLID there is S for "single responsibility", which in turn gives us simplicity on a component (class/object/file, depends on language) level but lead to complexity due to number of components. 100 tasks gives us 100 components, then software grows to 1000 and even 10000 tasks. But then comes letter D for dependency inversion. This principle gives us back simplicity due to abstracting away any set of those components as meta-components like repositories, queues, services, factories and so on. This is about integfaces, e.g. contracts, as you call them. My zealotry comes from about 30 years of writing and SUPPORTING code. SOLID code is harder to write, but much more easier to maintain. All those war stories about "business needs that feature week ago" comes from unmaintainable code, when you just cannot add new features by any other means except spaghetti with epoxy glue and nuts and bolts all around. Business wants to move fast and break things. Our mission as software engineers is to create and maintain codebases solid enough (pun intended) to sustain years of development, pivots, M&As and whatnot.
- gdy 7y ago
  "But then comes letter D for dependency inversion. This principle gives us back simplicity due to abstracting away any set of those components as meta-components" I honestly don't understand how this produces simplicity. Could you possibly elaborate?
  - SergeAx 7y ago
    I'll try. Actually it's just what we did on my job this week. For a long time we've had a single Cassandra database for write-performant storage in our project. Let's say it was a storage for Foo objects, which we have to write like thousands per second and read back aggregated stats like total numbers, number per parent object, writes per second/minute/hour etc. With time it's read performance degraded. Also from time to time the database was diving into some internal service state (like index compacting or something like that), which further affected performance. At the end Cassandra cluster consisted of 30 hosts, which costs us about US$1000/day. So we decided to cut it off and change it for MongoDB, where we have much more expertise on the inside. In our code we had a service class, let's call it FooService. All operations with Foo objects (write, read, stats, delete) was made via that service object, like FooService::addFoo(Foo, Bar, Baz). Any business logic class wanting to deal with Foo objects just declares FooService as is dependency, which is injected in runtime on calling business logic class constructor. So when certain code path doesn't need Foo objects - system don't have to initialise FooService at all. FooService, in turn, dependes on FooRepository, which is a descendant of abstract CassandraRepository, which has code to connect to cluster, send requests, handle errors and so on. CassandraRepository is a descendant of AbstractRepository with methods like getItemById, getItemsByIds and so on. Overall LOC count in our project is about 400k, it is not very complex, but complex enough to stuck with task like that if your architecture is not clean enough. But in our case the task was super easy: 1.Spin up MongoDB cluster with just 7 hosts (master, 3 sharded replicas and 3 shadow replicas) 2.Inherit FooNewRepository from MongoRepository, point it on new cluster. 3.Inject FooNewRepository into FooService and start performing mutable operations (create, update, delete) into both Mongo and Cassandra bound repos 4.Migrate historical data with script, which is using the same common codebase (it took about 3 weeks actually) 5.Switch reads to FooNewRepository 6.Check that everyting is okay 7.Drop all Cassandra-related code from codebase and rename FooNewRepository to just FooRepository Because FooService declares it's repository dependencies as AbstractRepository, ALL the new code except FooRepository wass looking EXACTLY the same as before. The team noticed changes only due to our weekly all-hands, where we discussing all such matters. All the coding took about 2 days net, most of it was a migrating script, so with about US$750/day cost cut we adjusted that work in a matter of days. And now the cherry on a cake. When we tried to shutdown a Cassandra cluster, we've got errors on another instance of our service, which used the same database, but was forked about 6 months ago to temporarily serve another country/language. We turned Cassandra back on, I checked out that fork, cherry-picked only Cassandra->Mongo commits from master branch, run the migration script (there were much less data for that country), and voila - all tests passed like a charm, new code working now (we will migrate it to full master branch later). edit: formatting and grammar
craftinator 7y ago
I agree; it seems that SOLID does a good job of marshalling the complexity out of individual elements into the system as a whole. This can be a good thing, if the system design itself was created with SOLID in mind, and is well implemented. However if the system design didn't account for this complexity shift, it will likely cause a lot of mayhem.
- gdy 7y ago
  "if the system design itself was created with SOLID in mind" Could you possibly elaborate what you mean by that?
  - craftinator 7y ago
    Yes! In part I kept that ambiguous, because I've observed many edge cases and was trying to capture the overarching theme. The core idea is that often a project is designed without respect to the "design guidelines" that developers are asked to adhere to during implementation. I see SOLID as a set of design AND implementation guidelines; it contains rules for both the global and local systems. I've observed that often, the design of a system framework ignores the design portion, but requires the implementation portion (I've done it myself, and wondered why my project turned into a hot mess even though I followed all the rules!). With SOLID, complexity is pushed out of individual components and into the global scope. Instead of having a component manage multiple responsibilities, it must have only a single responsibility; now you have multiple components each managing single responsibilities. Some problems that can arise from this are multiple classes that have shared or overlapping responsibilities, classes that only handle edge cases, data innaccesibility, data and code duplication... This is different than (but related to) abstraction, in that you aren't hiding implementation, instead you are dividing it. With that division comes some non-trivial overhead, which the global design of the system needs to have planned for. This is not necessarily a bad thing; it reduces the burden of designing each individual component, while increasing the burden of designing the global system to manage how the components must interact. If, in designing a system using the SOLID principles, you do not plan for how much of the complexity is being pushed towards the global scope, you can have a huge codebase that isn't cohesive enough to function as a whole. That might've been too much elaboration, but it is kind of difficult to explain a portion without being thorough.

SergeAx 7y ago

In SOLID there is S for "single responsibility", which in turn gives us simplicity on a component (class/object/file, depends on language) level but lead to complexity due to number of components. 100 tasks gives us 100 components, then software grows to 1000 and even 10000 tasks. But then comes letter D for dependency inversion. This principle gives us back simplicity due to abstracting away any set of those components as meta-components like repositories, queues, services, factories and so on. This is about integfaces, e.g. contracts, as you call them. My zealotry comes from about 30 years of writing and SUPPORTING code. SOLID code is harder to write, but much more easier to maintain. All those war stories about "business needs that feature week ago" comes from unmaintainable code, when you just cannot add new features by any other means except spaghetti with epoxy glue and nuts and bolts all around. Business wants to move fast and break things. Our mission as software engineers is to create and maintain codebases solid enough (pun intended) to sustain years of development, pivots, M&As and whatnot.

gdy 7y ago
"But then comes letter D for dependency inversion. This principle gives us back simplicity due to abstracting away any set of those components as meta-components" I honestly don't understand how this produces simplicity. Could you possibly elaborate?
- SergeAx 7y ago
  I'll try. Actually it's just what we did on my job this week. For a long time we've had a single Cassandra database for write-performant storage in our project. Let's say it was a storage for Foo objects, which we have to write like thousands per second and read back aggregated stats like total numbers, number per parent object, writes per second/minute/hour etc. With time it's read performance degraded. Also from time to time the database was diving into some internal service state (like index compacting or something like that), which further affected performance. At the end Cassandra cluster consisted of 30 hosts, which costs us about US$1000/day. So we decided to cut it off and change it for MongoDB, where we have much more expertise on the inside. In our code we had a service class, let's call it FooService. All operations with Foo objects (write, read, stats, delete) was made via that service object, like FooService::addFoo(Foo, Bar, Baz). Any business logic class wanting to deal with Foo objects just declares FooService as is dependency, which is injected in runtime on calling business logic class constructor. So when certain code path doesn't need Foo objects - system don't have to initialise FooService at all. FooService, in turn, dependes on FooRepository, which is a descendant of abstract CassandraRepository, which has code to connect to cluster, send requests, handle errors and so on. CassandraRepository is a descendant of AbstractRepository with methods like getItemById, getItemsByIds and so on. Overall LOC count in our project is about 400k, it is not very complex, but complex enough to stuck with task like that if your architecture is not clean enough. But in our case the task was super easy: 1.Spin up MongoDB cluster with just 7 hosts (master, 3 sharded replicas and 3 shadow replicas) 2.Inherit FooNewRepository from MongoRepository, point it on new cluster. 3.Inject FooNewRepository into FooService and start performing mutable operations (create, update, delete) into both Mongo and Cassandra bound repos 4.Migrate historical data with script, which is using the same common codebase (it took about 3 weeks actually) 5.Switch reads to FooNewRepository 6.Check that everyting is okay 7.Drop all Cassandra-related code from codebase and rename FooNewRepository to just FooRepository Because FooService declares it's repository dependencies as AbstractRepository, ALL the new code except FooRepository wass looking EXACTLY the same as before. The team noticed changes only due to our weekly all-hands, where we discussing all such matters. All the coding took about 2 days net, most of it was a migrating script, so with about US$750/day cost cut we adjusted that work in a matter of days. And now the cherry on a cake. When we tried to shutdown a Cassandra cluster, we've got errors on another instance of our service, which used the same database, but was forked about 6 months ago to temporarily serve another country/language. We turned Cassandra back on, I checked out that fork, cherry-picked only Cassandra->Mongo commits from master branch, run the migration script (there were much less data for that country), and voila - all tests passed like a charm, new code working now (we will migrate it to full master branch later). edit: formatting and grammar

craftinator 7y ago

I agree; it seems that SOLID does a good job of marshalling the complexity out of individual elements into the system as a whole. This can be a good thing, if the system design itself was created with SOLID in mind, and is well implemented. However if the system design didn't account for this complexity shift, it will likely cause a lot of mayhem.

gdy 7y ago
"if the system design itself was created with SOLID in mind" Could you possibly elaborate what you mean by that?
- craftinator 7y ago
  Yes! In part I kept that ambiguous, because I've observed many edge cases and was trying to capture the overarching theme. The core idea is that often a project is designed without respect to the "design guidelines" that developers are asked to adhere to during implementation. I see SOLID as a set of design AND implementation guidelines; it contains rules for both the global and local systems. I've observed that often, the design of a system framework ignores the design portion, but requires the implementation portion (I've done it myself, and wondered why my project turned into a hot mess even though I followed all the rules!). With SOLID, complexity is pushed out of individual components and into the global scope. Instead of having a component manage multiple responsibilities, it must have only a single responsibility; now you have multiple components each managing single responsibilities. Some problems that can arise from this are multiple classes that have shared or overlapping responsibilities, classes that only handle edge cases, data innaccesibility, data and code duplication... This is different than (but related to) abstraction, in that you aren't hiding implementation, instead you are dividing it. With that division comes some non-trivial overhead, which the global design of the system needs to have planned for. This is not necessarily a bad thing; it reduces the burden of designing each individual component, while increasing the burden of designing the global system to manage how the components must interact. If, in designing a system using the SOLID principles, you do not plan for how much of the complexity is being pushed towards the global scope, you can have a huge codebase that isn't cohesive enough to function as a whole. That might've been too much elaboration, but it is kind of difficult to explain a portion without being thorough.