6 ms·

> Increasing architectural complexity to enforce boundaries is never a solution to a lack of organizational discipline, And yet we do this all the time. Your C

> Increasing architectural complexity to enforce boundaries is never a solution to a lack of organizational discipline, And yet we do this all the time. Your CI/CD blocking your PRs until tests pass? That's a costly technical solution to solve an issue of organizational discipline.

vrosas 3y ago
The other problem is that these self-imposed roadblocks are so engrained in the modern SDLC that developers literally cannot imagine a world where they do not exist. I got _reamed_ by some "senior" engineers for merging a small PR without an approval recently. And we're not some megacorp, we're a 12 person engineering startup! We can make our own rules! We don't even have any customers...
- insanitybit 3y ago
  Indeed, dogmatic adherence to arbitrary patterns is a huge problem in our field. People have strong beliefs, "X good" or "X bad", with almost no idea of what X even is, what the alternatives are, why X was something people did or did not like, etc.
  - jacquesm 3y ago
    Another problem is people making decisions with potentially far reaching consequences well above their paygrade without authorization.
    - insanitybit 3y ago
      I don't think that problem is nearly as significant.
      - jacquesm 3y ago
        What you think is usually the limit of your experience, which effectively makes it anecdata. I've looked at enough companies and dealt with the aftermath of enough such instances that I beg to differ. It's possible that due to the nature of my business my experience is skewed in the other direction which makes that anecdata as well. But it is probably more important than you think (and less important than I think).
- jacquesm 3y ago
  Your 'senior' engineer is likely right: they are trying to get some kind of process going and you are actively sabotaging that. This could come back to haunt you later on when you by your lonesome decide to merge a 'small PR' with massive downtime as a result of not having your code reviewed. Ok, you say, I'm perfect. And I believe you. But now you have another problem: the other junior devs on your team who see vrosas commit and merge stuff by themselves will see you as their shining example. And as a result they by their lonesomes decide to merge 'small PR's with massive downtime as a result. If you got _reamed_ you got off lucky: in plenty of places you'd be out on the street. It may well be that you had it right but from context as given I hope this shows you some alternative perspective that might give you pause the next time you decide to throw out the rulebook, even in emergencies - especially in emergencies - these rules are there to keep you, your team and the company safe. In regulated industries you can multiply all of that by a factor of five or so.
  - vrosas 3y ago
    I'm challenging my team to actually think about that process, why it's in place, how it's helping (or actively hurting!) us. Comparing ourselves to companies that have regulatory requirements (spoiler: we don't and likely won't for a long, long time) just furthers my point that no one really thinks about these things. They just cargo cult how everyone else does it.
    - jacquesm 3y ago
      You can challenge them without actually violating established process. I wasn't comparing you to companies that have regulatory requirements, I was merely saying that all of the above will factor in much, much stronger still in a regulated industry. But not being in a regulated industry doesn't mean there isn't a very good reason to have a code review in your process, assuming it is used effectively and not for nitpicking. Not having a code review step is usually a bad idea, unless everybody on your team is of absolutely amazing quality and they never make silly mistakes. I've yet to come across a team like that, but maybe you are the exception to the rule.
  - insanitybit 3y ago
- elzbardico 3y ago
  Really man. I have almost two decades developing software and yet, I feel a lot more comfortable having all my code reviewed. If anything I get annoyed by junior developers in my team when they just rub-stamp my PRs because supposedly I am this super senior guy that can't err. Code Reviews are supposed to give you peace of mind, not being a hassle. During all this time, I've seen plenty of "small changes" having completely unexpected consequences, and sometimes all it would take to avoid would someone else seeing it from another perspective.
  - vrosas 3y ago
    Not a man. I’m not convinced bad code would get merged more often if we didn’t require approvals. I am convinced we’d deliver code faster though, and that’s what I’m trying to optimize for. Your company and engineering problems are not the same as mine.
    - jeltz 3y ago
      Optimizing for delivering faster is usually the wrong thing to optimize for. Usually the issue is making sure the right things are delivered.
  - foobiekr 3y ago
    At 30 years of coding professionally in great engineering-focused organizations, and that on top of nearly a lifetime of having coded for myself, I’ve concluded code reviews barely work. I agree with everything you say here, but honestly it’s quite ineffective. I wish we had found something better than unit tests (often misleading and fragile) and the ability of a qualified reviewer to maintain attention and context that a real review entails.
    - vvanders 3y ago
      IMO catching bugs is a nice side-effect of code reviews. The primary value I've seen across teams has been more on having shared team context across a codebase, if something goes bump in the night you've got a sense on how that part of the codebase works. It's also a great opportunity for other engineers to ask "why" and explain parts that aren't obvious or other context that's relevant. We'll find the occasional architectural mismatch(although we like to catch those much earlier in the design process) and certainly prevented bugs from shipping but if that's the primary focus I think a team is missing a lot of the value from regular code reviews.
nevinera 3y ago
That's technical, and not architectural. I'm _all about_ technical solutions to lack of discipline, and in fact I think technical and process solutions are the only immediate way to create cultural solutions (which are the long-term ones). I'd even consider minor increases to architectural complexity for that purpose justifiable - it's a real problem, and trading to solve it is reasonable. But architectural complexity has outsized long-term cost, and service-orientation in particular has a _lot_ of it. And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined.
- jaxr 3y ago
  Agreed. isn't that why strongly typed languages made a comeback?
- insanitybit 3y ago
  Can you explain the salient distinction between a "technical" versus "architectural" solution? Candidly, I'm not convinced that there is one. > But architectural complexity has outsized long-term cost As do technical solutions, of course. CI/CD systems are very expensive, just from a monetary perspective, but also impose significant burdens to developers in terms of blocking PRs, especially if there are flaky or expensive tests. > And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined. Ignoring microservices, just focusing on underlying SoA for a moment, the boundary is the process. That is an enforceable boundary. I think what you're saying amounts to, in microservice parlance, that there is no way to prevent a single microservice from crossing multiple bounded contexts, that it ultimately relies on developers. This is true, but it's also just as true for good monolithic designs around modules - there is no technical constraint for a module to not expand into domains, becoming cluttered and overly complex. Microservices do not make that problem harder, but SoA does give you a powerful technical tool for isolation.
  - nevinera 3y ago
    > Can you explain the salient distinction between a "technical" versus "architectural" solution? Candidly, I'm not convinced that there is one. Not concisely in the general case, but in this case the difference is fairly straightforward - CI/CD doesn't affect the structure of your executing application at all, only the surrounding context. I don't want to spend the hours it would take to characterize architecture as distinct from implementation, but the vast number of textbooks on the topic all generally agree that there is one, though they draw the lines in slightly different places. > I think what you're saying amounts to, Very much no - my point is about the process of implementation. The services.. _do_ enforce boundaries, but the boundaries they enforce may not be good ones. In order to successfully extract services from a monolith, you have to go through a process that includes finding and creating those domain boundaries for the domain being extracted. If it's your first time, you might be doing that implicitly and without realizing it's what you're doing, but under the hood it's the bulk of the mental effort. The part where you actually _introduce a service_ can be anywhere from a tenth to half of the work (that fraction varies a lot by technical stack and depending on how coupled the domain in question is to the central behavioral tangle in the monolith), but by the time you've gotten the domain boundary created you've _already solved_ the original problem. Now you're facing a trade of "extract this well-factored domain out to a separate service application, to prevent its now-clear boundaries from being violated in the future". And I contend that that's a trade that should rarely be made.
mvdtnz 3y ago
No, not at all. CI/CD blocking pull requests is in place because large systems have large test suites and challenging dependencies which mean that individual developers literally can't run every test on their local machine and can often break things without realising it. It's not about organisational discipline, it's about ensuring correctness.
- bluGill 3y ago
  I can run every test on my machine if I want. It would be a manual effort, but wouldn't be hard to automate if I cared to try. However it would take about 5 days to finish. It isn't worth it when such tests rarely fail - the CI system just spin them off to many AWS nodes and if something fails then run just that test locally and I get results in a few hours (some of the tests are end to end integration that need more than half an hour). Like any good test system I have a large suite of "unit" tests that run quick that I run locally before committing code - it takes a few minutes to get high code coverage if you care about that metric. Even then I just run the tests for x86-64, if they fail on arm that is something for my CI system to figure out for me.

vrosas 3y ago

The other problem is that these self-imposed roadblocks are so engrained in the modern SDLC that developers literally cannot imagine a world where they do not exist. I got _reamed_ by some "senior" engineers for merging a small PR without an approval recently. And we're not some megacorp, we're a 12 person engineering startup! We can make our own rules! We don't even have any customers...

insanitybit 3y ago
Indeed, dogmatic adherence to arbitrary patterns is a huge problem in our field. People have strong beliefs, "X good" or "X bad", with almost no idea of what X even is, what the alternatives are, why X was something people did or did not like, etc.
- jacquesm 3y ago
  Another problem is people making decisions with potentially far reaching consequences well above their paygrade without authorization.
  - insanitybit 3y ago
    I don't think that problem is nearly as significant.
    - jacquesm 3y ago
      What you think is usually the limit of your experience, which effectively makes it anecdata. I've looked at enough companies and dealt with the aftermath of enough such instances that I beg to differ. It's possible that due to the nature of my business my experience is skewed in the other direction which makes that anecdata as well. But it is probably more important than you think (and less important than I think).
jacquesm 3y ago
Your 'senior' engineer is likely right: they are trying to get some kind of process going and you are actively sabotaging that. This could come back to haunt you later on when you by your lonesome decide to merge a 'small PR' with massive downtime as a result of not having your code reviewed. Ok, you say, I'm perfect. And I believe you. But now you have another problem: the other junior devs on your team who see vrosas commit and merge stuff by themselves will see you as their shining example. And as a result they by their lonesomes decide to merge 'small PR's with massive downtime as a result. If you got _reamed_ you got off lucky: in plenty of places you'd be out on the street. It may well be that you had it right but from context as given I hope this shows you some alternative perspective that might give you pause the next time you decide to throw out the rulebook, even in emergencies - especially in emergencies - these rules are there to keep you, your team and the company safe. In regulated industries you can multiply all of that by a factor of five or so.
- vrosas 3y ago
  I'm challenging my team to actually think about that process, why it's in place, how it's helping (or actively hurting!) us. Comparing ourselves to companies that have regulatory requirements (spoiler: we don't and likely won't for a long, long time) just furthers my point that no one really thinks about these things. They just cargo cult how everyone else does it.
  - jacquesm 3y ago
    You can challenge them without actually violating established process. I wasn't comparing you to companies that have regulatory requirements, I was merely saying that all of the above will factor in much, much stronger still in a regulated industry. But not being in a regulated industry doesn't mean there isn't a very good reason to have a code review in your process, assuming it is used effectively and not for nitpicking. Not having a code review step is usually a bad idea, unless everybody on your team is of absolutely amazing quality and they never make silly mistakes. I've yet to come across a team like that, but maybe you are the exception to the rule.
- insanitybit 3y ago
elzbardico 3y ago
Really man. I have almost two decades developing software and yet, I feel a lot more comfortable having all my code reviewed. If anything I get annoyed by junior developers in my team when they just rub-stamp my PRs because supposedly I am this super senior guy that can't err. Code Reviews are supposed to give you peace of mind, not being a hassle. During all this time, I've seen plenty of "small changes" having completely unexpected consequences, and sometimes all it would take to avoid would someone else seeing it from another perspective.
- vrosas 3y ago
  Not a man. I’m not convinced bad code would get merged more often if we didn’t require approvals. I am convinced we’d deliver code faster though, and that’s what I’m trying to optimize for. Your company and engineering problems are not the same as mine.
  - jeltz 3y ago
    Optimizing for delivering faster is usually the wrong thing to optimize for. Usually the issue is making sure the right things are delivered.
- foobiekr 3y ago
  At 30 years of coding professionally in great engineering-focused organizations, and that on top of nearly a lifetime of having coded for myself, I’ve concluded code reviews barely work. I agree with everything you say here, but honestly it’s quite ineffective. I wish we had found something better than unit tests (often misleading and fragile) and the ability of a qualified reviewer to maintain attention and context that a real review entails.
  - vvanders 3y ago
    IMO catching bugs is a nice side-effect of code reviews. The primary value I've seen across teams has been more on having shared team context across a codebase, if something goes bump in the night you've got a sense on how that part of the codebase works. It's also a great opportunity for other engineers to ask "why" and explain parts that aren't obvious or other context that's relevant. We'll find the occasional architectural mismatch(although we like to catch those much earlier in the design process) and certainly prevented bugs from shipping but if that's the primary focus I think a team is missing a lot of the value from regular code reviews.

nevinera 3y ago

That's technical, and not architectural. I'm _all about_ technical solutions to lack of discipline, and in fact I think technical and process solutions are the only immediate way to create cultural solutions (which are the long-term ones). I'd even consider minor increases to architectural complexity for that purpose justifiable - it's a real problem, and trading to solve it is reasonable. But architectural complexity has outsized long-term cost, and service-orientation in particular has a _lot_ of it. And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined.

jaxr 3y ago
Agreed. isn't that why strongly typed languages made a comeback?
insanitybit 3y ago
Can you explain the salient distinction between a "technical" versus "architectural" solution? Candidly, I'm not convinced that there is one. > But architectural complexity has outsized long-term cost As do technical solutions, of course. CI/CD systems are very expensive, just from a monetary perspective, but also impose significant burdens to developers in terms of blocking PRs, especially if there are flaky or expensive tests. > And in this particular case, it doesn't actually solve the problem, since you _can't_ successfully enforce those domain boundaries unless you already have them well-defined. Ignoring microservices, just focusing on underlying SoA for a moment, the boundary is the process. That is an enforceable boundary. I think what you're saying amounts to, in microservice parlance, that there is no way to prevent a single microservice from crossing multiple bounded contexts, that it ultimately relies on developers. This is true, but it's also just as true for good monolithic designs around modules - there is no technical constraint for a module to not expand into domains, becoming cluttered and overly complex. Microservices do not make that problem harder, but SoA does give you a powerful technical tool for isolation.
- nevinera 3y ago
  > Can you explain the salient distinction between a "technical" versus "architectural" solution? Candidly, I'm not convinced that there is one. Not concisely in the general case, but in this case the difference is fairly straightforward - CI/CD doesn't affect the structure of your executing application at all, only the surrounding context. I don't want to spend the hours it would take to characterize architecture as distinct from implementation, but the vast number of textbooks on the topic all generally agree that there is one, though they draw the lines in slightly different places. > I think what you're saying amounts to, Very much no - my point is about the process of implementation. The services.. _do_ enforce boundaries, but the boundaries they enforce may not be good ones. In order to successfully extract services from a monolith, you have to go through a process that includes finding and creating those domain boundaries for the domain being extracted. If it's your first time, you might be doing that implicitly and without realizing it's what you're doing, but under the hood it's the bulk of the mental effort. The part where you actually _introduce a service_ can be anywhere from a tenth to half of the work (that fraction varies a lot by technical stack and depending on how coupled the domain in question is to the central behavioral tangle in the monolith), but by the time you've gotten the domain boundary created you've _already solved_ the original problem. Now you're facing a trade of "extract this well-factored domain out to a separate service application, to prevent its now-clear boundaries from being violated in the future". And I contend that that's a trade that should rarely be made.

mvdtnz 3y ago

No, not at all. CI/CD blocking pull requests is in place because large systems have large test suites and challenging dependencies which mean that individual developers literally can't run every test on their local machine and can often break things without realising it. It's not about organisational discipline, it's about ensuring correctness.

bluGill 3y ago
I can run every test on my machine if I want. It would be a manual effort, but wouldn't be hard to automate if I cared to try. However it would take about 5 days to finish. It isn't worth it when such tests rarely fail - the CI system just spin them off to many AWS nodes and if something fails then run just that test locally and I get results in a few hours (some of the tests are end to end integration that need more than half an hour). Like any good test system I have a large suite of "unit" tests that run quick that I run locally before committing code - it takes a few minutes to get high code coverage if you care about that metric. Even then I just run the tests for x86-64, if they fail on arm that is something for my CI system to figure out for me.