3 ms·

For my consulting, I primarily practice "reference first architectures." The idea is we identify the rough shape of what we are going to build and the componen

For my consulting, I primarily practice "reference first architectures." The idea is we identify the rough shape of what we are going to build and the components needed to deliver it (Linux? Terraform? K8S? HTML/CSS/JS? etc.). Next we measure up what we can "take for granted" for the engineering skillset the organization hires for. Then we pick books, official project documentation, etc. that will act as our "reference." We spend our upfront time pouring ourselves into this documentation and come away with a general "philosophy" of an approach to the architecture. Then we build the architecture, updating our philosophy with our learnings along the way. At the end of the project, we commit the philosophy to paper. We deliver the system we built, the philosophy behind it, and the stack of references used to build the system. What this means is I can take any engineer at the target level they hire for, hand them the deliverable and say "go spend a week reading these, you'll come back with sufficient expertise to own this system." It also acts as documentation for myself for future contracts if I get brought back in. Prior to starting the contract I can go back in and review all of those deliverables myself to hit the ground running once I'm back on the project.

jrvarela56 4y ago
This sounds like the right way to do it. For me it has been tough to come up with principles that don't sound like they apply to any system. You start off with a generic CRUD app but as it grows the default/usual web framework constructs tend to leave you with a ball of mud. You can couple anything in there together and since you're pressed for time, you tend to do it. Abstractions feel premature and when they start emerging there's lack of conviction to push through with them and clean up the whole thing. Do you have any starter resources to come up with principles for a system? Maybe something showing how certain principles lead you to implementation choices that would've been different under another philosophy.
- r3trohack3r 4y ago
  For a 2 week Terraform audit, these are the high level philosophy points I put together. The final doc was 10 pages. Each point lists the reason for choosing this approach and any trade-offs that come with it. * Small composable Terraform modules * Don't manage IaC declarations alongside code in a polyrepo * Direnv for managing env configs across repos * Manage k8s using k8s manifests and not terraform files (kubectl provider gives us this) * Delegate flux management to flux-cli * Auto-unseal Vault to capture and protect the vault token Then a list of recommended reading: * Terraform Up and Running * Building Microservices * Site Reliability Engineering This list is more tactical since we didn't build the system, we were auditing their current setup.
  - mattacular 4y ago
    > Don't manage IaC declarations alongside code in a polyrepo Can you elaborate on this one? (And thanks for the interesting comments)
    - r3trohack3r 4y ago
      > The source code for services should not be coupled to their deployments when managed in SCM. The lifecycle of changes for infrastructure are different than the lifecycles of artifacts for services. Any artifact should be able to be configured and deployed into any (supported) infrastructure configuration. For example, running git revert on a service should be able to yield a deployable artifact regardless of how the infrastructure is configured. By coupling these, you tie changes to infrastructure to changes in services. A rollback for your service can also unintentionally rollback how that service gets deployed – and avoiding that requires an engineer hold both the context of the infrastructure and the context of the service in their head whenever they are manging git history. It becomes difficult to deploy an older version of a service for testing. It also breaks git-bisect since, now, searching for a regression in software also changes how that software is getting deployed. This is an extension of managing IaC as small composable modules. The source code for a service should, itself, be viewed as a composable module. That module may take on a different format than other pieces of IaC (i.e. a .tar.gz, a .deb, or a docker image instead of a terraform module or a terraform provider) – but it’s API contracts are still drawn around the unit of deployment and not the monolithic infrastructure stack that will be deployed with it. This, of course, does not apply to projects using a monorepo.
      - perfmode 4y ago
        how would one manage independent infra and services in a monorepo?
        deleted 4y ago
        [deleted]
        r3trohack3r 4y ago
amw-zero 4y ago
Sounds like an architecture decision record. Here's an example ADR template: https://github.com/joelparkerhenderson/architecture-decision-record/blob/main/templates/decision-record-template-by-michael-nygard/index.md https://github.com/joelparkerhenderson/architecture-decision....
deleted 4y ago
[deleted]

jrvarela56 4y ago

This sounds like the right way to do it. For me it has been tough to come up with principles that don't sound like they apply to any system. You start off with a generic CRUD app but as it grows the default/usual web framework constructs tend to leave you with a ball of mud. You can couple anything in there together and since you're pressed for time, you tend to do it. Abstractions feel premature and when they start emerging there's lack of conviction to push through with them and clean up the whole thing. Do you have any starter resources to come up with principles for a system? Maybe something showing how certain principles lead you to implementation choices that would've been different under another philosophy.

r3trohack3r 4y ago
For a 2 week Terraform audit, these are the high level philosophy points I put together. The final doc was 10 pages. Each point lists the reason for choosing this approach and any trade-offs that come with it. * Small composable Terraform modules * Don't manage IaC declarations alongside code in a polyrepo * Direnv for managing env configs across repos * Manage k8s using k8s manifests and not terraform files (kubectl provider gives us this) * Delegate flux management to flux-cli * Auto-unseal Vault to capture and protect the vault token Then a list of recommended reading: * Terraform Up and Running * Building Microservices * Site Reliability Engineering This list is more tactical since we didn't build the system, we were auditing their current setup.
- mattacular 4y ago
  > Don't manage IaC declarations alongside code in a polyrepo Can you elaborate on this one? (And thanks for the interesting comments)
  - r3trohack3r 4y ago
    > The source code for services should not be coupled to their deployments when managed in SCM. The lifecycle of changes for infrastructure are different than the lifecycles of artifacts for services. Any artifact should be able to be configured and deployed into any (supported) infrastructure configuration. For example, running git revert on a service should be able to yield a deployable artifact regardless of how the infrastructure is configured. By coupling these, you tie changes to infrastructure to changes in services. A rollback for your service can also unintentionally rollback how that service gets deployed – and avoiding that requires an engineer hold both the context of the infrastructure and the context of the service in their head whenever they are manging git history. It becomes difficult to deploy an older version of a service for testing. It also breaks git-bisect since, now, searching for a regression in software also changes how that software is getting deployed. This is an extension of managing IaC as small composable modules. The source code for a service should, itself, be viewed as a composable module. That module may take on a different format than other pieces of IaC (i.e. a .tar.gz, a .deb, or a docker image instead of a terraform module or a terraform provider) – but it’s API contracts are still drawn around the unit of deployment and not the monolithic infrastructure stack that will be deployed with it. This, of course, does not apply to projects using a monorepo.
    - perfmode 4y ago
      how would one manage independent infra and services in a monorepo?
      - deleted 4y ago
        [deleted]
      - r3trohack3r 4y ago

amw-zero 4y ago

Sounds like an architecture decision record. Here's an example ADR template: https://github.com/joelparkerhenderson/architecture-decision-record/blob/main/templates/decision-record-template-by-michael-nygard/index.md https://github.com/joelparkerhenderson/architecture-decision....

deleted 4y ago

[deleted]