3 ms·

> you have to standardize build, deployment, configuration, serialization, logging, and many "little" things that are essential but secondary to the application

> you have to standardize build, deployment, configuration, serialization, logging, and many "little" things that are essential but secondary to the application ...so then you can go reimplementing those standards in 25 different services written in 11 different languages. Sounds like fun! I don't think it's possible to just write a library/framework that would encapsulate all of those standards and re-use it in different services because, again: different languages.

nerdix 4y ago
A standardized build and deployment process can be done in a language agnostic way with containers. Every service defines its own Dockerfile. The CI/CD process just runs docker build and whatever process is needed to pull an image and run it in whatever environment. Configuration is either done with ENVs or with some standard configuration service (which is an API call that can made on service start) Logging is just standard out in the container. Each team uses whatever logging library is appropriate for their language. Only requirement is that it supports logging to console. There are a plethora of options for shipping those logs to somewhere centralized (fluentd/fluentbit, logstash, etc) Some of this stuff can also be done with sidecars (see something like Dapr) So, no, you can't write a library but you can standardize.
photonbeam 4y ago
You’d have to force everybody to use only one or two languages. For a time at Uber the direction was go/java only
- JAlexoid 4y ago
  It's a rational decision, to standardize on programming languages and paradigms. Having more than two programming languages is honestly a nightmare to deal with
- gen220 4y ago
  I worked for a company that did microservices well, and this was the norm, too. The term in my head is "golden path" or "sandbox". The languages were Go and Python, speaking protobufs over gRPC. Developing software outside that sandbox was not disallowed, but you were "on your own" in terms of infra support if you chose to do so. FWIW, the main "tricks" we found were (1) using a good build tool (2) use a good ci/cd tool (3) use a monorepo. There were some minor downsides, and it took a long time for tooling capabilities to line up with our ambitions. But when I left, we had 200-300 engineers happily deploying to `main` across 100ish services every day.
  - philote 4y ago
    Just curious, why is using a monorepo a useful trick? I would think it'd be better to have internal libraries that provide common functionality across services, and have a repo for each service. Otherwise, you're deploying code changes for one service that could, in theory, mess with another service that you don't maintain.
    - gen220 4y ago
      In a phrase, it's having a single "bleeding edge" for the entire company, vs 1 bleeding edge per service. Some benefits of this include: You have one commit hash in one repo that tells you what version service/consumer X is expecting / providing. If you want to understand why a service isn't working as expected, it's trivial to grep its implementation and contribute solutions. As a service owner, you can grep for all places where your service's client is initialized, and update them in one PR (vs 12 PRs in 12 repos, that you have to manage independently). Basically, it reduces the coordination cost of breaking changes. --- You can get these benefits in a multi-repo setup only if you have adequate tooling around multi-repo PRs, code search, etc. It's not impossible, but the barrier to highly effective work is higher, imo.
      - deleted 4y ago
        [deleted]

nerdix 4y ago

A standardized build and deployment process can be done in a language agnostic way with containers. Every service defines its own Dockerfile. The CI/CD process just runs docker build and whatever process is needed to pull an image and run it in whatever environment. Configuration is either done with ENVs or with some standard configuration service (which is an API call that can made on service start) Logging is just standard out in the container. Each team uses whatever logging library is appropriate for their language. Only requirement is that it supports logging to console. There are a plethora of options for shipping those logs to somewhere centralized (fluentd/fluentbit, logstash, etc) Some of this stuff can also be done with sidecars (see something like Dapr) So, no, you can't write a library but you can standardize.

photonbeam 4y ago

You’d have to force everybody to use only one or two languages. For a time at Uber the direction was go/java only

JAlexoid 4y ago
It's a rational decision, to standardize on programming languages and paradigms. Having more than two programming languages is honestly a nightmare to deal with
gen220 4y ago
I worked for a company that did microservices well, and this was the norm, too. The term in my head is "golden path" or "sandbox". The languages were Go and Python, speaking protobufs over gRPC. Developing software outside that sandbox was not disallowed, but you were "on your own" in terms of infra support if you chose to do so. FWIW, the main "tricks" we found were (1) using a good build tool (2) use a good ci/cd tool (3) use a monorepo. There were some minor downsides, and it took a long time for tooling capabilities to line up with our ambitions. But when I left, we had 200-300 engineers happily deploying to `main` across 100ish services every day.
- philote 4y ago
  Just curious, why is using a monorepo a useful trick? I would think it'd be better to have internal libraries that provide common functionality across services, and have a repo for each service. Otherwise, you're deploying code changes for one service that could, in theory, mess with another service that you don't maintain.
  - gen220 4y ago
    In a phrase, it's having a single "bleeding edge" for the entire company, vs 1 bleeding edge per service. Some benefits of this include: You have one commit hash in one repo that tells you what version service/consumer X is expecting / providing. If you want to understand why a service isn't working as expected, it's trivial to grep its implementation and contribute solutions. As a service owner, you can grep for all places where your service's client is initialized, and update them in one PR (vs 12 PRs in 12 repos, that you have to manage independently). Basically, it reduces the coordination cost of breaking changes. --- You can get these benefits in a multi-repo setup only if you have adequate tooling around multi-repo PRs, code search, etc. It's not impossible, but the barrier to highly effective work is higher, imo.
    - deleted 4y ago
      [deleted]