4 ms·

It seems crazy to me that you'd have to do a HTTP(S) request to find out whether a day was a working day. A HTTP(S) call which then probably does a DB lookup ov

It seems crazy to me that you'd have to do a HTTP(S) request to find out whether a day was a working day. A HTTP(S) call which then probably does a DB lookup over the network. Just seems like an extra layer of overhead, when you could just have a library instead. I've never worked on microservices, so I may just have a fundamental misunderstanding of how granular you're supposed to be & also don't know if this is the norm. Would be nice to hear some other opinions on this.

tonyedgecombe 7y ago
I must admit I'm struggling to see what problem they are trying to solve with this. I'm not aware of a single platform where you can't modularise your code.
- rtpg 7y ago
  Though I’m against micro services for lots of reasons, there is truth to the “library update” problem If you update the business day library, then you also need to bump the version of this lib used in your 100+ “real services”, or risk some services seeing differences on this front. Imagine a bug from different logic due to version differences between libs in a distributed system. Would be a mess to debug. If you say “oh well services should just install latest” now you have the distributed monolith problem. A new version of a library could get pushed out but have a regression bringing down half your services
  - marcosdumay 7y ago
    How do you deal with incompatible updates of the services? I fail to see how they won't have the exact same problems.
    - tempay 7y ago
      You just roll back that one service and every other service picks up the change automatically. Doing the same thing with shared libraries requires redeploying everything that is dependent on the bad library (i.e. hundreds of containers instead of two or three). It's not necessarily that one model is better than the other just that using a hybrid approach often means having the issues of both with only a few (if any!) of the benefits.
    - onefuncman 7y ago
      You don't make incompatible updates, via adhering to service contracts. You can use protobuf or Avro as the schema for these service contracts and get varying levels of forward and backwards compatibility guarantees for schema changes.
    - rtpg 7y ago
      Just like when developping an API, you build changes in backwards-compatible ways for the most part. If you have to make a backwards-incompatible change, you probably need to make your service support two different API versions for a while, and hunt down services that are calling old stuff for a while. This is basically how you can move forward on improvements without coupling "improve this microservice" to "update every single other microservice that is calling to it". Though I believe part of the theory with microservices is that since you are working on such tiny surfaces, most services will never need to roll out a backwards incompatible change
p10jkle 7y ago
The network in an AWS VPC is insanely fast. It's not a big deal. If they add a new national holiday, now you have to deploy all your services. I guess microservices self perpetuate themselves in that sense
- marcinzm 7y ago
  If you're running in a multi-az environment then you're looking at 1-3ms per network request across azs (plus some $ per request).
  - p10jkle 7y ago
    Yeah we're aware of the cost particularly. We're looking into preferring inter-AZ routing with envoy
eigenrick 7y ago
Figuring out whether today is a working day is actually a dynamic problem. It is different for every locale, and it changes more frequently than one might expect. It's actually a great example of something that changes rather frequently, and it's a concern that cuts across dozens (or in Monzo's case, maybe hundreds) of services. Causing a roll-out to hundreds of critical business functions (whether hosted in a monolith or microservices) because Uganda added a holiday seems quite excessive. The rule I and my company follows is that the shape of a module should follow its deployment model and scope. Some services are global in scope (literally targeted at the earth) others are scoped to a small subset of our inner network. They should have a different code repo because they have a different rollout schedule. In the case of the working-days services. Only 1 rollout needs to occur, and it seems like a fairly safe rollout. Let's consider another service, the CriticalTransactionService, or CTS. It executes critical business transactions in a very stateful way. When deploying a new version, in order to avoid any loss of availability, often a special rollout dance must be executed. E.g. switching the master transaction writer to another region, which means changing databases from passive replication to active master, and vice-versa. One might consider this rollout "risky" and therefore only limits it to happen at 3am on a Saturday. It seems like a good idea to limit the scope of this 3am Saturday rollout only to the CriticalTransactionService, and most other services are free to deploy to prod whenever they wish.
- trollied 7y ago
  I guess what I don't get is that the service needs to be rolled out/restarted too. Doesn't the data just live in a database & there's a cache going on that can be invalidated? Or is everything just so overcomplicated these days?
  - eigenrick 7y ago
    This particular case could be a couple tables in a database. To follow proper encapsulation guidelines, it would have to be front-ended by some stored procedures, so that the underlying representation is free to change. How would you roll out this change to production? If you just swapped out tables and stored procedures, you'd have downtime. You can't just install a parallel table and stored procedures, because there is no way to tell all of the consumers to use the v2 of your functions. So you'd have to temporarily remove availability of the WorkingDays functionality. If it were deployed as a microservice, you are free to deploy a v2 of WorkingDays in parallel. When it's live, the old one goes away, and there is no loss of availability.

tonyedgecombe 7y ago

I must admit I'm struggling to see what problem they are trying to solve with this. I'm not aware of a single platform where you can't modularise your code.

rtpg 7y ago
Though I’m against micro services for lots of reasons, there is truth to the “library update” problem If you update the business day library, then you also need to bump the version of this lib used in your 100+ “real services”, or risk some services seeing differences on this front. Imagine a bug from different logic due to version differences between libs in a distributed system. Would be a mess to debug. If you say “oh well services should just install latest” now you have the distributed monolith problem. A new version of a library could get pushed out but have a regression bringing down half your services
- marcosdumay 7y ago
  How do you deal with incompatible updates of the services? I fail to see how they won't have the exact same problems.
  - tempay 7y ago
    You just roll back that one service and every other service picks up the change automatically. Doing the same thing with shared libraries requires redeploying everything that is dependent on the bad library (i.e. hundreds of containers instead of two or three). It's not necessarily that one model is better than the other just that using a hybrid approach often means having the issues of both with only a few (if any!) of the benefits.
  - onefuncman 7y ago
    You don't make incompatible updates, via adhering to service contracts. You can use protobuf or Avro as the schema for these service contracts and get varying levels of forward and backwards compatibility guarantees for schema changes.
  - rtpg 7y ago
    Just like when developping an API, you build changes in backwards-compatible ways for the most part. If you have to make a backwards-incompatible change, you probably need to make your service support two different API versions for a while, and hunt down services that are calling old stuff for a while. This is basically how you can move forward on improvements without coupling "improve this microservice" to "update every single other microservice that is calling to it". Though I believe part of the theory with microservices is that since you are working on such tiny surfaces, most services will never need to roll out a backwards incompatible change

p10jkle 7y ago

The network in an AWS VPC is insanely fast. It's not a big deal. If they add a new national holiday, now you have to deploy all your services. I guess microservices self perpetuate themselves in that sense

marcinzm 7y ago
If you're running in a multi-az environment then you're looking at 1-3ms per network request across azs (plus some $ per request).
- p10jkle 7y ago
  Yeah we're aware of the cost particularly. We're looking into preferring inter-AZ routing with envoy

eigenrick 7y ago

Figuring out whether today is a working day is actually a dynamic problem. It is different for every locale, and it changes more frequently than one might expect. It's actually a great example of something that changes rather frequently, and it's a concern that cuts across dozens (or in Monzo's case, maybe hundreds) of services. Causing a roll-out to hundreds of critical business functions (whether hosted in a monolith or microservices) because Uganda added a holiday seems quite excessive. The rule I and my company follows is that the shape of a module should follow its deployment model and scope. Some services are global in scope (literally targeted at the earth) others are scoped to a small subset of our inner network. They should have a different code repo because they have a different rollout schedule. In the case of the working-days services. Only 1 rollout needs to occur, and it seems like a fairly safe rollout. Let's consider another service, the CriticalTransactionService, or CTS. It executes critical business transactions in a very stateful way. When deploying a new version, in order to avoid any loss of availability, often a special rollout dance must be executed. E.g. switching the master transaction writer to another region, which means changing databases from passive replication to active master, and vice-versa. One might consider this rollout "risky" and therefore only limits it to happen at 3am on a Saturday. It seems like a good idea to limit the scope of this 3am Saturday rollout only to the CriticalTransactionService, and most other services are free to deploy to prod whenever they wish.

trollied 7y ago
I guess what I don't get is that the service needs to be rolled out/restarted too. Doesn't the data just live in a database & there's a cache going on that can be invalidated? Or is everything just so overcomplicated these days?
- eigenrick 7y ago
  This particular case could be a couple tables in a database. To follow proper encapsulation guidelines, it would have to be front-ended by some stored procedures, so that the underlying representation is free to change. How would you roll out this change to production? If you just swapped out tables and stored procedures, you'd have downtime. You can't just install a parallel table and stored procedures, because there is no way to tell all of the consumers to use the v2 of your functions. So you'd have to temporarily remove availability of the WorkingDays functionality. If it were deployed as a microservice, you are free to deploy a v2 of WorkingDays in parallel. When it's live, the old one goes away, and there is no loss of availability.