6 ms·

Locking down this network of services is a massive security improvement and they've used some very neat ways of achieving it. Overall, I really appreciate them

• segmondy 7y ago
  Code libraries in microservices can lead to a distributed monolith. each service needs to be be able to evolve independently. if you end up with a library where an update will require more than one service to be updated at the same time, then you are losing the advantage of microservices. nevertheless 1500 is a lot, but if you think it's a lot, wait till someone builds out their entire business with lambda/serveless. :-)
  • p10jkle 7y ago
    Yeah, our (mostly) ban on shared code is a major reason we end up with a lot of services; there's even service.business-day which tells you if a day is a working day
    • d4nt 7y ago
      I’m interested in the term “distributed monolith” and why shared libraries are bad, can you elaborate? (I’m an experienced dev who’s never worked on micro services). Say a service that consumed 6 other services became a service that consumed 3 other services and 3 libraries. The interfaces were the same, the libraries are consumed via package management and have good test coverage. Why is that worse? I’m actually not surprised you have a service for handling working days. The list of UK bank holidays should ideally be stored in one place, with everything else consuming that. But not all of the 1500 services have their own data stores, do they?
      • p10jkle 7y ago
        The common idea is that if you have to regularly roll out all your services, then you still basically have a monolith. So if you have a lot of shared code in libaries, there are no benefits to microservices. In our case, maybe we do have core libaries like to read from Cassandra, but you don't necessarily have to roll out everything if you improve them. Some pods could be even a year old All the services have their own keyspace in Cassandra, there is near-zero cross service data access
        • virgilp 7y ago
          That's not what makes a distributed monolith, IMO. Or to put it differently, it's not what makes a "monolith" be equal to "bad" in people minds. You want to avoid the badness, don't want to avoid the label. So, what is bad about a monolith? The complexity. Coupling of unrelated things; because it's simple to do so, because it's right there next to you and you know how it's implemented and what it performs, so you do thing A in component X knowing/relying on the fact that component Y will do the thing B, because that's how it's implemented (not because it's inherent unavoidable business logic). How do you end up with a distributed monolith? E.g. assume that microservice B will call microservice C only after you (i.e. microservice A) had a chance to call C first. Yes your code is distributed, but you don't get the benefit of logic segregation between the modules - you just get all the downsides of distributed systems. Do note that my scenario requires no shared code or library. Not sharing code doesn't save you from the distributed monolith - careful thought does.
          • jacquesm 7y ago
            Amen. Lots of micro service engineering is at the level of cargo cults and label avoidance. There are companies that do it well but relatively few of those. Whether TFA is a good or a bad example can not be told from the article contents, it would need a lot more information.
      • dsirola 7y ago
        I think the more common term is distributed ball of mud
    • viraptor 7y ago
      How do you deal with passing common data between services? For example, does every service checking the holiday endpoint need to write its own access/deserialisation/mapping starting from http or some other RPC layer? If yes, how do you keep that in sync/compatible? Do you have an explosion of versions on each endpoint? This is likely trivial for the holidays endpoint, but I'm interested in the more complex ones.
      • lwb 7y ago
        This is what I want to know too -- seems like you have to have at least one common library, that is, the library that lets you write a service call in one line instead of twenty lines. IME Go can be quite verbose, especially with >3 lines of error checking after every function call.
      • lossolo 7y ago
        They are probably using grpc and automatically generate clients libraries.
    • ealexhudson 7y ago
      Why (mostly) ban shared code? If you have that many microservices the deployment has to be well automated; so rolling out updated services on library update can't be difficult, and the interface to a compiled library is about as controllable as the interface to a network service. Is there a separate reason to those? Do you worry about the additional latency of additional service calls (for some trivial functions that doesn't feel terribly worthwhile)?
      • cameronbrown 7y ago
        I can imagine these shared libraries being in an inconsistent state at one time or the risks of updating thousands of services at once, would be two factors.
    • marcinzm 7y ago
      I'm curious why the investment in so many services instead of investing in improved CI/CD/build tools. For example, automatically deploying services when the libraries they depend on are updated. Monitoring to automatically check if a deployment is having issues. I mean, due to security concerns you still will need to re-deploy services every so often so having great CI/CD seems a more worthwhile investment. edit: Also, for the same reason chaos engineering, short lived certificates and periodic backup tests make sense; constantly testing out the deployment of all services makes sense to me. If you do something like deploying a service rarely then the chance of something going wrong (forgot a step, radical external changes, etc.) is decently higher.
  • random3 7y ago
    > wait till someone builds out their entire business with lambda/serveless. Perhaps you could built in a "regular" fashion and just compile it like that? You could write normal code and let the compiler do the massive work to handle all that and compile everything into somethign that distributes over the network.
    • dsirola 7y ago
      In my opinion, this is the point at which FaaS becomes viable
    • mattacular 7y ago
      Interesting idea. Did you mean to imply that such a compiler exists or just that it's theoretically possible?
      • random3 7y ago
        It's 1) theoretically possible in generalized fashion (although the language could make things easier) 2) it does exist in loose sense if you consider actor models (like someone else was mentioning)
    • lkschubert8 7y ago
      If I'm not mistaken that's how most of the actor model frameworks are supposed to work. An example is Akka. The developer writes code communicating between actors using an interface that abstracts how they are actually communicating.
• akie 7y ago
  The author wrote (somewhere) on this Twitter thread [1] that they have 150 developers, so that's 10 services for every employee. [1] https://twitter.com/JackKleeman/status/1190354757308862468 https://twitter.com/JackKleeman/status/1190354757308862468
  • tonyedgecombe 7y ago
    I bet 10% of their developers wrote 90% of the services.
    • p10jkle 7y ago
      Not really