11 ms·

Microservices, Containers and Kubernetes in Ten Minutes

384 points by old-gregg 8y ago

devcriollo 8y ago
example code ?
- CodeMage 8y ago
  There isn't any. This is essentially a thinly disguised ad for Gravity.
  - rafaelgoncalves 8y ago
    exactly, code sample would help on this ad feeling, for me diagrams are too abstract.
Datenstrom 8y ago
> In fact, you may discover that you already have a dozen of microservices deployed at your organization. Very true, robotics/avionics/etc. have been using "microservices" for a long time, nothing new here. Robot Operating System (ROS), Data Distribution Service (DDS), Lightweight Communications and Marshalling (LCM), and others all encourage that architecture.
- justinsaccount 8y ago
  Yep... any enterprise is likely full of them: ldap: authentication microservice syslog: logging microservice smtp: messaging microservice smb/nfs: file storage microservice They haven't been called that for the past 40 years, but that's what they are.
  - stingraycharles 8y ago
    Back in the day it was called Service Oriented Architecture. Then CORBA happened and that became a curse word, and guess we will repeat this when HTTP/4.0 will feature RPC?
    - ealexhudson 8y ago
      CORBA is a bit different: that's more about making remote code appear as if it were local, and having the network disappear. SOA isn't necessarily RPC: it could be message passing, for example. Async invocation like that in RPC is much tougher. I do think the distinction between "services" and "microservices" is a bit overblown. Clearly something like IMAP is not a microservice - it contains auth, storage, search and a few other pieces. Calling LDAP a microservice is potentially a stretch for similar reasons. But fundamentally, the concerns and architecture of both are extremely similar.
      - euske 8y ago
        > I do think the distinction between "services" and "microservices" is a bit overblown. I just realized that adding micro- or nano- or -oriented is a great way to create a buzzword in the current climate. Say, microsecurity, nanolearning, or anger-oriented user interface.
  - Thaxll 8y ago
    Except that they don't really talk to each other and if they do it's using custom protocols. Also I wouldn't call those "micro".
    - justinsaccount 8y ago
      They don't talk to each other? So you've never seen an smtp server store its spool on an nfs mount, authenticate against ldap, and log to a remote syslog server?
    - sbov 8y ago
      A microservice doesn't have to be micro! They are oftentimes a single bounded context in domain driven design, which can be huge.
      - newaccoutnas 8y ago
        So, a service, then?
  - entity345 8y ago
    Don't confuse applications and protocols with microservices. Microservices are services within an application.
    - cestith 8y ago
      There's every chance your MX MTA authenticates people against LDAP, uses LDAP again to figure out where to forward the mail to another system for final account storage, then the MDA stores the message on a clustered file system or an object store, then tells the logging system to log the delivery. The LDAP systems for authentication and for where the user's mail lives might be separate instances. Then the MUA connects to an IMAP proxy which talks to LDAP to authenticate and to determine where the messages for this user are stored (again, possibly different LDAP instances), then connects to an IMAP backend that retrieves the data from a clustered file system or object store. The IMAP proxy, IMAP backend, and MDA are separate systems. The object store is, likewise, a separate system. Meanwhile some of your users are using a webmail client as their MUA. That talks to an outbound-only MTA and the IMAP proxy, but it may talk directly to LDAP for authentication rather than authenticating to the mail servers first. It can pull user preferences from LDAP. It pulls their contact book from LDAP. These might be three different LDAP instances. I has a calendar app in the same page in another tab, but that talks instead to a separate CalDav server. The folder pane which updates with the number of unread messages in each folder updates through a different backend process on a different web server from the listing of mail in your current folder, which is a separate web server from the one that just fetched the content of the highlighted message into your preview pane. Meanwhile. half of these systems actually forward through another MTA which makes no final delivery decision itself but scans for spam scoring. Those messages that make it through the filtering get forwarded to another service which only scans for malware. Then those messages which pass go to the system that forwards the mail for final delivery to the user or to the remote party's mx server. All of these systems need their timestamps correct, so they all talk to an NTP service running alone on a VM or container that does nothing else. All of these systems send their logs to a central logging cluster via a defined protocol. The logging servers do nothing else. Just because your mail server might run qmail, Courier, Amavis, SpamAssassin, procmail, and mutt on one box with local storage and local logging doesn't mean that's how mail is done at scale. It's pretty clear to me how if you think of "email" as the application that it is composed of microservices.
- mikepurvis 8y ago
  I'm a ROS user (~7 years) and occasional contributor. One interesting wart you find in the ROS community is that there are two main supported ways of testing your ROS code: - Unit test the individual functions and classes using gtest or nose (this is built into catkin, this buildsystem). - Integration test your ROS APIs bringing up an actual ROS multi-process system and having a designated test node which exits to deliver the pass/fail verdict (this is managed by the "rostest" package). The much-touted advantage of microservices is supposed to be that you have these obvious service interface boundaries at which to test, but the reality in ROS land is that it's a lot of work to test there. It's work to generate the binaries or playback data, failures are harder to understand, and because you're at the mercy of RPC timing, nothing is deterministic, so your tests end up full of fudge factors and tolerances. And on top of that, the tests themselves execute way slower, since there's so much more set up and teardown. Is that a failure in the framework? Not sure, but the end result of it is that you only test at the RPC boundary that functionality which can only be tested there; everything else is stuffed into library functions that can be verified with gtest.
mathie25 8y ago
very good explanation!
gdulli 8y ago
> Small projects should not shy from the monolithic design. It offers higher productivity for smaller teams. Yes. In my next job search I'd like to find a way to feel out a team for whether they adopt new patterns reflexively and dogmatically vs. consciously and contextually.
peterwwillis 8y ago
When you should use microservices: when you have many independent two-pizza dev teams. (If you live in Pittsburgh, three-pizza teams...)
- myroon5 8y ago
  I'll bite. Amazon Pittsburgh office inside joke?
dagss 8y ago
It is possible to build balls of mud either with microservices or larger-sized services. There are pros and cons to both. But when "easier testing" (from OP) is said to be an advantage of microservices specifically...I am not sure what to say. I mean it is not like every test has to test the entire monolith. And it is not like testing each microservice in isolation is always going to be sufficient. It is entirely orthogonal. Of course a well-designed microservice is easier to test than a monolithic ball of mud, but a well designed monolith is also easier to test than a poorly designed spaghetti of microservices. It is as if some people think "good code" and "microservices" are synonyms. No. They are orthogonal. Industry is always going in circles. Fads come and go. In many ways, Cloud Functions are very similar to a horizontally scalable stateless monolith backend. When you break up services small enough, the "monolith" arises again as simply the sum of what you deploy in an organization.
- colonelpopcorn 8y ago
  It just goes to show how few folks truly test in isolation.
  - stingraycharles 8y ago
    Could you elaborate on what you mean with this? It seems like you are hinting at something which I apparently did not pick up.
  - quickthrower2 8y ago
    I don’t always, but I’ve never had the luxury of working on a greenfield project. With legacy code it’s a mammoth task to pull apart the spaghetti (more like a drawer of tangled headphones) to get the ideal test.
  - jimmy1 8y ago
    A better way of stating it, "It goes to show how few folks truly value testing in isolation" that is, having good tests in isolation is seen as a nice ideal to pursue, but delivering working code first trumps all. "How do you know it works?" is a common reply to this line of reasoning, obviously we all have an idea of what "working" looks like, it's just whether or not we have gone the extra step of formalizing it in the form of tests.
chuhnk 8y ago
If you want to build microservices on top of kubernetes I'd recommend go-micro https://github.com/micro/go-micro https://github.com/micro/go-micro
rolleiflex 8y ago
The main question I've always had about microservices has been, how do you handle services that are truly depended on by all other services? For example, the authentication service is depended on by pretty much anything else. This is not a hypothetical question — I'm actually looking for a better way to solve this right now in Aether's business infrastructure right now.
- jsty 8y ago
  What's the particular issue you're running into? A few general notes though: - Uptime SLAs will probably be very high for such a central service, so HA backing database choices and / or read-only replicas. For authn & authz caching session tokens / API keys / policy decisions / ... is fairly essential to not overload your DB and keep latency down to acceptable levels. - Work out what latency budget you can afford given that every user action is going to have to go through this service, possibly multiple times. Stop the world garbage collected languages probably aren't ideal here. Judicious use of caching goes a long way. - Have load shedding / circuit breaking / per-service quota mechanisms in place to prevent issues cascading around your systems. Exponential back-off is a lifesaver. - Have good integration tests to catch regressions (functionality & performance), roll-out / roll-back mechanisms to catch the ones you miss. Test these with known-bad changes every once in a while. I've recently been doing a load of development on similar systems, so happy to give my two cents - email is in profile if you'd prefer. The two SRE books are a worthwhile read.
- entity345 8y ago
  Authentication is a service that is obviously used by other microservices, otherwise it would have no use. But with microservices coupling is as loose as possible. Your authentication microservice provides an interface (REST or whatever) independent on the underlying implementation and it does not matter how it implemented. As long has the interface 'contract' is fulfilled you can re-implement it from Java to Rails, or move it from here to there, if you wish and that is completely transparent to all other microservices.
- thorgaardian 8y ago
  This is a problem I've seen at every company I've ever worked at. Everyone starts off thinking they'll have independence, but it's only natural to build on what already exists and thus you end up with complex dependency topologies. Large software companies like Google/Facebook home build their own opinionated frameworks for publishing services that include citing of dependencies via config files. Internal engines then scrape for these configs and manage the relationship topology across environments. As far as SWEs are concerned its like magic. I'm working on trying to standardize such a framework, https://docs.architect.io https://docs.architect.io, and would love preliminary feedback.
thinkersilver 8y ago
There's so much to comment on in this article. First off gravity looks like an interesting product, it's shame a blog article on that hadn't made it onto the front page. I hate to be nit-picky but I do feel that articles like this do more harm than good by oversimplifying a fairly advanced architectural pattern by downplaying the testing, resiliency and deployment approaches. * easier to test - this is flat out wrong. Your application or system logic is spread out across multiple process boundaries - the only way to test it, is to deploy the dependent services on your dev machine and test the set or design your services so that all the application logic can run in a single process. Think spark and spark workers that move from a thread to a distributed model through configuration. Application logic can be tested with this approach, but not necessarily system behaviour (which can be simulated) * rapid and flexible deployment models - in a large microservices fleet where code is managed by multiple teams and sits in several repositories the dependencies are not explicit (I can not do a "find usages" on a API call and see all microservices that are using it) - so deployments can be decoupled but there tends to be lots of breakages unless you have sophisticated well-thought out testing (see first point). * resiliency - I'm not going to expand on that because 1st and 2nd point allude to a brittle system and hence reduced resiliency. There's also data and transnational boundaries across services which need to be addressed to. To be fair, there is a hint to solving this problem through something like kakfa but it isn't called out. Microservices can be simpler with good tooling. Kubernetes is only a very small part of it. Maybe a follow up article may be good to clarify on what tooling can be provided for 1) better testing 2) reliable deployments 3) greater resiliency.
- old-gregg 8y ago
  OP here. Largely agree with your clarifications. Your comment is a fantastic example of how nuanced the real world is. Maybe it was a bit naive on my part to even try to compress these topics into a 10 minute read, but for anybody who wants to learn more, comments like these are pure gold. (and gave me a few ideas for what to write about) Thank you.
  - pergadad 8y ago
    As a total amateur I found your post a very helpful introduction. Thanks!
rconti 8y ago
The Amazon example reminds me of a former coworker telling me about how their processes ran back in '99-ish. Every webserver process was huge (128MB sticks in my head, if I have the period-correct units straight), and only lived for the length of the session before being terminated, due to memory leaks and the like.
coleifer 8y ago
How do you want your spaghetti served? Baked-in the code, or the rpc variety?
pilif 8y ago
> higher overall resiliency. yes. because making a page render dependent on the availability of 10s of independent network services is going to greatly improve your resiliency. I guess it's a matter of perspective. If you are willing to count a partially working experience as still working, then yes, you might be able to say you're more resilient because only, say the service to add a product to the basket is down - you can still look at products, so technically, you're still up. That's not how availability is measured normally though. In my (and definitely in my users) book, a site where some stuff is broken is broken. Period. And if that's the environment you're working in, you're really not improving matters by adding unnecessary network layers between components of your application.
mtarnovan 8y ago
You should not ask: "When to use Microservices?" if your previous question was: "What is the difference between microservices and containers?"
nfrankel 8y ago
> Small projects should not shy from the monolithic design. It offers higher productivity for smaller teams Wow. Has the author any real-world experience from using micro-services? Or is it just word-of-mouth? Because I have a complete different take on this.

384 points by old-gregg 8y ago

devcriollo 8y ago

example code ?

CodeMage 8y ago
There isn't any. This is essentially a thinly disguised ad for Gravity.
- rafaelgoncalves 8y ago
  exactly, code sample would help on this ad feeling, for me diagrams are too abstract.

Datenstrom 8y ago

> In fact, you may discover that you already have a dozen of microservices deployed at your organization. Very true, robotics/avionics/etc. have been using "microservices" for a long time, nothing new here. Robot Operating System (ROS), Data Distribution Service (DDS), Lightweight Communications and Marshalling (LCM), and others all encourage that architecture.

justinsaccount 8y ago
Yep... any enterprise is likely full of them: ldap: authentication microservice syslog: logging microservice smtp: messaging microservice smb/nfs: file storage microservice They haven't been called that for the past 40 years, but that's what they are.
- stingraycharles 8y ago
  Back in the day it was called Service Oriented Architecture. Then CORBA happened and that became a curse word, and guess we will repeat this when HTTP/4.0 will feature RPC?
  - ealexhudson 8y ago
    CORBA is a bit different: that's more about making remote code appear as if it were local, and having the network disappear. SOA isn't necessarily RPC: it could be message passing, for example. Async invocation like that in RPC is much tougher. I do think the distinction between "services" and "microservices" is a bit overblown. Clearly something like IMAP is not a microservice - it contains auth, storage, search and a few other pieces. Calling LDAP a microservice is potentially a stretch for similar reasons. But fundamentally, the concerns and architecture of both are extremely similar.
    - euske 8y ago
      > I do think the distinction between "services" and "microservices" is a bit overblown. I just realized that adding micro- or nano- or -oriented is a great way to create a buzzword in the current climate. Say, microsecurity, nanolearning, or anger-oriented user interface.
- Thaxll 8y ago
  Except that they don't really talk to each other and if they do it's using custom protocols. Also I wouldn't call those "micro".
  - justinsaccount 8y ago
    They don't talk to each other? So you've never seen an smtp server store its spool on an nfs mount, authenticate against ldap, and log to a remote syslog server?
  - sbov 8y ago
    A microservice doesn't have to be micro! They are oftentimes a single bounded context in domain driven design, which can be huge.
    - newaccoutnas 8y ago
      So, a service, then?
- entity345 8y ago
  Don't confuse applications and protocols with microservices. Microservices are services within an application.
  - cestith 8y ago
    There's every chance your MX MTA authenticates people against LDAP, uses LDAP again to figure out where to forward the mail to another system for final account storage, then the MDA stores the message on a clustered file system or an object store, then tells the logging system to log the delivery. The LDAP systems for authentication and for where the user's mail lives might be separate instances. Then the MUA connects to an IMAP proxy which talks to LDAP to authenticate and to determine where the messages for this user are stored (again, possibly different LDAP instances), then connects to an IMAP backend that retrieves the data from a clustered file system or object store. The IMAP proxy, IMAP backend, and MDA are separate systems. The object store is, likewise, a separate system. Meanwhile some of your users are using a webmail client as their MUA. That talks to an outbound-only MTA and the IMAP proxy, but it may talk directly to LDAP for authentication rather than authenticating to the mail servers first. It can pull user preferences from LDAP. It pulls their contact book from LDAP. These might be three different LDAP instances. I has a calendar app in the same page in another tab, but that talks instead to a separate CalDav server. The folder pane which updates with the number of unread messages in each folder updates through a different backend process on a different web server from the listing of mail in your current folder, which is a separate web server from the one that just fetched the content of the highlighted message into your preview pane. Meanwhile. half of these systems actually forward through another MTA which makes no final delivery decision itself but scans for spam scoring. Those messages that make it through the filtering get forwarded to another service which only scans for malware. Then those messages which pass go to the system that forwards the mail for final delivery to the user or to the remote party's mx server. All of these systems need their timestamps correct, so they all talk to an NTP service running alone on a VM or container that does nothing else. All of these systems send their logs to a central logging cluster via a defined protocol. The logging servers do nothing else. Just because your mail server might run qmail, Courier, Amavis, SpamAssassin, procmail, and mutt on one box with local storage and local logging doesn't mean that's how mail is done at scale. It's pretty clear to me how if you think of "email" as the application that it is composed of microservices.
mikepurvis 8y ago
I'm a ROS user (~7 years) and occasional contributor. One interesting wart you find in the ROS community is that there are two main supported ways of testing your ROS code: - Unit test the individual functions and classes using gtest or nose (this is built into catkin, this buildsystem). - Integration test your ROS APIs bringing up an actual ROS multi-process system and having a designated test node which exits to deliver the pass/fail verdict (this is managed by the "rostest" package). The much-touted advantage of microservices is supposed to be that you have these obvious service interface boundaries at which to test, but the reality in ROS land is that it's a lot of work to test there. It's work to generate the binaries or playback data, failures are harder to understand, and because you're at the mercy of RPC timing, nothing is deterministic, so your tests end up full of fudge factors and tolerances. And on top of that, the tests themselves execute way slower, since there's so much more set up and teardown. Is that a failure in the framework? Not sure, but the end result of it is that you only test at the RPC boundary that functionality which can only be tested there; everything else is stuffed into library functions that can be verified with gtest.

mathie25 8y ago

very good explanation!

gdulli 8y ago

> Small projects should not shy from the monolithic design. It offers higher productivity for smaller teams. Yes. In my next job search I'd like to find a way to feel out a team for whether they adopt new patterns reflexively and dogmatically vs. consciously and contextually.

peterwwillis 8y ago

When you should use microservices: when you have many independent two-pizza dev teams. (If you live in Pittsburgh, three-pizza teams...)

myroon5 8y ago
I'll bite. Amazon Pittsburgh office inside joke?

dagss 8y ago

It is possible to build balls of mud either with microservices or larger-sized services. There are pros and cons to both. But when "easier testing" (from OP) is said to be an advantage of microservices specifically...I am not sure what to say. I mean it is not like every test has to test the entire monolith. And it is not like testing each microservice in isolation is always going to be sufficient. It is entirely orthogonal. Of course a well-designed microservice is easier to test than a monolithic ball of mud, but a well designed monolith is also easier to test than a poorly designed spaghetti of microservices. It is as if some people think "good code" and "microservices" are synonyms. No. They are orthogonal. Industry is always going in circles. Fads come and go. In many ways, Cloud Functions are very similar to a horizontally scalable stateless monolith backend. When you break up services small enough, the "monolith" arises again as simply the sum of what you deploy in an organization.

colonelpopcorn 8y ago
It just goes to show how few folks truly test in isolation.
- stingraycharles 8y ago
  Could you elaborate on what you mean with this? It seems like you are hinting at something which I apparently did not pick up.
- quickthrower2 8y ago
  I don’t always, but I’ve never had the luxury of working on a greenfield project. With legacy code it’s a mammoth task to pull apart the spaghetti (more like a drawer of tangled headphones) to get the ideal test.
- jimmy1 8y ago
  A better way of stating it, "It goes to show how few folks truly value testing in isolation" that is, having good tests in isolation is seen as a nice ideal to pursue, but delivering working code first trumps all. "How do you know it works?" is a common reply to this line of reasoning, obviously we all have an idea of what "working" looks like, it's just whether or not we have gone the extra step of formalizing it in the form of tests.

chuhnk 8y ago

If you want to build microservices on top of kubernetes I'd recommend go-micro https://github.com/micro/go-micro https://github.com/micro/go-micro

rolleiflex 8y ago

The main question I've always had about microservices has been, how do you handle services that are truly depended on by all other services? For example, the authentication service is depended on by pretty much anything else. This is not a hypothetical question — I'm actually looking for a better way to solve this right now in Aether's business infrastructure right now.

jsty 8y ago
What's the particular issue you're running into? A few general notes though: - Uptime SLAs will probably be very high for such a central service, so HA backing database choices and / or read-only replicas. For authn & authz caching session tokens / API keys / policy decisions / ... is fairly essential to not overload your DB and keep latency down to acceptable levels. - Work out what latency budget you can afford given that every user action is going to have to go through this service, possibly multiple times. Stop the world garbage collected languages probably aren't ideal here. Judicious use of caching goes a long way. - Have load shedding / circuit breaking / per-service quota mechanisms in place to prevent issues cascading around your systems. Exponential back-off is a lifesaver. - Have good integration tests to catch regressions (functionality & performance), roll-out / roll-back mechanisms to catch the ones you miss. Test these with known-bad changes every once in a while. I've recently been doing a load of development on similar systems, so happy to give my two cents - email is in profile if you'd prefer. The two SRE books are a worthwhile read.
entity345 8y ago
Authentication is a service that is obviously used by other microservices, otherwise it would have no use. But with microservices coupling is as loose as possible. Your authentication microservice provides an interface (REST or whatever) independent on the underlying implementation and it does not matter how it implemented. As long has the interface 'contract' is fulfilled you can re-implement it from Java to Rails, or move it from here to there, if you wish and that is completely transparent to all other microservices.
thorgaardian 8y ago
This is a problem I've seen at every company I've ever worked at. Everyone starts off thinking they'll have independence, but it's only natural to build on what already exists and thus you end up with complex dependency topologies. Large software companies like Google/Facebook home build their own opinionated frameworks for publishing services that include citing of dependencies via config files. Internal engines then scrape for these configs and manage the relationship topology across environments. As far as SWEs are concerned its like magic. I'm working on trying to standardize such a framework, https://docs.architect.io https://docs.architect.io, and would love preliminary feedback.

thinkersilver 8y ago

There's so much to comment on in this article. First off gravity looks like an interesting product, it's shame a blog article on that hadn't made it onto the front page. I hate to be nit-picky but I do feel that articles like this do more harm than good by oversimplifying a fairly advanced architectural pattern by downplaying the testing, resiliency and deployment approaches. * easier to test - this is flat out wrong. Your application or system logic is spread out across multiple process boundaries - the only way to test it, is to deploy the dependent services on your dev machine and test the set or design your services so that all the application logic can run in a single process. Think spark and spark workers that move from a thread to a distributed model through configuration. Application logic can be tested with this approach, but not necessarily system behaviour (which can be simulated) * rapid and flexible deployment models - in a large microservices fleet where code is managed by multiple teams and sits in several repositories the dependencies are not explicit (I can not do a "find usages" on a API call and see all microservices that are using it) - so deployments can be decoupled but there tends to be lots of breakages unless you have sophisticated well-thought out testing (see first point). * resiliency - I'm not going to expand on that because 1st and 2nd point allude to a brittle system and hence reduced resiliency. There's also data and transnational boundaries across services which need to be addressed to. To be fair, there is a hint to solving this problem through something like kakfa but it isn't called out. Microservices can be simpler with good tooling. Kubernetes is only a very small part of it. Maybe a follow up article may be good to clarify on what tooling can be provided for 1) better testing 2) reliable deployments 3) greater resiliency.

old-gregg 8y ago
OP here. Largely agree with your clarifications. Your comment is a fantastic example of how nuanced the real world is. Maybe it was a bit naive on my part to even try to compress these topics into a 10 minute read, but for anybody who wants to learn more, comments like these are pure gold. (and gave me a few ideas for what to write about) Thank you.
- pergadad 8y ago
  As a total amateur I found your post a very helpful introduction. Thanks!

rconti 8y ago

The Amazon example reminds me of a former coworker telling me about how their processes ran back in '99-ish. Every webserver process was huge (128MB sticks in my head, if I have the period-correct units straight), and only lived for the length of the session before being terminated, due to memory leaks and the like.

coleifer 8y ago

How do you want your spaghetti served? Baked-in the code, or the rpc variety?

pilif 8y ago

> higher overall resiliency. yes. because making a page render dependent on the availability of 10s of independent network services is going to greatly improve your resiliency. I guess it's a matter of perspective. If you are willing to count a partially working experience as still working, then yes, you might be able to say you're more resilient because only, say the service to add a product to the basket is down - you can still look at products, so technically, you're still up. That's not how availability is measured normally though. In my (and definitely in my users) book, a site where some stuff is broken is broken. Period. And if that's the environment you're working in, you're really not improving matters by adding unnecessary network layers between components of your application.

mtarnovan 8y ago

You should not ask: "When to use Microservices?" if your previous question was: "What is the difference between microservices and containers?"

nfrankel 8y ago

> Small projects should not shy from the monolithic design. It offers higher productivity for smaller teams Wow. Has the author any real-world experience from using micro-services? Or is it just word-of-mouth? Because I have a complete different take on this.