5 ms·

Is there really that big of an advantage to avoiding the network boundary though?

gen220 6y ago
I think there used to be, before "off-the-shelf" RPC frameworks, service discovery, and the like were mature. There still are, for very small companies. In 2020, if you have an eng count of >50: you use gRPC, some sort of service discovery solution (consul, envoy, whatever), and you basically never have to think about the costs of network hops. Opentracing is also pretty mature these days, although in my experience it's never been necessary when I can read the source of the services my code depends on. Network boundaries are really useful for enforcing interface boundaries, because we should trust N>50 programmers to correctly-implement bounded contexts as much as we trust PG&E to maintain the power grid. That being said, if you have a small, crack team, bounded contexts will take you all the way there and you don't need network boundaries to enforce them.
nthj 6y ago
Absolutely: * Avoid network and JSON serialization overhead * Perform larger refactorings or renamings without considering deployment staggering or API versioning * testing locally is far easier * Debugging in production is far easier * Useful error stack traces are included for free * Avoid (probable in my experience, at least in larger security software organizations) dependency on SecOps to make network changes to support a refactoring or introducing new components If an organization is or will pursue a FedRAMP certification, as I understand it, that organization must propose and receive approval every time data may hit a network. Avoiding the network in that case may be the difference between a 50-line MR that's merged before lunch and a multi-week process involving multiple people.
- closeparen 6y ago
  How are you getting around API versioning with independently deployable components?
  - sokoloff 6y ago
    If you call a piece of functionality from your own single deployable that you are refactoring, it’s much more like refactoring a function call than if it were an independent micro-service across a network.
- gen220 6y ago
  FWIW, I think that gRPC/protobufs have pretty compelling answers to each of the historically-valid complaints you've listed here. - cpu cycle overhead: this is valid if the overhead is very high or very important. otherwise, most companies would love to trade off cpu cycles for dev productivity. - refactorings/renamings without deployment staggering. protobufs were specifically designed with this in mind, insofar as they support deprecating fields and whatnot. However, writing a deprecatable-API is a skill, even with protos. If you have many clients and want to redo everything by scratch, you will have problems. - "testing locally" (which I take to mean integration testing locally) is the only one that requires some imagination to solve, assuming all your traffic is guarded by short-term-lease certs issued by vault or something similar. But even this is quite achievable. - error stack traces included for free: may I introduce you to context.abort(). It's not a stack trace by default, but you can actually wrap the stack trace into the message if you so-care to. opentracing isn't quite free, in a performance sense, but in a required-eng-time-to-setup-and-maintain-sense, it is pretty cheap. - dependency on secops to make network changes: I've never encountered this, but I bet you that a good platform team can provide a system where application teams effectively don't need to worry about this. It's impossible to overcome this challenge in an existing company that's used to doing things this way, though.
  - deleted 6y ago
    [deleted]
  - lmm 6y ago
    Thrift or protobuf is a huge step up from the alternatives, but you still have a lot of overhead. Generics are limited and you're essentially forced to "defunctionalise the continuation" everywhere: any time you want to pass a callback around you have to turn it into a command object instead.
    - vosper 6y ago
      People who are downvoting the parent comment: I’d love to know why? I won’t claim expertise here, but it doesn’t strike me as clearly incorrect.
    - 6y ago
- heavenlyblue 6y ago
  What is a network?
  - Spivak 6y ago
    Any application boundary that requires that you serialize your calls/requests to the other service/component in some form. Any form of IPC basically.
jurre 6y ago
It depends, but it means you don’t have to serialize/deserialize data, deal with slow connections, retries, network failures, circuit breakers etc
twunde 6y ago
It depends on your speed requirements and whether calls are being sent async or not. Also keep in mind that even with internal apis, an api call is usually multiple network boundaries (service1 --> service2 (potential DNS lookup) --> WAF/security proxy --> Firewall --> Load balancer --> SSL handshake --> server/container firewall --> server/container). Then you get into whether the service you're calling calls other apis etc. You can quickly burn 50ms or more with multiple hops. If you're trying to return responses within 200ms you now have very little margin.
- gen220 6y ago
  Acknowledging that there are indeed many hops, I think it might be a bit disingenuous to say 50ms is easy to burn, depending on what p-value we're talking about. IIRC, a typical round trip service call at my current place of work (gRPC with protobufs, vault/ssl for verification, consul for dns, etc) carries a p99 minimum latency (i.e. returning a constant) of around 2ms. A cold roundtrip obviously takes longer (because DNS, ssl, etc). It depends on how many 9's you want within 10ms, but there are various simple tricks (transparent to the application developer) that a platform team can apply to get you there. As a sidenote on calling other APIs, my anecdata suggests that most companies microservice call graphs are at most 3-4 services deep, with the vast majority being 1-2 services deep. This doesn't show the call graph, but it does demonstrate how many companies end up building a handful of services that gatekeep the core data models, and the rest simply compose over those services: https://twitter.com/adrianco/status/441883572618948608/photo/1 https://twitter.com/adrianco/status/441883572618948608/photo...

I think there used to be, before "off-the-shelf" RPC frameworks, service discovery, and the like were mature. There still are, for very small companies. In 2020, if you have an eng count of >50: you use gRPC, some sort of service discovery solution (consul, envoy, whatever), and you basically never have to think about the costs of network hops. Opentracing is also pretty mature these days, although in my experience it's never been necessary when I can read the source of the services my code depends on. Network boundaries are really useful for enforcing interface boundaries, because we should trust N>50 programmers to correctly-implement bounded contexts as much as we trust PG&E to maintain the power grid. That being said, if you have a small, crack team, bounded contexts will take you all the way there and you don't need network boundaries to enforce them.

nthj 6y ago

Absolutely: * Avoid network and JSON serialization overhead * Perform larger refactorings or renamings without considering deployment staggering or API versioning * testing locally is far easier * Debugging in production is far easier * Useful error stack traces are included for free * Avoid (probable in my experience, at least in larger security software organizations) dependency on SecOps to make network changes to support a refactoring or introducing new components If an organization is or will pursue a FedRAMP certification, as I understand it, that organization must propose and receive approval every time data may hit a network. Avoiding the network in that case may be the difference between a 50-line MR that's merged before lunch and a multi-week process involving multiple people.

closeparen 6y ago
How are you getting around API versioning with independently deployable components?
- sokoloff 6y ago
  If you call a piece of functionality from your own single deployable that you are refactoring, it’s much more like refactoring a function call than if it were an independent micro-service across a network.
gen220 6y ago
FWIW, I think that gRPC/protobufs have pretty compelling answers to each of the historically-valid complaints you've listed here. - cpu cycle overhead: this is valid if the overhead is very high or very important. otherwise, most companies would love to trade off cpu cycles for dev productivity. - refactorings/renamings without deployment staggering. protobufs were specifically designed with this in mind, insofar as they support deprecating fields and whatnot. However, writing a deprecatable-API is a skill, even with protos. If you have many clients and want to redo everything by scratch, you will have problems. - "testing locally" (which I take to mean integration testing locally) is the only one that requires some imagination to solve, assuming all your traffic is guarded by short-term-lease certs issued by vault or something similar. But even this is quite achievable. - error stack traces included for free: may I introduce you to context.abort(). It's not a stack trace by default, but you can actually wrap the stack trace into the message if you so-care to. opentracing isn't quite free, in a performance sense, but in a required-eng-time-to-setup-and-maintain-sense, it is pretty cheap. - dependency on secops to make network changes: I've never encountered this, but I bet you that a good platform team can provide a system where application teams effectively don't need to worry about this. It's impossible to overcome this challenge in an existing company that's used to doing things this way, though.
- deleted 6y ago
  [deleted]
- lmm 6y ago
  Thrift or protobuf is a huge step up from the alternatives, but you still have a lot of overhead. Generics are limited and you're essentially forced to "defunctionalise the continuation" everywhere: any time you want to pass a callback around you have to turn it into a command object instead.
  - vosper 6y ago
    People who are downvoting the parent comment: I’d love to know why? I won’t claim expertise here, but it doesn’t strike me as clearly incorrect.
  - 6y ago
heavenlyblue 6y ago
What is a network?
- Spivak 6y ago
  Any application boundary that requires that you serialize your calls/requests to the other service/component in some form. Any form of IPC basically.

jurre 6y ago

It depends, but it means you don’t have to serialize/deserialize data, deal with slow connections, retries, network failures, circuit breakers etc

twunde 6y ago

It depends on your speed requirements and whether calls are being sent async or not. Also keep in mind that even with internal apis, an api call is usually multiple network boundaries (service1 --> service2 (potential DNS lookup) --> WAF/security proxy --> Firewall --> Load balancer --> SSL handshake --> server/container firewall --> server/container). Then you get into whether the service you're calling calls other apis etc. You can quickly burn 50ms or more with multiple hops. If you're trying to return responses within 200ms you now have very little margin.

gen220 6y ago
Acknowledging that there are indeed many hops, I think it might be a bit disingenuous to say 50ms is easy to burn, depending on what p-value we're talking about. IIRC, a typical round trip service call at my current place of work (gRPC with protobufs, vault/ssl for verification, consul for dns, etc) carries a p99 minimum latency (i.e. returning a constant) of around 2ms. A cold roundtrip obviously takes longer (because DNS, ssl, etc). It depends on how many 9's you want within 10ms, but there are various simple tricks (transparent to the application developer) that a platform team can apply to get you there. As a sidenote on calling other APIs, my anecdata suggests that most companies microservice call graphs are at most 3-4 services deep, with the vast majority being 1-2 services deep. This doesn't show the call graph, but it does demonstrate how many companies end up building a handful of services that gatekeep the core data models, and the rest simply compose over those services: https://twitter.com/adrianco/status/441883572618948608/photo/1 https://twitter.com/adrianco/status/441883572618948608/photo...