6 ms·

Network is a high performance system, and each layer you add adds latency. Consider a traditional monolithic application. In comes your HTTP request in one end

by lykr0n 8y ago

Network is a high performance system, and each layer you add adds latency. Consider a traditional monolithic application. In comes your HTTP request in one end, a bunch of cross thread communication happens, and database queries come out the other end. With that, you have 2 points of network communication. Now with a micro-service, you might have 4 or 5 applications that are needed to replace the above monolith. Throw in a service mesh on top of your cloud providers SDN, you've turned 2 points of network communication into 20 or more. The 5 micro-services talking to each other and the service meshes talking to each other. Add on top the additional processing overhead of maybe 1 to 2ms, you've just added at best 10ms round trip time to get to your databases and some more CPU. And to what benefit? TLS? You can do this in your application, or trust your private network is private. Tracing? You can do this with PID matching and watching the kernel's networking stack.

devereaux 8y ago
So true. For some low latency applications, anything above the bare minimal virtualization is not acceptable. For what I do, in theory, many things should not impact results. In practice, anything that upon measurement impact results is stripped away. Think A/B testing but for every single component - including the major version of say the python interpreter. That's how you end up running many things baremetal. I'll say the future is not serverless but cloudless
- hjacobs 8y ago
  I would argue that the longtail of applications does not really care about the impact of overlay networks. For us, the biggest impact on low-latency applications (where 1ms makes a difference) on Kubernetes was disabling CPU throttling in all clusters (you can also remove container limits). Background: a Kernel CFS quota bug leads to throttling even if quota is not yet reached, see https://www.youtube.com/watch?v=eBChCFD9hfs&feature=youtu.be&t=810 https://www.youtube.com/watch?v=eBChCFD9hfs&feature=youtu.be...
  - lykr0n 8y ago
    Overlay networks compound unstable networks. If your internal network latency spikes to 2ms from 0.5ms, normally that's not a huge issue but if you have micro-services that need to talk to each other- using my example, a 1.5ms round trip time would cause an additional 8ms of latency. Sure, that's a low number but if you already have 150ms of processing, adding another 10ms might cause issues. Also, if you're disabling the resource controls on kubernetes- you're kinda defeating the whole point.
- bostik 8y ago
  If you do trading or adtech (which, let's face it, is trading) then yes, the overhead from virtualisation and cloud environments is an unacceptable tradeoff. For most companies and their operating environments cloud gives flexibility in capacity planning, freedom to experiment wildly, access to practically unlimited storage[ß], plus reliable perimeter load balancing. And of course, for most companies engineering costs far more than compute. Paying the premium for cloud environment makes business sense if it means your development teams don't have to spend time waiting for available resources. Funnily enough, even a number of high-end trading firms (including HFT shops) are moving to cloud. Number crunching for models, backtesting, CI, analysis and simulation pipelines, ... All of those require resources and rarely need to operate in tandem with the real-time trading systems. The same flexibility, ease of capacity expansion and freedom to bring transient resources online as needed saves on expensive development time. If you're big enough, or need to operate fast enough then cloudless and baremetal are going to bring better ROI. Most companies aren't at these extreme ends, though. ß: within limits; if you need to produce and store petabytes of changing data on a daily basis...
- romeisendcoming 8y ago
  I'll be retired by then..and I disagree that it will be cloudless but for some compute the cloud is a risky option being forced upon them by exec level marketing, solutions hype (like k8s and docker). Some of the old hands always keep a colo + baremetal in the back pocket for always on and base testing; only pushing to the cloud after due diligence in comparative testing. That's a more realistic approach than 'cloudless' for scale.
kronin 8y ago
Trusting your private network is private turns that whole network into a candy store once a beach head in that network has been established. Defense in depth exists for a reason.
- abraae 8y ago
  If you're a smaller organisation, then spending too much time on "what if my private network is not private" will also cause you to struggle. For most people, in most practical scenarios, you have to hang your hat on something. Yes, take basic precautions, but if you lacked the chops to keep your private network private, then you have little or no chance of preventing the ensuing attacks.

by lykr0n 8y ago

devereaux 8y ago

So true. For some low latency applications, anything above the bare minimal virtualization is not acceptable. For what I do, in theory, many things should not impact results. In practice, anything that upon measurement impact results is stripped away. Think A/B testing but for every single component - including the major version of say the python interpreter. That's how you end up running many things baremetal. I'll say the future is not serverless but cloudless

hjacobs 8y ago
I would argue that the longtail of applications does not really care about the impact of overlay networks. For us, the biggest impact on low-latency applications (where 1ms makes a difference) on Kubernetes was disabling CPU throttling in all clusters (you can also remove container limits). Background: a Kernel CFS quota bug leads to throttling even if quota is not yet reached, see https://www.youtube.com/watch?v=eBChCFD9hfs&feature=youtu.be&t=810 https://www.youtube.com/watch?v=eBChCFD9hfs&feature=youtu.be...
- lykr0n 8y ago
  Overlay networks compound unstable networks. If your internal network latency spikes to 2ms from 0.5ms, normally that's not a huge issue but if you have micro-services that need to talk to each other- using my example, a 1.5ms round trip time would cause an additional 8ms of latency. Sure, that's a low number but if you already have 150ms of processing, adding another 10ms might cause issues. Also, if you're disabling the resource controls on kubernetes- you're kinda defeating the whole point.
bostik 8y ago
If you do trading or adtech (which, let's face it, is trading) then yes, the overhead from virtualisation and cloud environments is an unacceptable tradeoff. For most companies and their operating environments cloud gives flexibility in capacity planning, freedom to experiment wildly, access to practically unlimited storage[ß], plus reliable perimeter load balancing. And of course, for most companies engineering costs far more than compute. Paying the premium for cloud environment makes business sense if it means your development teams don't have to spend time waiting for available resources. Funnily enough, even a number of high-end trading firms (including HFT shops) are moving to cloud. Number crunching for models, backtesting, CI, analysis and simulation pipelines, ... All of those require resources and rarely need to operate in tandem with the real-time trading systems. The same flexibility, ease of capacity expansion and freedom to bring transient resources online as needed saves on expensive development time. If you're big enough, or need to operate fast enough then cloudless and baremetal are going to bring better ROI. Most companies aren't at these extreme ends, though. ß: within limits; if you need to produce and store petabytes of changing data on a daily basis...
romeisendcoming 8y ago
I'll be retired by then..and I disagree that it will be cloudless but for some compute the cloud is a risky option being forced upon them by exec level marketing, solutions hype (like k8s and docker). Some of the old hands always keep a colo + baremetal in the back pocket for always on and base testing; only pushing to the cloud after due diligence in comparative testing. That's a more realistic approach than 'cloudless' for scale.

kronin 8y ago

Trusting your private network is private turns that whole network into a candy store once a beach head in that network has been established. Defense in depth exists for a reason.

abraae 8y ago
If you're a smaller organisation, then spending too much time on "what if my private network is not private" will also cause you to struggle. For most people, in most practical scenarios, you have to hang your hat on something. Yes, take basic precautions, but if you lacked the chops to keep your private network private, then you have little or no chance of preventing the ensuing attacks.