5 ms·

Instead of composing libraries into an executable you can use containers to compose services into an application [0]. As noted in the sibling comment, this stil

Instead of composing libraries into an executable you can use containers to compose services into an application [0]. As noted in the sibling comment, this still can have a high utility for just a single workstation/service. Also as you noted, scaling to a datacenter is already partly automated. The entire environment is captured in code and reproducible/portable. But you need to be familiar with the tools and ecosystem to make use of them. [0] We don’t have concise terms for the types of distributed, complex systems which have become standard.

muxator 3y ago
> Instead of composing libraries into an executable you can use containers to compose services into an application This is the most concise and exact definition I've heard lately. My only problem with the consequences of this approach is that the amount of overhead for performing the same operations is astonishing. Messages pass on a network instead of stsying in RAM, CPU context switches every other ms to handle IO for what could have been a simple function call, etc. It's amazing when it's needed, but seeing this approach becoming the standard for a big part of the industry almost turns it into an environmental problem. How much energy is wasted juggling bits around?
- lmz 3y ago
  It doesn't have to be a library replacement. It could be that your app uses a DB server and a message queue for async tasks. Containers allow you to easily run those servers locally for dev if you don't care about performance and maybe in prod you'll have a dedicated server for them, or use some DB as a service offering.
  - muxator 3y ago
    Yeah, agreed: in that case spinning a container is evidently an operational advantage. However, I was specifically answering to the different case highlighted by the parent comment: how a potentially cohesive application is cut along some of its internal APIs, and some functionality is allocated to different processes living in different containers. It is an extreme point in the continuum "single thread" -> "multi thread" -> "multiple processes" -> "fully distributed". In that continuum scalability increases, while efficiency progressively decreases. Cornering oneself to a specific point in the design space is problematic, and for some cases has direct implications on how many resources are wasted. A very didactic experience is, for example, running a simple local application under a microarchitecture profiler, such as Intel Vtune. It is not uncommon seeing that even straightforward C/C++ programs use a core resources less than 10%. What I am reflecting about is that the choice of fragmenting that program among tens of systems (maybe in a scripting langiage) should be conscious, and done after encountering performance or scalability bottlenecks. How much of the resulting total system workload would be useful work? The quantity of potentially wasted resources is astonishing if you think about it.

muxator 3y ago

> Instead of composing libraries into an executable you can use containers to compose services into an application This is the most concise and exact definition I've heard lately. My only problem with the consequences of this approach is that the amount of overhead for performing the same operations is astonishing. Messages pass on a network instead of stsying in RAM, CPU context switches every other ms to handle IO for what could have been a simple function call, etc. It's amazing when it's needed, but seeing this approach becoming the standard for a big part of the industry almost turns it into an environmental problem. How much energy is wasted juggling bits around?

lmz 3y ago
It doesn't have to be a library replacement. It could be that your app uses a DB server and a message queue for async tasks. Containers allow you to easily run those servers locally for dev if you don't care about performance and maybe in prod you'll have a dedicated server for them, or use some DB as a service offering.
- muxator 3y ago
  Yeah, agreed: in that case spinning a container is evidently an operational advantage. However, I was specifically answering to the different case highlighted by the parent comment: how a potentially cohesive application is cut along some of its internal APIs, and some functionality is allocated to different processes living in different containers. It is an extreme point in the continuum "single thread" -> "multi thread" -> "multiple processes" -> "fully distributed". In that continuum scalability increases, while efficiency progressively decreases. Cornering oneself to a specific point in the design space is problematic, and for some cases has direct implications on how many resources are wasted. A very didactic experience is, for example, running a simple local application under a microarchitecture profiler, such as Intel Vtune. It is not uncommon seeing that even straightforward C/C++ programs use a core resources less than 10%. What I am reflecting about is that the choice of fragmenting that program among tens of systems (maybe in a scripting langiage) should be conscious, and done after encountering performance or scalability bottlenecks. How much of the resulting total system workload would be useful work? The quantity of potentially wasted resources is astonishing if you think about it.