5 ms·

Presumably because it becomes a shared cost. 100 JVMs in 100 containers have to pay that cost individually. Now 1 or 2 central instances have the resources fo

Presumably because it becomes a shared cost. 100 JVMs in 100 containers have to pay that cost individually. Now 1 or 2 central instances have the resources for compiling, the 100 worker bees don't need them. They also might becable to optimize faster as they have runtime statistics from other instances, skipping warmup.

dragontamer 5y ago
100 JVMs in 100 containers (on one machine) also have to pay the costs of 100x individual garbage collections, instead of centralizing those garbage collections and merging the memories of everyone together. Hmmm... maybe just having a monolith at that point is better? If you can run 100 containers on one box, why not just run one environment on one box?
- vlovich123 5y ago
  I know you're joking but there is an important distinction that enables this. The optimization choices you're going to make of how to lower bytecode to native are going to be largely the same across all your machines - the minute differences in profiles aren't going to make significant differences unless you have the same codebase doing drastically different things. Thus you can share cost easily that so you accumulate execution profiles across your cluster and then lower to native code efficiently. The garbage collection choices are going to be unique to the workload running on that server and have little information worth sharing that will speed up operations.
  - hinkley 5y ago
    We could in theory get a lot of this benefit by supporting profile guided optimization and building a simple distribution system. You could even instrument a fraction of your servers and distribute to the rest.
  - dragontamer 5y ago
    > The garbage collection choices are going to be unique to the workload running on that server and have little information worth sharing that will speed up operations. Java's GC is generational though. Which means that Java-style GC benefits from excessive memory (ie: the more free memory you have, the less the GC is run, and the more that any such object is "dead" before the GC is run). With more "dead objects" left in the heap, a "bigger heap" benefits all processes. -------- Consider this simple case: 1MB of live data in a 6MB heap. The garbage collector could run 10 times in some given time period (the heap drops down to 1MB each time the GC is run, and 10-runs means that ~51MBs were allocated in this timeframe 6MBs before the first garbage collection, and then 5MB for each of the remaining 9 collections). Now what happens if you had a 11MB heap instead? Well, the garbage collector won't run until 11MBs were allocated (1st run), and with 1MB of live data after each collection, each remaining run would only be done after 10MBs of allocations. That is to say: only 5-collections would happen. So with the same algorithm, the 11MB heap with 1MB of live data is twice-as-efficient (ie: half the collections) as the 6MB heap (with 1MB of live data). ------- Java and its garbage collector is multithreaded these days and the analysis is more difficult. But that doesn't change the fact that generational collectors have this property, where the more RAM they use between collections, the "better they perform". So just "defacto-sharing more RAM" means all of the individual Java-threads will be more efficient at garbage collection. Well-written heaps get more efficient the more you throw at them.
    - vlovich123 5y ago
      Oh. I misread your suggestion. Yes, if you can somehow have all the services living within the same address space, a shared heap could probably be better. In the model described here we're talking about a multi-computer distributed system though, not a bunch of containers on one machine. Also, by merging everything into one address space you're now potentially making the blast radius of a security vulnerability much larger. Also, it's not instantly clear things actually get better because while your heap space is larger, so is your object graph and the rate at which your objects are getting created is larger. You'll get some compression out of it by sharing the class information across services but not much more than that.
      - 5y ago

dragontamer 5y ago

100 JVMs in 100 containers (on one machine) also have to pay the costs of 100x individual garbage collections, instead of centralizing those garbage collections and merging the memories of everyone together. Hmmm... maybe just having a monolith at that point is better? If you can run 100 containers on one box, why not just run one environment on one box?

vlovich123 5y ago
I know you're joking but there is an important distinction that enables this. The optimization choices you're going to make of how to lower bytecode to native are going to be largely the same across all your machines - the minute differences in profiles aren't going to make significant differences unless you have the same codebase doing drastically different things. Thus you can share cost easily that so you accumulate execution profiles across your cluster and then lower to native code efficiently. The garbage collection choices are going to be unique to the workload running on that server and have little information worth sharing that will speed up operations.
- hinkley 5y ago
  We could in theory get a lot of this benefit by supporting profile guided optimization and building a simple distribution system. You could even instrument a fraction of your servers and distribute to the rest.
- dragontamer 5y ago
  > The garbage collection choices are going to be unique to the workload running on that server and have little information worth sharing that will speed up operations. Java's GC is generational though. Which means that Java-style GC benefits from excessive memory (ie: the more free memory you have, the less the GC is run, and the more that any such object is "dead" before the GC is run). With more "dead objects" left in the heap, a "bigger heap" benefits all processes. -------- Consider this simple case: 1MB of live data in a 6MB heap. The garbage collector could run 10 times in some given time period (the heap drops down to 1MB each time the GC is run, and 10-runs means that ~51MBs were allocated in this timeframe 6MBs before the first garbage collection, and then 5MB for each of the remaining 9 collections). Now what happens if you had a 11MB heap instead? Well, the garbage collector won't run until 11MBs were allocated (1st run), and with 1MB of live data after each collection, each remaining run would only be done after 10MBs of allocations. That is to say: only 5-collections would happen. So with the same algorithm, the 11MB heap with 1MB of live data is twice-as-efficient (ie: half the collections) as the 6MB heap (with 1MB of live data). ------- Java and its garbage collector is multithreaded these days and the analysis is more difficult. But that doesn't change the fact that generational collectors have this property, where the more RAM they use between collections, the "better they perform". So just "defacto-sharing more RAM" means all of the individual Java-threads will be more efficient at garbage collection. Well-written heaps get more efficient the more you throw at them.
  - vlovich123 5y ago
    Oh. I misread your suggestion. Yes, if you can somehow have all the services living within the same address space, a shared heap could probably be better. In the model described here we're talking about a multi-computer distributed system though, not a bunch of containers on one machine. Also, by merging everything into one address space you're now potentially making the blast radius of a security vulnerability much larger. Also, it's not instantly clear things actually get better because while your heap space is larger, so is your object graph and the rate at which your objects are getting created is larger. You'll get some compression out of it by sharing the class information across services but not much more than that.
    - 5y ago