5 ms·

> When they can choose (and there is a choice) to use collectors that don't pause, the pressure to keep allocation rates down changes, moving the "this is too m

> When they can choose (and there is a choice) to use collectors that don't pause, the pressure to keep allocation rates down changes, moving the "this is too much" lines up by more than an order of magnitude. You mean - like choose the only garbage collector on earth that actually can do this - for the JVM only - which your old company wrote and keeps guarded as a heavily proprietary secret, with the currently published specifications for the design (in the C4 paper) requiring untenable, unmergeable and extensive hacks in the operating system virtual memory layer that will never be largely accepted, with the current design in your product (which alleviates this) again being a heavily guarded secret that will never be revealed, and never have source available, and costs a shitload of money? Which basically means Azul is going to have the only available pauseless GC for the forseeable future? Well, now that you said it like that - you're right! We really do have a choice... a choice of actually using and working with things that exist and are widely available today, or already living in a land of unlimited money for licensing of a single product that might be completely and totally irrelevant to most of us. I guess the 'choice' isn't so clear, when you say it like that. Look, that aside, great blog post, and extremely informative. I'm a giant fan of Gene's work, seriously. But this work is Azul's brainchild and they'll never let it go. And I don't blame them for that. But drop the rhetoric, please - few people have the 'choice' of spending shitloads of money on a proprietary JVM like Azul.

fleitz 12y ago
You can buy servers or you can buy software. Welcome to the real world. Another choice is to write your software without a VM, many languages support this.
- thoughtpolice 12y ago
  Did I ever say otherwise? I don't blame Azul for making this work proprietary. I'm taking a stance against the rhetoric - this isn't a realistic choice for most people. Most people will buy more hardware over licensing expensive software, especially as hardware generally has reusable value, independent of your technology stack. This is especially true at the low end, where not everyone has massive A series or continuous funding. You have to plan a lot up front at that level to pay off your dividends here if you don't already have steady cash. And many people don't use the JVM at all. So the 'choice' of a pauseless collector is mostly a farce, as far as I can see. > Another choice is to write your software without a VM, many languages support this. Thank you for completely and totally missing the point. Didn't I just say something about "Not everyone uses the JVM", and your response is "just use another completely different language and technology stack"? We're going in circles here. Which other platforms support the C4 collector, including its pause times and sustained allocation rates, might I ask? Freely available open source ones, I might add.
wahern 12y ago
I don't think that there's anything magical about C4. All the basics are discernible from this presentation: http://www.youtube.com/watch?v=5uljtqyBLxI http://www.youtube.com/watch?v=5uljtqyBLxI Azul originally started out as a hardware company. A pauseless collector is easy as long as you have transactional memory (at a minimum hardware which supports strong LL/SC, allowing you to build the rest in software). No commodity CPUs support this, then or now, so Azul's first product was a specialized CPU and memory architecture with real transactional memory semantics. But nobody wants to buy specialized hardware these days. So they ended up having to emulate transactional memory on x86. That means _emulating_ the memory architecture of their original CPU. Again, there's no secret sauce to do doing this. Things like QEMU are free--which is to say, many people know how to do it. It just takes a lot of programmer hours to build it, and many, many more to optimize it. Long story short, Azul's collector is truly pauseless, but its transactional throughput in the vast majority of workloads sucks because it's an emulator (for the JVM) built atop another emulator (for their original memory architecture). And while you can speed up emulating the JVM using JIT techniques, that doesn't work for emulating a transactional memory architecture. There are endless tweaks you can make, including playing around with memory page tables, etc, but at the end up the day you're still stuck emulating the semantics you need, causing huge latencies relative to your clock speed, which already suck on Ghz scale systems. This won't change anytime time soon. Even Intel's new TSX features are not strong enough to be able to implement true transactional memory. Real transactional memory requires a way to atomically signal contention for every word of memory, or at least lines of words. It requires not only a specialized CPU, but specialized memory controllers, and specialized memory. Something like TSX would only provide a marginal performance improvement for C4, if anything at all, because almost all their effort goes into arranging to avoid locking, not making locking faster.
- giltene 12y ago
  While it's true that there is nothing magical in C4, pretty much everything you describe here is wrong (as in the exact opposite of what is going on). Specifically: - C4 makes no use of hardware transactional memory. - C4 works great (and natively) on commodity x86-64 cores. - C4 does not emulate anything (not memory architecture, not HTM). - C4's transactional throughput kicks ass. If you want to know how C4 works, all you need to do is read the actual C4 ISMM paper: http://www.azulsystems.com/sites/default/files/images/c4_paper_acm.pdf http://www.azulsystems.com/sites/default/files/images/c4_pap... <And yes, I'm one of the authors> How you got to your posted conclusions from the link you posted is a mystery to me. Cliff doesn't say anything much about the GC mechanism there: he is talking about lessons learned in designing custom hardware. And yes, Vega had some cool GC-assist features, but they are in no way part of the C4 collector mechanism. Since you raised the bogus, unsubstantiated assertion that "[Zing's] transactional throughput in the vast majority of workloads sucks" based purely on your mis-interpretation of what C4 actually does, I feel I must correct that notion. To do that, I'll point out that Zing (with C4) is currently used, in production, to run some of the highest transactional throughput and most latency sensitive applications in the world. Zing's sustainable throughput (the throughput at which the system still meets the required SLA) is generally dramatically better than other JVMs running on exactly the same modern x86 hardware. Yes, you read that right: Zing gets more production-worthy TPS out of the same x86 hardware. Not less. This is why people who actually need good throughput and good latency tend to graduate to using Zing, and start enjoying time in their hammocks as a result. See http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-October/002045.html http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-O... for a happy example. C4 is being used in everything from low latency trading (Algo, HFT, smart routing wire risk) to Online Retail (think black Friday workloads) and travel sites. C4 is used for everything from 1GB compute-heavy and messgaing workloads to 1TB data-heavy analysts applications. It powers big data workloads. It powers search. It powers Java servers of all kinds, big and small. And none of those are complaining about throughput. Quite the opposite. Peace.
giltene 12y ago
I don't really know where the "shitloads of money" notion comes from. I guess "shitloads" is a relative term, but Zing is usually no more expensive (and often cheaper) than the cost of the machines it runs on. Last I checked, those machines are usually powered by Intel-brainchild Xeon chips that Intel insists on getting money for, too. Zing is certainly cheaper than the engineering time it offsets. If you feel that you need to avoid using Zing at all costs, you can certainly avoid using it by spending shitloads of your best talent, time, and effort (which are presumably not free) to successfully force your apps only run at 300MB/sec on those 20GB/sec-capable Xeons you paid shitloads of money for, and to dramatically under-utilize them. That's certainly a choice. I'm pretty sure one shitload is bigger than the other. The whole trick is to pay with the smaller pile.
nitsanw 12y ago
Hmmm.... who'd have thought you can CHOOSE to pay some money and solve a problem... wonder who came up with that idea? people work hard and build a product and don't give it away for free? Can you qualify shitload? certainly it costs more than nothing. How much is shitload? "We really do have a choice... a choice of actually using and working with things that exist and are widely available today, or already living in a land of unlimited money for licensing of a single product that might be completely and totally irrelevant to most of us" Assuming your software is making you money (but you wouldn't make money through software, right? crazy idea), you might have a choice between spending your developers time (which is worth money, I assume) trying to reduce the rate of allocation to something the freebie JVM can handle without pausing, or spending some money on an "existing and widely available" solution to the allocation/collection problem. If application pauses don't cost you money, indeed a pauseless JVM is "completely and totally irrelevant". If they do, get a quote from Azul, maybe it's a good deal for you, maybe it isn't... Finally, if you are confused about money and how a free market works, just hang in there. Marx promised us the end of history is around the corner and all them capitalist pigs will be the first up against the wall come the revolution!

fleitz 12y ago

You can buy servers or you can buy software. Welcome to the real world. Another choice is to write your software without a VM, many languages support this.

thoughtpolice 12y ago
Did I ever say otherwise? I don't blame Azul for making this work proprietary. I'm taking a stance against the rhetoric - this isn't a realistic choice for most people. Most people will buy more hardware over licensing expensive software, especially as hardware generally has reusable value, independent of your technology stack. This is especially true at the low end, where not everyone has massive A series or continuous funding. You have to plan a lot up front at that level to pay off your dividends here if you don't already have steady cash. And many people don't use the JVM at all. So the 'choice' of a pauseless collector is mostly a farce, as far as I can see. > Another choice is to write your software without a VM, many languages support this. Thank you for completely and totally missing the point. Didn't I just say something about "Not everyone uses the JVM", and your response is "just use another completely different language and technology stack"? We're going in circles here. Which other platforms support the C4 collector, including its pause times and sustained allocation rates, might I ask? Freely available open source ones, I might add.

wahern 12y ago

I don't think that there's anything magical about C4. All the basics are discernible from this presentation: http://www.youtube.com/watch?v=5uljtqyBLxI http://www.youtube.com/watch?v=5uljtqyBLxI Azul originally started out as a hardware company. A pauseless collector is easy as long as you have transactional memory (at a minimum hardware which supports strong LL/SC, allowing you to build the rest in software). No commodity CPUs support this, then or now, so Azul's first product was a specialized CPU and memory architecture with real transactional memory semantics. But nobody wants to buy specialized hardware these days. So they ended up having to emulate transactional memory on x86. That means _emulating_ the memory architecture of their original CPU. Again, there's no secret sauce to do doing this. Things like QEMU are free--which is to say, many people know how to do it. It just takes a lot of programmer hours to build it, and many, many more to optimize it. Long story short, Azul's collector is truly pauseless, but its transactional throughput in the vast majority of workloads sucks because it's an emulator (for the JVM) built atop another emulator (for their original memory architecture). And while you can speed up emulating the JVM using JIT techniques, that doesn't work for emulating a transactional memory architecture. There are endless tweaks you can make, including playing around with memory page tables, etc, but at the end up the day you're still stuck emulating the semantics you need, causing huge latencies relative to your clock speed, which already suck on Ghz scale systems. This won't change anytime time soon. Even Intel's new TSX features are not strong enough to be able to implement true transactional memory. Real transactional memory requires a way to atomically signal contention for every word of memory, or at least lines of words. It requires not only a specialized CPU, but specialized memory controllers, and specialized memory. Something like TSX would only provide a marginal performance improvement for C4, if anything at all, because almost all their effort goes into arranging to avoid locking, not making locking faster.

giltene 12y ago
While it's true that there is nothing magical in C4, pretty much everything you describe here is wrong (as in the exact opposite of what is going on). Specifically: - C4 makes no use of hardware transactional memory. - C4 works great (and natively) on commodity x86-64 cores. - C4 does not emulate anything (not memory architecture, not HTM). - C4's transactional throughput kicks ass. If you want to know how C4 works, all you need to do is read the actual C4 ISMM paper: http://www.azulsystems.com/sites/default/files/images/c4_paper_acm.pdf http://www.azulsystems.com/sites/default/files/images/c4_pap... <And yes, I'm one of the authors> How you got to your posted conclusions from the link you posted is a mystery to me. Cliff doesn't say anything much about the GC mechanism there: he is talking about lessons learned in designing custom hardware. And yes, Vega had some cool GC-assist features, but they are in no way part of the C4 collector mechanism. Since you raised the bogus, unsubstantiated assertion that "[Zing's] transactional throughput in the vast majority of workloads sucks" based purely on your mis-interpretation of what C4 actually does, I feel I must correct that notion. To do that, I'll point out that Zing (with C4) is currently used, in production, to run some of the highest transactional throughput and most latency sensitive applications in the world. Zing's sustainable throughput (the throughput at which the system still meets the required SLA) is generally dramatically better than other JVMs running on exactly the same modern x86 hardware. Yes, you read that right: Zing gets more production-worthy TPS out of the same x86 hardware. Not less. This is why people who actually need good throughput and good latency tend to graduate to using Zing, and start enjoying time in their hammocks as a result. See http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-October/002045.html http://mail.openjdk.java.net/pipermail/hotspot-gc-use/2014-O... for a happy example. C4 is being used in everything from low latency trading (Algo, HFT, smart routing wire risk) to Online Retail (think black Friday workloads) and travel sites. C4 is used for everything from 1GB compute-heavy and messgaing workloads to 1TB data-heavy analysts applications. It powers big data workloads. It powers search. It powers Java servers of all kinds, big and small. And none of those are complaining about throughput. Quite the opposite. Peace.

giltene 12y ago

I don't really know where the "shitloads of money" notion comes from. I guess "shitloads" is a relative term, but Zing is usually no more expensive (and often cheaper) than the cost of the machines it runs on. Last I checked, those machines are usually powered by Intel-brainchild Xeon chips that Intel insists on getting money for, too. Zing is certainly cheaper than the engineering time it offsets. If you feel that you need to avoid using Zing at all costs, you can certainly avoid using it by spending shitloads of your best talent, time, and effort (which are presumably not free) to successfully force your apps only run at 300MB/sec on those 20GB/sec-capable Xeons you paid shitloads of money for, and to dramatically under-utilize them. That's certainly a choice. I'm pretty sure one shitload is bigger than the other. The whole trick is to pay with the smaller pile.

nitsanw 12y ago

Hmmm.... who'd have thought you can CHOOSE to pay some money and solve a problem... wonder who came up with that idea? people work hard and build a product and don't give it away for free? Can you qualify shitload? certainly it costs more than nothing. How much is shitload? "We really do have a choice... a choice of actually using and working with things that exist and are widely available today, or already living in a land of unlimited money for licensing of a single product that might be completely and totally irrelevant to most of us" Assuming your software is making you money (but you wouldn't make money through software, right? crazy idea), you might have a choice between spending your developers time (which is worth money, I assume) trying to reduce the rate of allocation to something the freebie JVM can handle without pausing, or spending some money on an "existing and widely available" solution to the allocation/collection problem. If application pauses don't cost you money, indeed a pauseless JVM is "completely and totally irrelevant". If they do, get a quote from Azul, maybe it's a good deal for you, maybe it isn't... Finally, if you are confused about money and how a free market works, just hang in there. Marx promised us the end of history is around the corner and all them capitalist pigs will be the first up against the wall come the revolution!