8 ms·

This is very similar to Java's CMS collector[1]. Unfortunately, 10ms is still far too long for some applications. I have hopes for something like Azul's pause

This is very similar to Java's CMS collector[1]. Unfortunately, 10ms is still far too long for some applications. I have hopes for something like Azul's pauseless GC[2] eventually becoming a common GC strategy, but I'm not holding my breath. In the meantime, there's always sun.misc.Unsafe[3] in the JVM world :-( [1] - https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/cms.html https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gc... [2] - http://www.azulsystems.com/zing/pgc http://www.azulsystems.com/zing/pgc [3] - http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot-unsafe/ http://mishadoff.com/blog/java-magic-part-4-sun-dot-misc-dot...

hencq 11y ago
In this old thread [1] one of the Azul developers outlines a potential path to having a C4 type GC for Go. Interestingly he mentions moving to a generational collector first, before moving to concurrent. It seems the Go developers have decided to go the other way, i.e. make the collector concurrent, but not generational. One of the reasons given for going generational first is implementing a write-barrier, but from the blog post it sounds like that is implemented for this concurrent collector anyway, so maybe it doesn't really matter. [1] - https://groups.google.com/d/msg/golang-dev/GvA0DaCI2BU/SmEelQsFybkJ https://groups.google.com/d/msg/golang-dev/GvA0DaCI2BU/SmEel...
rancur 11y ago
I'm really surprised this is so new. I'm only beginning to enter the field and falling asleep the other night one of my thoughts was 'why doesn't the GC happen piecemeal?' AKA 'who said we have to collect everything, just unload 2 bins worth'. Mainly WRT use in Android, where it causes perceivable hitching
- Gibbon1 11y ago
  My experience is a lot of people/groups get obsessed with one metric in this case average speed. A pauseless GC is always going to be a bit slower than one that pauses once in a blue moon (meaning all the time from the users standpoint). Requires a really strong will to go against that sort of grain. Just remembered though: Friend who is a Azul developer said that 0x86 processors did not have the correct instructions to support pausesless GC up until very recently. So probably a chicken and the egg problem.
  - rancur 11y ago
    can you open that up more? I'm skeptical. Which instructions?
    - Gibbon1 11y ago
      Over my pay grade, which is why I'm slopping away at ARM Cortex firmware instead of at Azul. I'd ask my friend but he's on vacation. I do know Azul originally ran their JVM on custom processors. (Target market high speed trading systems, etc) I can't see why they would have done that unless older 0x86 etc processors just couldn't do what they needed. A little internet searching, leads to this and some other stuff. https://www.artima.com/lejava/articles/azul_pauseless_gc.html https://www.artima.com/lejava/articles/azul_pauseless_gc.htm... I think the deal is, if the GC moves something, it can leave a bunch of dangling pointers. Those need to be fixed up to point to the new place before they get dereferenced. Except Azul uses a read barrier to trap when a dangling pointer is dereferenced, so the fix up can be lazy.
- on_and_off 11y ago
  The Android Framework team has been working on improving the runtime and its garbage collector for quite some time. There have been many improvements to the garbage collection strategy. With ART on Lollipop, according to its creator, there should never be a visible GC pause when the app is in the foreground. More information here : http://www.anandtech.com/show/8231/a-closer-look-at-android-runtime-art-in-android-l/2 http://www.anandtech.com/show/8231/a-closer-look-at-android-... and in various I/O talks. Extremely bad code will still cause problems (but that's true for whatever memory management strategy you rely on) but in my experience, on a nexus 5, GC pauses are no longer a real issue on Android. Still, Samsung is here to samsung things up. Comparisons of scrolling performances between a flagship samsung device and a nexus are just as expected : https://www.reddit.com/r/GalaxyS6/comments/3ck9no/scrolling_comparison_between_s6_and_nexus_5/ https://www.reddit.com/r/GalaxyS6/comments/3ck9no/scrolling_... That's another topic entirely, but I think that another key factor is to move almost as much as possible of the UI work to a separate thread, not just to make heavy work in the background. It is starting to happen with ripple animations on a RenderThread, but I think that this strategy could be generalized.
  - pjmlp 11y ago
    On my tablet (not nexus) with Android 5.0 I can surely see when everything comes to a stall.
    - on_and_off 11y ago
      Tablets are indeed complex. At best they have the same SOC as a good phone but they typically have way more pixels to carry around. NoName Chinese tablets are often a disaster and even high end tablets are hard to get right. Even putting graphics aside, things like a slow internal memory can also wreak performances.
    - rancur 11y ago
      might be 5.0 mem leak issues and general bugs
  - rancur 11y ago
    > Still, Samsung is here to samsung things up 3rd time I've laughed with more than 60% intensity at something I read on the internet in the last 2 years. thank you!!! hahahaaha for reference, my last 100% laugh that returned me to the days of cartoons and childhood was this comment about how to install and activate your non-rental-Cable-modem: https://www.reddit.com/r/technology/comments/27szuw/comcast_plans_to_turn_50000_home_routers_into/ci4hlof https://www.reddit.com/r/technology/comments/27szuw/comcast_...
nickpsecurity 11y ago
+1 on Azul. They've pretty much solved it by improving on past methods combining hardware and software. Go could do the same thing. I keep wondering about putting a dedicated FPGA on the memory bus that does nothing but concurrent GC. Have a mechanism to keep the processor (s) and it from stepping on each others' toes. Might work wonders.
- thrownaway2424 11y ago
  Why would an FPGA be a better solution than software running on another core?
  - anarcticpuffin 11y ago
    I'm guessing since it's sitting on the memory bus it could intercept pointer modifications and synchronously update it's graph?
    - nickpsecurity 11y ago
      Memory bus is just for speed. You don't want it doing stuff like that lol. See these two LISP machines for where my inspiration of putting it on memory bus came from: http://diyhpl.us/~bryan/papers2/paperbot/Design%20of%20a%20LISP-based%20microprocessor.pdf http://diyhpl.us/~bryan/papers2/paperbot/Design%20of%20a%20L... (See section 7 for a radical... err realy old... way to do concurrent GC. Full paper available at ACM/IEEE or if you Google LISP processors Guy Steele enough.) Scheme machine by Burger 1995 http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=569B0A6FC0F670004BB323ADB5D75C9E?doi=10.1.1.73.590&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=569... (See the main graphic and specifics in storage section later. Once again, GC-like stuff is handled in memory management part of the processor. This processor knows that, though, to assist GC a bit. Also different in that it was specified and then synthesized to heterogenous hardware with DDD "correct-by-construction" toolkit.) So, have fun with those. Plus, Google hardware-assisted or hardware garbage collection to get lots of interesting results already done.
  - nickpsecurity 11y ago
colin_mccabe 11y ago
Go already gives better tools than Java for managing native memory through cgo, which is a far less painful interface than JNI (believe me, I've written a lot of JNI for Hadoop). Go also has value types which is a huge win for managing memory. (And even if Java gets value types later, the whole Java standard library will still take years to change to use them, if it ever does.) Plus, sun.misc.Unsafe is probably going away, according to Oracle. http://www.infoq.com/news/2015/07/oracle-plan-remove-unsafe http://www.infoq.com/news/2015/07/oracle-plan-remove-unsafe From what I've heard, Azul has a great GC, but the throughput is extremely low. It's really only a practical solution for high frequency finance and places like that where latency is everything, and throughput is nothing (can buy another 100 servers or high end hardware.) Note: I'm talking about their software product which runs on vanilla hardware, not their hardware product, which I understand is far superior. A lot of people on HN also seem to be taking the statement that maximum GC latency will be 10ms as a statement that there will often be 10ms pauses. Hopefully, the average latency will be far less, in the 1 or 2ms, and 10ms will be something that only happens on huge heaps in certain conditions. This should be similar to what has happened on Android, where GC pauses are pretty rare and typically only 1 or 2ms.
- pron 11y ago
  > the whole Java standard library will still take years to change to use them, if it ever does. All Java collections (which are what really matters) are being retrofitted alongside with the introduction of value types. As soon as value types are introduced (and the HotSpot team is working hard on that right now), all collections will be fully value-ready. > Go already gives better tools than Java for managing native memory through cgo, which is a far less painful interface than JNI JNI is being replaced by Project Panama: http://openjdk.java.net/projects/panama/ http://openjdk.java.net/projects/panama/ (you can already use a similar FFI already with JNR, which serves as a blueprint for Panama's FFI: https://github.com/jnr https://github.com/jnr I've used JNR to write a FUSE filesystem in Java without a line of C, and unlike JNA, it's fast!) Besides, HotSpot runs C now, too, and quite well: http://www.chrisseaton.com/rubytruffle/cext/ http://www.chrisseaton.com/rubytruffle/cext/ https://dl.dropboxusercontent.com/u/292832/useR_multilang_demo.mp4 https://dl.dropboxusercontent.com/u/292832/useR_multilang_de... > Plus, sun.misc.Unsafe is probably going away, according to Oracle. ... only to be replaced by something much better: https://www.youtube.com/watch?v=ycKn18LtNtk https://www.youtube.com/watch?v=ycKn18LtNtk (Unsafe isn't going away until replacements are available). > From what I've heard, Azul has a great GC, but the throughput is extremely low. Not at all. Just a little lower than with HotSpot's throughput collector, and possibly higher than with G1 (although G1 changes a lot, so that might not be true).
the8472 11y ago
There are plans for adding one to hotspot[1]. As I understand it it can't use the same tricks as C4 because that would require kernel support, but unlike CMS it will be compacting, thus avoiding the terrible failure modes of CMS. There also is some work being done to make the CMS failures a little less terrible by parallelizing them.[2] [1] http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189 [2] https://bugs.openjdk.java.net/browse/JDK-8086706 https://bugs.openjdk.java.net/browse/JDK-8086706
- pron 11y ago
  As HotSpot's GC team is mostly working on G1 and grooming it as a CMS replacement (it has replaced the parallel GC as the default GC in current JDK9 builds), I don't think there's much further work on CMS.
  - the8472 11y ago
    The CMS improvements I've mentioned to are being contributed by a 3rd party. Based on the mailing list posts[1] I think it was google. [1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-June/013784.html http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-J...
    - pron 11y ago
      That's really cool! I hope it gets merged...
bitmapbrother 11y ago
Interesting side note - Azul's GC isn't "pauseless" it's pause less.

hencq 11y ago

In this old thread [1] one of the Azul developers outlines a potential path to having a C4 type GC for Go. Interestingly he mentions moving to a generational collector first, before moving to concurrent. It seems the Go developers have decided to go the other way, i.e. make the collector concurrent, but not generational. One of the reasons given for going generational first is implementing a write-barrier, but from the blog post it sounds like that is implemented for this concurrent collector anyway, so maybe it doesn't really matter. [1] - https://groups.google.com/d/msg/golang-dev/GvA0DaCI2BU/SmEelQsFybkJ https://groups.google.com/d/msg/golang-dev/GvA0DaCI2BU/SmEel...

rancur 11y ago

I'm really surprised this is so new. I'm only beginning to enter the field and falling asleep the other night one of my thoughts was 'why doesn't the GC happen piecemeal?' AKA 'who said we have to collect everything, just unload 2 bins worth'. Mainly WRT use in Android, where it causes perceivable hitching

Gibbon1 11y ago
My experience is a lot of people/groups get obsessed with one metric in this case average speed. A pauseless GC is always going to be a bit slower than one that pauses once in a blue moon (meaning all the time from the users standpoint). Requires a really strong will to go against that sort of grain. Just remembered though: Friend who is a Azul developer said that 0x86 processors did not have the correct instructions to support pausesless GC up until very recently. So probably a chicken and the egg problem.
- rancur 11y ago
  can you open that up more? I'm skeptical. Which instructions?
  - Gibbon1 11y ago
    Over my pay grade, which is why I'm slopping away at ARM Cortex firmware instead of at Azul. I'd ask my friend but he's on vacation. I do know Azul originally ran their JVM on custom processors. (Target market high speed trading systems, etc) I can't see why they would have done that unless older 0x86 etc processors just couldn't do what they needed. A little internet searching, leads to this and some other stuff. https://www.artima.com/lejava/articles/azul_pauseless_gc.html https://www.artima.com/lejava/articles/azul_pauseless_gc.htm... I think the deal is, if the GC moves something, it can leave a bunch of dangling pointers. Those need to be fixed up to point to the new place before they get dereferenced. Except Azul uses a read barrier to trap when a dangling pointer is dereferenced, so the fix up can be lazy.
on_and_off 11y ago
The Android Framework team has been working on improving the runtime and its garbage collector for quite some time. There have been many improvements to the garbage collection strategy. With ART on Lollipop, according to its creator, there should never be a visible GC pause when the app is in the foreground. More information here : http://www.anandtech.com/show/8231/a-closer-look-at-android-runtime-art-in-android-l/2 http://www.anandtech.com/show/8231/a-closer-look-at-android-... and in various I/O talks. Extremely bad code will still cause problems (but that's true for whatever memory management strategy you rely on) but in my experience, on a nexus 5, GC pauses are no longer a real issue on Android. Still, Samsung is here to samsung things up. Comparisons of scrolling performances between a flagship samsung device and a nexus are just as expected : https://www.reddit.com/r/GalaxyS6/comments/3ck9no/scrolling_comparison_between_s6_and_nexus_5/ https://www.reddit.com/r/GalaxyS6/comments/3ck9no/scrolling_... That's another topic entirely, but I think that another key factor is to move almost as much as possible of the UI work to a separate thread, not just to make heavy work in the background. It is starting to happen with ripple animations on a RenderThread, but I think that this strategy could be generalized.
- pjmlp 11y ago
  On my tablet (not nexus) with Android 5.0 I can surely see when everything comes to a stall.
  - on_and_off 11y ago
    Tablets are indeed complex. At best they have the same SOC as a good phone but they typically have way more pixels to carry around. NoName Chinese tablets are often a disaster and even high end tablets are hard to get right. Even putting graphics aside, things like a slow internal memory can also wreak performances.
  - rancur 11y ago
    might be 5.0 mem leak issues and general bugs
- rancur 11y ago
  > Still, Samsung is here to samsung things up 3rd time I've laughed with more than 60% intensity at something I read on the internet in the last 2 years. thank you!!! hahahaaha for reference, my last 100% laugh that returned me to the days of cartoons and childhood was this comment about how to install and activate your non-rental-Cable-modem: https://www.reddit.com/r/technology/comments/27szuw/comcast_plans_to_turn_50000_home_routers_into/ci4hlof https://www.reddit.com/r/technology/comments/27szuw/comcast_...

nickpsecurity 11y ago

+1 on Azul. They've pretty much solved it by improving on past methods combining hardware and software. Go could do the same thing. I keep wondering about putting a dedicated FPGA on the memory bus that does nothing but concurrent GC. Have a mechanism to keep the processor (s) and it from stepping on each others' toes. Might work wonders.

thrownaway2424 11y ago
Why would an FPGA be a better solution than software running on another core?
- anarcticpuffin 11y ago
  I'm guessing since it's sitting on the memory bus it could intercept pointer modifications and synchronously update it's graph?
  - nickpsecurity 11y ago
    Memory bus is just for speed. You don't want it doing stuff like that lol. See these two LISP machines for where my inspiration of putting it on memory bus came from: http://diyhpl.us/~bryan/papers2/paperbot/Design%20of%20a%20LISP-based%20microprocessor.pdf http://diyhpl.us/~bryan/papers2/paperbot/Design%20of%20a%20L... (See section 7 for a radical... err realy old... way to do concurrent GC. Full paper available at ACM/IEEE or if you Google LISP processors Guy Steele enough.) Scheme machine by Burger 1995 http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=569B0A6FC0F670004BB323ADB5D75C9E?doi=10.1.1.73.590&rep=rep1&type=pdf http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=569... (See the main graphic and specifics in storage section later. Once again, GC-like stuff is handled in memory management part of the processor. This processor knows that, though, to assist GC a bit. Also different in that it was specified and then synthesized to heterogenous hardware with DDD "correct-by-construction" toolkit.) So, have fun with those. Plus, Google hardware-assisted or hardware garbage collection to get lots of interesting results already done.
- nickpsecurity 11y ago

colin_mccabe 11y ago

Go already gives better tools than Java for managing native memory through cgo, which is a far less painful interface than JNI (believe me, I've written a lot of JNI for Hadoop). Go also has value types which is a huge win for managing memory. (And even if Java gets value types later, the whole Java standard library will still take years to change to use them, if it ever does.) Plus, sun.misc.Unsafe is probably going away, according to Oracle. http://www.infoq.com/news/2015/07/oracle-plan-remove-unsafe http://www.infoq.com/news/2015/07/oracle-plan-remove-unsafe From what I've heard, Azul has a great GC, but the throughput is extremely low. It's really only a practical solution for high frequency finance and places like that where latency is everything, and throughput is nothing (can buy another 100 servers or high end hardware.) Note: I'm talking about their software product which runs on vanilla hardware, not their hardware product, which I understand is far superior. A lot of people on HN also seem to be taking the statement that maximum GC latency will be 10ms as a statement that there will often be 10ms pauses. Hopefully, the average latency will be far less, in the 1 or 2ms, and 10ms will be something that only happens on huge heaps in certain conditions. This should be similar to what has happened on Android, where GC pauses are pretty rare and typically only 1 or 2ms.

pron 11y ago
> the whole Java standard library will still take years to change to use them, if it ever does. All Java collections (which are what really matters) are being retrofitted alongside with the introduction of value types. As soon as value types are introduced (and the HotSpot team is working hard on that right now), all collections will be fully value-ready. > Go already gives better tools than Java for managing native memory through cgo, which is a far less painful interface than JNI JNI is being replaced by Project Panama: http://openjdk.java.net/projects/panama/ http://openjdk.java.net/projects/panama/ (you can already use a similar FFI already with JNR, which serves as a blueprint for Panama's FFI: https://github.com/jnr https://github.com/jnr I've used JNR to write a FUSE filesystem in Java without a line of C, and unlike JNA, it's fast!) Besides, HotSpot runs C now, too, and quite well: http://www.chrisseaton.com/rubytruffle/cext/ http://www.chrisseaton.com/rubytruffle/cext/ https://dl.dropboxusercontent.com/u/292832/useR_multilang_demo.mp4 https://dl.dropboxusercontent.com/u/292832/useR_multilang_de... > Plus, sun.misc.Unsafe is probably going away, according to Oracle. ... only to be replaced by something much better: https://www.youtube.com/watch?v=ycKn18LtNtk https://www.youtube.com/watch?v=ycKn18LtNtk (Unsafe isn't going away until replacements are available). > From what I've heard, Azul has a great GC, but the throughput is extremely low. Not at all. Just a little lower than with HotSpot's throughput collector, and possibly higher than with G1 (although G1 changes a lot, so that might not be true).

the8472 11y ago

There are plans for adding one to hotspot[1]. As I understand it it can't use the same tricks as C4 because that would require kernel support, but unlike CMS it will be compacting, thus avoiding the terrible failure modes of CMS. There also is some work being done to make the CMS failures a little less terrible by parallelizing them.[2] [1] http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189 [2] https://bugs.openjdk.java.net/browse/JDK-8086706 https://bugs.openjdk.java.net/browse/JDK-8086706

pron 11y ago
As HotSpot's GC team is mostly working on G1 and grooming it as a CMS replacement (it has replaced the parallel GC as the default GC in current JDK9 builds), I don't think there's much further work on CMS.
- the8472 11y ago
  The CMS improvements I've mentioned to are being contributed by a 3rd party. Based on the mailing list posts[1] I think it was google. [1] http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-June/013784.html http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2015-J...
  - pron 11y ago
    That's really cool! I hope it gets merged...

bitmapbrother 11y ago

Interesting side note - Azul's GC isn't "pauseless" it's pause less.