9 ms·

People use the JVM in soft realtime / financial applications, and the trick is to reduce allocations, especially of objects that are long-lived enough to make i

by eloff 12y ago

People use the JVM in soft realtime / financial applications, and the trick is to reduce allocations, especially of objects that are long-lived enough to make it to gen 2. Go is better suited than Java for those kinds of applications, because it's easier to avoid allocations (Java doesn't have stack allocations, but Go does.) Also hard upper limits on GC time are very helpful for those cases where allocations can't be reduced any further. The standard library additionally has a Pool type that allows for reducing pressure on the GC through object reuse.

munificent 12y ago
> (Java doesn't have stack allocations, but Go does.) Go also lets you have fields in place in structs, like C and C++. In Java, all fields (except primitives) are references.
- seanmcdirmid 12y ago
  C# has had structs since V1 I think.
- bad_user 12y ago
  Value Types are coming in the next version of Java. Yes, it is great to be able to allocate on the stack, but then Java has better garbage collectors, including a completely pauseless one available commercially. Even in this announcement the new GC mentioned for Go is non generational. I have a feeling that even Java's good old CMS is better. Java also does escape analisys. All in all, once value classes are in, it is going to be awesome.
_ak 12y ago
We use Go for a soft realtime application (RTB), and it works well enough. We essentially have two performance issues, one being the particular database we use (which is not directly related to Go itself), the other one being GC pressure. Before Go 1.3, too many allocations were _very_ noticeable with GC pauses sometimes being close to 100ms (which is a problem when your maximum RTT is 100ms), but Go 1.3 improved things enormously. I'm confident things will get even better with Go 1.4.
- kator 12y ago
  When you're 90ms into your bid calc and Go pauses for 10ms STW you will loose all your connections and timeout your bids. It's fine for basic stuff but if you want to really listen to full RTB bid stream and have real time responses you need pauseless GC to win. Or you have to budget for considerably less time to make a decision. Also there are places in RTB where you have 10ms RTT and Go fails ought right in that situation at scale. :-)
  - _ak 12y ago
    I'm talking about online advertising. RTTs for common ad exchanges and SSPs are usually 100 to 120ms.
pcwalton 12y ago
The HotSpot JVM stack allocates (actually, even better: it SROAs) in very similar conditions to when Go does: when escape analysis decides that a value does not escape. Even if it doesn't stack allocate, bump allocation in the nursery ends up being about as fast as stack allocation (which is, as far as I know, why HotSpot chose only to lock elide and not to stack allocate when they implemented escape analysis).
- crawshaw 12y ago
  I haven't looked into the JVM's stack allocation logic in several years, so it may have improved to the point where that is now true. (It wasn't so in 2010.) However, Go does have features that make it easier to control allocations than Java. One example, type s struct { x int32 y int32 } v := make([]s, 1e5) The object v uses 1e5 * 8 bytes of storage. Equivalent Java code would require boxing of each s value. That is, instead of []s, you end up with []*s and allocating each s.
  - pcwalton 12y ago
    Sure, I wasn't disputing that Go has more interior allocation features than Java does.
  - chrisseaton 12y ago
    I wrote a blog post that showed how powerful JVM compilers like the new Graal can be with removing allocations. I can show a test case in Ruby running on Graal where there are many complicated objects that cross many method boundaries, and they're all entirely removed. http://www.chrisseaton.com/rubytruffle/pushing-pixels/ http://www.chrisseaton.com/rubytruffle/pushing-pixels/ Graal can do normal escape analysis, and so scalar replacements, but it can also do partial escape analysis - that is doing scalar replacement even if an object will later escape.
- bjourne 12y ago
  I don't believe that. Yes, the bump allocation step is just as fast as stack allocation because you just increment a pointer. But the extra allocation triggers more frequent collections of the nursery which is much slower than decrementing the stack pointer which is how you "free" objects on the stack. Intuitively nursery allocation just can't be as stack allocation but I haven't seen any benchmarks.
fmstephe 12y ago
In the financial space the Zing JVM from Azul is very popular, at least it is among the London developers. http://www.azulsystems.com/ http://www.azulsystems.com/ It has a pauseless generational collector called C4 (which is detailed in the Garbage Collection Handbook). Gil Tene, the CTO and founder of Azul, appeared on the Go mailing list a while back and outlined the steps towards implementing the C4 collector in Go. https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2BU https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2... This looks like progress towards that. But I don't know if C4 is any kind of concrete (or even speculative) goal. I would love it if standard Go was running a C4 GC implementation.
tomlu 12y ago
> Also hard upper limits on GC time are very helpful for those cases where allocations can't be reduced any further Yes, but 10ms is too long to be useful for games. I'd rather take 1ms every frame than 10ms sometimes.
- enneff 12y ago
  10ms is an upper bound. Games typically don't do a lot of allocations during gameplay, so it's unlikely that Go's GC would be an issue for most gaming applications. And, as I mention whenever this comes up, Minecraft is written in Java and suffers from huge GC pauses, yet it's one of the most successful games of the last decade.
  - Havvy 12y ago
    And yet the GC pauses are noticable and annoying. It's not game-breaking because Minecraft is just that good, but I doubt it'd work well for other games.
    - enneff 12y ago
      I brought up Minecraft as proof that gamers are willing to tolerate the occasional bit of lag. And, honestly, I haven't played an AAA game that doesn't lag out and drop frames from time to time. To return to the topic at hand: 60fps is 16ms per frame. The proposed GC would achieve sub-10ms pauses. So, worst case you might drop the odd frame. To assume that Go is unsuitable for games because of garbage collection is to ignore reality.
- eloff 12y ago
  I don't buy that argument. Lua has a much worse STW garbage collector and it's commonly used in games. Furthermore with games you can do tricks like allocating out of an arena with bump pointers and then resetting the pointer at the end of the frame. Lots of games are developed with JVM, .NET, or Lua. GC doesn't seem to be a show stopper for them, you just have to be smart about allocations.
  - 12y ago

by eloff 12y ago

munificent 12y ago

> (Java doesn't have stack allocations, but Go does.) Go also lets you have fields in place in structs, like C and C++. In Java, all fields (except primitives) are references.

seanmcdirmid 12y ago
C# has had structs since V1 I think.
bad_user 12y ago
Value Types are coming in the next version of Java. Yes, it is great to be able to allocate on the stack, but then Java has better garbage collectors, including a completely pauseless one available commercially. Even in this announcement the new GC mentioned for Go is non generational. I have a feeling that even Java's good old CMS is better. Java also does escape analisys. All in all, once value classes are in, it is going to be awesome.

_ak 12y ago

We use Go for a soft realtime application (RTB), and it works well enough. We essentially have two performance issues, one being the particular database we use (which is not directly related to Go itself), the other one being GC pressure. Before Go 1.3, too many allocations were _very_ noticeable with GC pauses sometimes being close to 100ms (which is a problem when your maximum RTT is 100ms), but Go 1.3 improved things enormously. I'm confident things will get even better with Go 1.4.

kator 12y ago
When you're 90ms into your bid calc and Go pauses for 10ms STW you will loose all your connections and timeout your bids. It's fine for basic stuff but if you want to really listen to full RTB bid stream and have real time responses you need pauseless GC to win. Or you have to budget for considerably less time to make a decision. Also there are places in RTB where you have 10ms RTT and Go fails ought right in that situation at scale. :-)
- _ak 12y ago
  I'm talking about online advertising. RTTs for common ad exchanges and SSPs are usually 100 to 120ms.

pcwalton 12y ago

The HotSpot JVM stack allocates (actually, even better: it SROAs) in very similar conditions to when Go does: when escape analysis decides that a value does not escape. Even if it doesn't stack allocate, bump allocation in the nursery ends up being about as fast as stack allocation (which is, as far as I know, why HotSpot chose only to lock elide and not to stack allocate when they implemented escape analysis).

crawshaw 12y ago
I haven't looked into the JVM's stack allocation logic in several years, so it may have improved to the point where that is now true. (It wasn't so in 2010.) However, Go does have features that make it easier to control allocations than Java. One example, type s struct { x int32 y int32 } v := make([]s, 1e5) The object v uses 1e5 * 8 bytes of storage. Equivalent Java code would require boxing of each s value. That is, instead of []s, you end up with []*s and allocating each s.
- pcwalton 12y ago
  Sure, I wasn't disputing that Go has more interior allocation features than Java does.
- chrisseaton 12y ago
  I wrote a blog post that showed how powerful JVM compilers like the new Graal can be with removing allocations. I can show a test case in Ruby running on Graal where there are many complicated objects that cross many method boundaries, and they're all entirely removed. http://www.chrisseaton.com/rubytruffle/pushing-pixels/ http://www.chrisseaton.com/rubytruffle/pushing-pixels/ Graal can do normal escape analysis, and so scalar replacements, but it can also do partial escape analysis - that is doing scalar replacement even if an object will later escape.
bjourne 12y ago
I don't believe that. Yes, the bump allocation step is just as fast as stack allocation because you just increment a pointer. But the extra allocation triggers more frequent collections of the nursery which is much slower than decrementing the stack pointer which is how you "free" objects on the stack. Intuitively nursery allocation just can't be as stack allocation but I haven't seen any benchmarks.

fmstephe 12y ago

In the financial space the Zing JVM from Azul is very popular, at least it is among the London developers. http://www.azulsystems.com/ http://www.azulsystems.com/ It has a pauseless generational collector called C4 (which is detailed in the Garbage Collection Handbook). Gil Tene, the CTO and founder of Azul, appeared on the Go mailing list a while back and outlined the steps towards implementing the C4 collector in Go. https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2BU https://groups.google.com/forum/#!topic/golang-dev/GvA0DaCI2... This looks like progress towards that. But I don't know if C4 is any kind of concrete (or even speculative) goal. I would love it if standard Go was running a C4 GC implementation.

tomlu 12y ago

> Also hard upper limits on GC time are very helpful for those cases where allocations can't be reduced any further Yes, but 10ms is too long to be useful for games. I'd rather take 1ms every frame than 10ms sometimes.

enneff 12y ago
10ms is an upper bound. Games typically don't do a lot of allocations during gameplay, so it's unlikely that Go's GC would be an issue for most gaming applications. And, as I mention whenever this comes up, Minecraft is written in Java and suffers from huge GC pauses, yet it's one of the most successful games of the last decade.
- Havvy 12y ago
  And yet the GC pauses are noticable and annoying. It's not game-breaking because Minecraft is just that good, but I doubt it'd work well for other games.
  - enneff 12y ago
    I brought up Minecraft as proof that gamers are willing to tolerate the occasional bit of lag. And, honestly, I haven't played an AAA game that doesn't lag out and drop frames from time to time. To return to the topic at hand: 60fps is 16ms per frame. The proposed GC would achieve sub-10ms pauses. So, worst case you might drop the odd frame. To assume that Go is unsuitable for games because of garbage collection is to ignore reality.
eloff 12y ago
I don't buy that argument. Lua has a much worse STW garbage collector and it's commonly used in games. Furthermore with games you can do tricks like allocating out of an arena with bump pointers and then resetting the pointer at the end of the frame. Lots of games are developed with JVM, .NET, or Lua. GC doesn't seem to be a show stopper for them, you just have to be smart about allocations.
- 12y ago