4 ms·

I'd be curious to hear performance numbers (assuming a reasonable front-end server to this library). I get it that the replicated in-memory caching part is val

I'd be curious to hear performance numbers (assuming a reasonable front-end server to this library). I get it that the replicated in-memory caching part is valuable. But (from painful experiences with Java) I also fear that a GC based memory management system is anti-optimal for an in-memory cache of small objects, especially as the size of heap grows to beyond a couple of GB( * ). (* ) http://cdn.parleys.com/p/5148922a0364bc17fc56c60f/GarbageCollection.pdf http://cdn.parleys.com/p/5148922a0364bc17fc56c60f/GarbageCol...

thrownaway2424 13y ago
Go's garbage collector sucks pretty hard for large heaps. A 16GB heap that's mostly live objects will take tens of seconds per round, during which no goroutines will run (it's a stop-the-world collector). Basically it's 15 years behind Java in this regard. If you're worried about this, stick with C++.
- nine_k 13y ago
  Is this figure for Go 1.0 or 1.1? The latter is said to have a much-improved parallel GC. http://golang.org/doc/go1.1#performance http://golang.org/doc/go1.1#performance
  - eggnet 13y ago
    It is still stop the world.
  - eggnet 13y ago
    It is still stop the world.
- lazyjones 13y ago
  A 16GB heap that's mostly live objects will take tens of seconds per round The situation you describe, as you describe it, is (no longer - was it ever?) a pathological case. I run stuff with 12-20GB active objects and a few dozen http hits a second and the only times I get a latency over 100ms is when I reload/refresh some data from disk (1.5GB of gob), then it goes to 150-250ms typically.
- dschiptsov 13y ago
  Basically it's 15 years behind Java in this regard this is fanboysm or just ignorance. It does not matter how sophisticated GC code is when you have millions of tiny objects from from crappy Java code with useless copying and referencing. This is why it uses so much memory. It is very difficult to find any worse thing that Java in terms of wasting of memory. Take any Hadoop none and measure the ratio between memory used by Java processes and amount of actual data stored in memory. Near factor of 2?
  - coldtea 13y ago
    >It does not matter how sophisticated GC code is when you have millions of tiny objects from from crappy Java code with useless copying and referencing. This is why it uses so much memory. Or, you know, you can avoid writing "crappy" code. Whereas with Go's GC, even proper code will have problems. You are arguing something that is beyond discussion: Java's GC is better than Go's.
    - dschiptsov 13y ago
      Or, you know, you can avoid writing "crappy" code. - In Java - hardly. It requires a discipline and skills beyond capacity of a typical developer. This phenomena could be called "memory is cheap and JVM will handle everything". Even in Clojure code, for example, they implement a node of a binary tree as a hash-table which is a meaningless waste. Position based selectors would be good enough. Java's GC is better than Go's - yes, in theory, on paper. In practice well-written Go code could outperform typical Java code, even with less sophisticated GC. Of course, I cannot prove that, but there are some intuitions to support such claims.
- jerf 13y ago
  I couldn't find in the code where the objects are actually stored, but theoretically there's nothing stopping you in Go from allocating yourself up an array of X bytes (for X in the mega or gigabytes) and completely managing it yourself, leaving the GC merely for incidental activities, which themselves can be minimized through various design patterns and some care. It is true that you should describe Go as a garbage-collected language, but unlike a lot of other such languages, Go has C-like mutable arrays readily available. In practice it's more like a sort of hybrid, in that even though it's garbage collected you have a lot of opportunities to write code that still doesn't really use it, without having to "drop down to" C or something. I still hope to see some improvements in it before I could make a big commitment to it, but I've certainly got my eye on it.
  - deleted 13y ago
    [deleted]
  - coldtea 13y ago
    >I couldn't find in the code where the objects are actually stored, but theoretically there's nothing stopping you in Go from allocating yourself up an array of X bytes (for X in the mega or gigabytes) and completely managing it yourself, Nothing stops you, except: - The determination NOT to do something the computer can do - Previous exposure to a proper, modern GC - The fact that you used Go to get away from this kind of shit in the first place
    - jerf 13y ago
      I did not mean this is a technique you'd use in general. I meant specifically in the context of groupcache, where an array of bytes that you manage yourself is a reasonable approach to the problem. If you don't want to do that, don't. I'd never start the prototype with that functionality, for instance. But when you discover that you need it, Go permits it in a way that Python or Perl do not. Go is GC'ed and you are free to use that to the extent you want, but it's easier to escape from the GC than it is in many GC'ed languages. It may not be obvious from a casual reading of the spec, but there's a lot of ways around garbage in Go. Not like, say, Rust, and certainly not the same level of Raw Unstoppable Power as C++ (at a corresponding huge complexity cost), but it's a very interesting middle ground. Note that if you were writing groupcache in Java, you'd probably end up doing the same thing and just allocating an expanse of bytes which you manage yourself.
tux21b 13y ago
Go's GC isn't nearly as powerful as any of the many GC implementations for the JVM, but Go's approach is also quite differently. In Go, you can reduce / avoid the garbage that you create. And when the GC turns out to be the limiting factor of your cache, then you can also mmap some dedicated pages from the OS and manage the memory on your own. But I guess that won't be necessary. Do you have any issues with Go's GC today?
- chetanahuja 13y ago
  Well I don't have issues with Go's GC today because my horrific experiences with various JVM's has been a powerful deterrent for any future experiences in performance critical large-heap server processes with a GC controlled heap. Other than that, I really really like the design of Go. This is in addition to the factor of my faith in the solid team and company backing the language. I'm quite interested in the two mitigations you mentioned. Especially the "manage memory on your own option" ? I was not aware that there was a go language (that is, not a C extension) blessed way to do such a thing. As for "reduce/avoid" creating garbage, how can I achieve that for a large heap caching application (like the one we're talking about here) ?
  - tux21b 13y ago
    The easiest way would be to store everything you can on the stack. The Go compiler does a very good job at escape analysis and tries to put everything on the stack that does not escape. Therefore, it's often better to use output parameters instead of return values to reduce (or eliminate) the number of allocations. Some careful thoughts about the memory layout of your structs, especially which of them should be embedded and / or passed around by pointers and which of them shouldn't, might also pay off. Another common optimization is to put the allocated objects back to a memory pool for later use. Take a look at the bufCache channel [1] from the bufio package for example (the http and the json package are using the same trick). [1]: http://tip.golang.org/src/pkg/bufio/bufio.go#L79 http://tip.golang.org/src/pkg/bufio/bufio.go#L79
  - twotwotwo 13y ago
    In any GC'd language, the less you allocate the less you GC, so "recycling" old objects when you need new ones can help. So where you'd normally make() a []byte, you can instead get one from a list or buffered channel you made of old discarded []bytes. So that gets you back to malloc/free/new/delete, but you can at least do it only for the most performance-relevant allocations in your program and keep being lazy everywhere else. Another sometimes-useful way to save allocations--not for a cache, but in general--is just to turn short-lived allocations into longer-lived ones: if fooBuf is used and thrown away by each of several calls to obj.baz(), then make a single fooBuf and store it as a (private) field on obj, assuming that doesn't present thread-safety or other problems in the specific context. There are certainly things you wouldn't use Go for, but now you know more about reducing memory pressure in GC'd languages. :)
bradfitz 13y ago
Go doesn't scan big []byte/strings, so the GC isn't a problem.
- chetanahuja 13y ago
  Brad could you elaborate on this statement please. I tried a few google searches to find documentation on that behaviour but failed.
  - twotwotwo 13y ago
    Roughly: a GC has to look at the pointers in use to see what objects are still reachable. But it doesn't have to look at the contents of strings, byte slices, int arrays, etc., because, by definition, those don't contain pointers. For dl.google.com, the first user of groupcache, it sounds like the Go runtime was working with relatively few, relatively large chunks of memory (think 2MB chunks of a Chrome binary). Since that doesn't have so many pointer-containing objects, just big chunks of inert data that don't need scanned, it's easyish. On the other hand, something like throwaway2424's case could be a GC stress test--many gigs of RAM holding a network of small, pointer-filled objects. Really, the tl;dr may be to prototype/measure if you need to know how a GC will do. Assumptions are easy; data is hard.
    - chetanahuja 13y ago
      Ahh got it. Also, the reason Brad specifically mentions big arrays of bytes/strings is that they don't look like pointers. Soo.... I'm guessing you still want to stay away from huge arrays of ints (or whatever type has the same size as pointers on your platform). Right ? And as for measuring... it's not cheap (developer time wise) to build a realistic enough model of your application and put realistic enough loads on it for long periods of time to test out each of your hypotheses (and GC theories do require long periods of load). So you have to narrow the space of choices informed by past experiences and sometimes by gathering semi-reliable folk lore. I'm currently engaging in the latter activity :-)
      - 13y ago

thrownaway2424 13y ago

Go's garbage collector sucks pretty hard for large heaps. A 16GB heap that's mostly live objects will take tens of seconds per round, during which no goroutines will run (it's a stop-the-world collector). Basically it's 15 years behind Java in this regard. If you're worried about this, stick with C++.

nine_k 13y ago
Is this figure for Go 1.0 or 1.1? The latter is said to have a much-improved parallel GC. http://golang.org/doc/go1.1#performance http://golang.org/doc/go1.1#performance
- eggnet 13y ago
  It is still stop the world.
- eggnet 13y ago
  It is still stop the world.
lazyjones 13y ago
A 16GB heap that's mostly live objects will take tens of seconds per round The situation you describe, as you describe it, is (no longer - was it ever?) a pathological case. I run stuff with 12-20GB active objects and a few dozen http hits a second and the only times I get a latency over 100ms is when I reload/refresh some data from disk (1.5GB of gob), then it goes to 150-250ms typically.
dschiptsov 13y ago
Basically it's 15 years behind Java in this regard this is fanboysm or just ignorance. It does not matter how sophisticated GC code is when you have millions of tiny objects from from crappy Java code with useless copying and referencing. This is why it uses so much memory. It is very difficult to find any worse thing that Java in terms of wasting of memory. Take any Hadoop none and measure the ratio between memory used by Java processes and amount of actual data stored in memory. Near factor of 2?
- coldtea 13y ago
  >It does not matter how sophisticated GC code is when you have millions of tiny objects from from crappy Java code with useless copying and referencing. This is why it uses so much memory. Or, you know, you can avoid writing "crappy" code. Whereas with Go's GC, even proper code will have problems. You are arguing something that is beyond discussion: Java's GC is better than Go's.
  - dschiptsov 13y ago
    Or, you know, you can avoid writing "crappy" code. - In Java - hardly. It requires a discipline and skills beyond capacity of a typical developer. This phenomena could be called "memory is cheap and JVM will handle everything". Even in Clojure code, for example, they implement a node of a binary tree as a hash-table which is a meaningless waste. Position based selectors would be good enough. Java's GC is better than Go's - yes, in theory, on paper. In practice well-written Go code could outperform typical Java code, even with less sophisticated GC. Of course, I cannot prove that, but there are some intuitions to support such claims.
jerf 13y ago
I couldn't find in the code where the objects are actually stored, but theoretically there's nothing stopping you in Go from allocating yourself up an array of X bytes (for X in the mega or gigabytes) and completely managing it yourself, leaving the GC merely for incidental activities, which themselves can be minimized through various design patterns and some care. It is true that you should describe Go as a garbage-collected language, but unlike a lot of other such languages, Go has C-like mutable arrays readily available. In practice it's more like a sort of hybrid, in that even though it's garbage collected you have a lot of opportunities to write code that still doesn't really use it, without having to "drop down to" C or something. I still hope to see some improvements in it before I could make a big commitment to it, but I've certainly got my eye on it.
- deleted 13y ago
  [deleted]
- coldtea 13y ago
  >I couldn't find in the code where the objects are actually stored, but theoretically there's nothing stopping you in Go from allocating yourself up an array of X bytes (for X in the mega or gigabytes) and completely managing it yourself, Nothing stops you, except: - The determination NOT to do something the computer can do - Previous exposure to a proper, modern GC - The fact that you used Go to get away from this kind of shit in the first place
  - jerf 13y ago
    I did not mean this is a technique you'd use in general. I meant specifically in the context of groupcache, where an array of bytes that you manage yourself is a reasonable approach to the problem. If you don't want to do that, don't. I'd never start the prototype with that functionality, for instance. But when you discover that you need it, Go permits it in a way that Python or Perl do not. Go is GC'ed and you are free to use that to the extent you want, but it's easier to escape from the GC than it is in many GC'ed languages. It may not be obvious from a casual reading of the spec, but there's a lot of ways around garbage in Go. Not like, say, Rust, and certainly not the same level of Raw Unstoppable Power as C++ (at a corresponding huge complexity cost), but it's a very interesting middle ground. Note that if you were writing groupcache in Java, you'd probably end up doing the same thing and just allocating an expanse of bytes which you manage yourself.

tux21b 13y ago

Go's GC isn't nearly as powerful as any of the many GC implementations for the JVM, but Go's approach is also quite differently. In Go, you can reduce / avoid the garbage that you create. And when the GC turns out to be the limiting factor of your cache, then you can also mmap some dedicated pages from the OS and manage the memory on your own. But I guess that won't be necessary. Do you have any issues with Go's GC today?

chetanahuja 13y ago
Well I don't have issues with Go's GC today because my horrific experiences with various JVM's has been a powerful deterrent for any future experiences in performance critical large-heap server processes with a GC controlled heap. Other than that, I really really like the design of Go. This is in addition to the factor of my faith in the solid team and company backing the language. I'm quite interested in the two mitigations you mentioned. Especially the "manage memory on your own option" ? I was not aware that there was a go language (that is, not a C extension) blessed way to do such a thing. As for "reduce/avoid" creating garbage, how can I achieve that for a large heap caching application (like the one we're talking about here) ?
- tux21b 13y ago
  The easiest way would be to store everything you can on the stack. The Go compiler does a very good job at escape analysis and tries to put everything on the stack that does not escape. Therefore, it's often better to use output parameters instead of return values to reduce (or eliminate) the number of allocations. Some careful thoughts about the memory layout of your structs, especially which of them should be embedded and / or passed around by pointers and which of them shouldn't, might also pay off. Another common optimization is to put the allocated objects back to a memory pool for later use. Take a look at the bufCache channel [1] from the bufio package for example (the http and the json package are using the same trick). [1]: http://tip.golang.org/src/pkg/bufio/bufio.go#L79 http://tip.golang.org/src/pkg/bufio/bufio.go#L79
- twotwotwo 13y ago
  In any GC'd language, the less you allocate the less you GC, so "recycling" old objects when you need new ones can help. So where you'd normally make() a []byte, you can instead get one from a list or buffered channel you made of old discarded []bytes. So that gets you back to malloc/free/new/delete, but you can at least do it only for the most performance-relevant allocations in your program and keep being lazy everywhere else. Another sometimes-useful way to save allocations--not for a cache, but in general--is just to turn short-lived allocations into longer-lived ones: if fooBuf is used and thrown away by each of several calls to obj.baz(), then make a single fooBuf and store it as a (private) field on obj, assuming that doesn't present thread-safety or other problems in the specific context. There are certainly things you wouldn't use Go for, but now you know more about reducing memory pressure in GC'd languages. :)

bradfitz 13y ago

Go doesn't scan big []byte/strings, so the GC isn't a problem.

chetanahuja 13y ago
Brad could you elaborate on this statement please. I tried a few google searches to find documentation on that behaviour but failed.
- twotwotwo 13y ago
  Roughly: a GC has to look at the pointers in use to see what objects are still reachable. But it doesn't have to look at the contents of strings, byte slices, int arrays, etc., because, by definition, those don't contain pointers. For dl.google.com, the first user of groupcache, it sounds like the Go runtime was working with relatively few, relatively large chunks of memory (think 2MB chunks of a Chrome binary). Since that doesn't have so many pointer-containing objects, just big chunks of inert data that don't need scanned, it's easyish. On the other hand, something like throwaway2424's case could be a GC stress test--many gigs of RAM holding a network of small, pointer-filled objects. Really, the tl;dr may be to prototype/measure if you need to know how a GC will do. Assumptions are easy; data is hard.
  - chetanahuja 13y ago
    Ahh got it. Also, the reason Brad specifically mentions big arrays of bytes/strings is that they don't look like pointers. Soo.... I'm guessing you still want to stay away from huge arrays of ints (or whatever type has the same size as pointers on your platform). Right ? And as for measuring... it's not cheap (developer time wise) to build a realistic enough model of your application and put realistic enough loads on it for long periods of time to test out each of your hypotheses (and GC theories do require long periods of load). So you have to narrow the space of choices informed by past experiences and sometimes by gathering semi-reliable folk lore. I'm currently engaging in the latter activity :-)
    - 13y ago