4 ms·
Go memory usage will definitely not be an order of magnitude less. This is something that cannot be emphasized enough: the idea that a Go program of any substa
by mdasen 10y ago
Go memory usage will definitely not be an order of magnitude less. This is something that cannot be emphasized enough: the idea that a Go program of any substantial size would use an order of magnitude less memory than a Java one is completely wrong.
First, the "proof": https://days2011.scala-lang.org/sites/days2011/files/ws3-1-Hundt.pdf https://days2011.scala-lang.org/sites/days2011/files/ws3-1-H.... That's a paper from Google. The memory footprint of the Go program was 501MB, the Java version 617MB, and the Scala version 293MB. Now, given that the Scala version and Java versions were so far off in memory just tells you that a bunch of it just depends on some specific things used. Most importantly, it should dispel the myth that Java uses more memory than Go. The JVM isn't perfect, but arguing that Java uses an order of magnitude more memory is simply wrong.
Java has a bad reputation for memory usage among people who have never run substantial programs because a Java hello-world (or other small programs) will often allocate itself a decently sized heap. Java programs also take a while to start up because of the JVM, but that doesn't make Java slow.
Go certainly has some good things. Structs can help with memory locality and avoiding some unnecessary pointers. But that's not going to help one with attaining an order of magnitude better memory usage.
But Go's garbage collector isn't made for memory efficiency, but rather short pause times. That can be great, but Go seems to have made the choice that shorter pause times are preferable to throughput (actually accomplishing work) or using less memory (might as well throw more RAM at the problem).
Now let's talk about Kafka specifically. It's been a while since I've looked at it, but Kafka actually doesn't keep the data on the JVM heap. It writes to the filesystem using the kernel's built-in async flushing and uses sendfile to write data directly from the OS page cache to sockets. In a lot of ways, Kafka bypasses its own runtime and just leans on the OS for the heavy lifting.
I like Go, but it isn't magical. It certainly doesn't offer an order of magnitude memory improvement over Java. Go has some great bits, but if you're thinking that you'll get an order of magnitude memory improvement, it's not there. It might even use more memory than Java.
- Thaxll 10y agoAnd yet it does, try to create a simple API in Java vs Go and look at the memory of both program, there is no need for white paper to see that the JVM at least will allocate 100MB~~+ where Go will start bellow 10MB.
- caconym_ 10y agoHe addressed this: > Java has a bad reputation for memory usage among people who have never run substantial programs because a Java hello-world (or other small programs) will often allocate itself a decently sized heap. Java programs also take a while to start up because of the JVM, but that doesn't make Java slow.
- Thaxll 10y agoIn a world where we break down monolithic applications into smaller services it's clear that the JVM memory model at startup is way bigger than any other language, I'm not sure how one can argue about that. Since those "micro services" have just a few features, their memory won't grow up that much over time, so the startup memory is an important factor.
- caconym_ 10y agoI'm not a Java expert (or even a semi-frequent user of it) but I think it would be pretty crazy if you could not override the default initial heap size and, indeed, it seems that you can: http://stackoverflow.com/questions/1951347/xms-initial-heap-size-or-minimum-heap-size http://stackoverflow.com/questions/1951347/xms-initial-heap-...
- dastbe 10y agoI agree that the focus on memory size is misguided, but I do think that startup times are critical even (especially?) in production. Compounding the startup time of the JVM w/ classpath searching and reflection heavy libraries and frameworks really impact dynamic scaling and deployment. When I can setup monitoring that dynamically scales my application fleet in seconds, having to eat the cost of both a large binary (JDK + all of my various libraries) and then the startup cost of that application hurts. I'm not saying that Java applications can't be built to be small and fast, but its not how most of industry had thought about building them. Golang has the benefit of not having that legacy to begin with.
- 10y ago
- geodel 10y agoThe link you gave was analysed and Go code fixed by Russ. See this: https://blog.golang.org/profiling-go-programs https://blog.golang.org/profiling-go-programs Better and updated link for benchmark is following: http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=scala http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan... Here are web related benchmarks: https://www.techempower.com/benchmarks/#section=data-r13&hw=ph&test=db&l=4fsz5r https://www.techempower.com/benchmarks/#section=data-r13&hw=... Following link details for Java memory bloat: https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/memory-efficient-java-tutorial.pdf https://www.cs.virginia.edu/kim/publicity/pldi09tutorials/me... In short Java memory bloat is not myth but well known fact.