7 ms·

> Better bindings for calling Go from Java. If that is available for general Java code (i.e. uses JNI) and not just for Android, that could be really huge for

> Better bindings for calling Go from Java. If that is available for general Java code (i.e. uses JNI) and not just for Android, that could be really huge for Go. Writing performance-sensitive low-level code in Java is still fairly painful, wheras Go still isn't great for writing big programs. I can imagine (for example) Hadoop, Lucene/Elasticsearch and PrestoDB all using this.

eternalban 12y ago
You jest? Go is GC'd just like JVM. The only possible benefit -- even if Go catches up at runtime -- is the compact form of memory objects in Go vs Java object. But then again, if you are writing such systems (in either language) you are very likely to spend quite a lot of time in 'unsafe' land.
- enneff 12y ago
  > if you are writing such systems (in either language) you are very likely to spend quite a lot of time in 'unsafe' land. I don't think that's necessarily true. Go does a much better job than Java at letting you manage your allocations and re-use memory. You can write tight, performance-critical code in Go without resorting to 'unsafe'; it just requires care, as it does in any language.
  - tylertreat 12y ago
    I don't know if I completely agree with this. To write truly performance-critical Go, you end up throwing away many of the language's qualities. Channels are slow, defers add overhead, interfaces add overhead (e.g. I2T), etc. Don't get me wrong, I like Go, but in my (and others I work with) experience, it's not the right choice for performance-critical systems.
    - enneff 12y ago
      I disagree with "channels are slow". They may be faster than you think. Defers needn't add as much overhead as they do today; this can be fixed. I think it really depends on your definition of "performance-critical". I agree Go isn't suitable for all performance-critical tasks, but it covers a vast swathe of them quite comfortably.
      - eternalban 12y ago
        > this can be fixed. I believe this and am counting on it. Go2.0 and beyond should be a solid choice. You should also note that it is entirely understood that more mature tech e.g. JVM have had the benefit of multibillion Dollar investment by SUN, IBM, Oracle, etc. I feel it is regrettable that (imo valid and reasonable) criticism of what is currently not up to par with this tech always seemingly requires a disclaimer that "I love Go". I have been using this language since the day it was released. I know it fairly well. I like it. But excessive hype and sensitivity around it is frankly somewhat irritating. peace out and happy v. day Go <3
        enneff 12y ago
        Valentines day was yesterday for me. You wouldn't see me spending my time responding here if it were today!
    - eternalban 12y ago
      Not an uncommon view: https://twitter.com/kellabyte/status/564531804837654528 https://twitter.com/kellabyte/status/564531804837654528 (That's not me, to be clear.) We all like Go. Just wish we could discuss these matters without unduly raising temperatures. "It's just code".
      - twotwotwo 12y ago
        Sommers is an absolute beast, does totally astounding work and is an asset to the community. But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire (https://twitter.com/kellabyte/status/547063455048404992 https://twitter.com/kellabyte/status/547063455048404992). In other words, pushing the hardware absolutely to its limits is another game (Sommers' game :) ) from simply getting a tricky part of your app performing at the level you need. And, confusingly, "high-performance" or "performance-critical" are terms you might use to describe either task. For context, the person whose presentation triggered Kelly's comment, MIT scholar Neha Narula, was experimenting with an 80 core machine, and she replied "it's kind of amazing I could push Go that far," and "to be fair they weren't really optimizing for my use case :)" -- her whole presentation is on YouTube at https://www.youtube.com/watch?v=Mbg1COjhsJU https://www.youtube.com/watch?v=Mbg1COjhsJU (some wild stuff--she got improvements for >48-core machines pushed into the Go GC) and those replies I quoted are at https://twitter.com/neha/status/564569903219634176 https://twitter.com/neha/status/564569903219634176 .
        danieldk 12y ago
        But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire We are not all writing web apps. Some of use are in machine learning, NLP, signal processing, etc. where squeezing out as much performance as possible does matter. In those fields Go is still weak. No autovectorization, no OpenMP, no direct CUDA integration, GC overhead, etc. Luckily, this can often be worked around since cgo is so good. One can write performance-intensive parts in C or C++, compile with the latest gcc or clang and link it with the Go code to drive it. This is often an understated advantage of Go compared to Java, where JNI calls are expensive. But the Go camp always advocate for porting everything to Go (because fast compile times).
    - twotwotwo 12y ago
      I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com). CloudFlare has it sitting in the middle of every request for some sites (in the form of Railgun, their delta compression for pages). I'm not exactly a perf ninja and got some Go code packing Wikipedia history dumps at >100 MB/s. Dropbox uses Go for, their words, "our performance-critical backends" (https://blogs.dropbox.com/tech/2014/07/open-sourcing-our-go-libraries/ https://blogs.dropbox.com/tech/2014/07/open-sourcing-our-go-...). So folks manage to heavy lifting with it. I see "I couldn't possibly use X for performance reasons" here a lot, as an almost immediate response about a wide range of different tools, and two things come up in my mind: 1) you can always set the standard arbitrarily high. If you're doing AAA shoot-'em-up games or something, yes, use something else. 2) you're often not as good at tools that're new or different. It's possible you could go further with just a few more tricks about profiling or using free pools or whatever, or a little more info about how your code executes or what the runtime does. FWIW, if you hit a specific wall that's a problem for your app, folks on golang-nuts (or StackOverflow, where I've hung out sometimes) are often happy to try and help. Hope this is more helpful than fussy. Mostly just don't want folks to be discouraged into thinking certain things are impossible when in some cases they're really being done in production out there.
      - coldtea 12y ago
        >I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com). Not as impressive as it sounds. Most of it is cached in memory anyway...
      - tylertreat 12y ago
        Agreed with setting the standard arbitrarily high. It depends on what problem you're trying to solve—just like deciding when to use any other tool. Context matters. Where I work, we've been trying to build some super-fast systems and things like the GC and even calling interface methods matter. We open sourced some data structures that show just how optimized we're trying to get (https://github.com/Workiva/go-datastructures https://github.com/Workiva/go-datastructures), but in hindsight, Go might not have been the right tool for the problem.
  - eropple 12y ago
    I don't believe that this is true. You use the exact same patterns and tricks in both for perf-critical code: off-GC primitives and object pools rule the day. What are you considering a "much better" option available in Go? (I considered stack-allocated structs in Go, but honestly that doesn't strike me as a particularly major thing; it may be slightly more terse, but fundamentally the same behavior.)
    - enneff 12y ago
      Values ("off-GC primitives?") are first class-citizens in Go. Stack-allocated structs have much more value than you think.
      - eropple 12y ago
        More value than I think? I'm pretty sure that I know exactly their value. I'll maintain that stack-allocated structs are a nice-to-have on top of the sort of stack-allocated primitives that Java provides; I'd like to have those structs as a convenience when I write Java, which is why I alluded to terseness in my prior post. But there's very little (or should be, I suppose, depending on implementation) performance difference to speak of between a struct and parallel declaration of the same values.
        enneff 12y ago
        You said: > I considered stack-allocated structs in Go, but honestly that doesn't strike me as a particularly major thing; But for me they're one of the major ways in which I control memory use in Go programs. So, yeah, I think you underestimate them. No condescension implied. Apologies if it came across that way. I think we probably agree more than we disagree.
        eropple 12y ago
        Yeah, I edited it out because after I took a breath I figured you didn't mean it that way. =) Hugs all around. But I really don't think it's a major thing, because there's no meaningful difference, in terms of performance, between "struct { int x; int y; } A" and "int Ax; int Ay;". I'm suspicious of claims that Go, as fundamentally a not that different language to a JVM language, is going to yield significant performance benefits. I'm not saying they aren't a nice convenience (value types are one of the things I love about C# when I'm writing code under MonoGame), I'm saying that I can't think of a reasonable way, barring bugs or short-sighted implementations, that they made for faster code.
- justinsb 12y ago
  If you're mmaping a file and processing the majority of it, you really are typically doing that in C today if you care about performance. You can get acceptable performance in Java with ByteBuffers, but because of the lack of value types it doesn't feel like Java any more. Go should be able to get much closer to C's performance, while still being closer to idiomatic Go code. And Go can plug in small pieces of C code / assembler for the really performance critical stuff. All this can be done with JNI, but JNI is so un-fun that I can see Go making big inroads here.
  - kasey_junk 12y ago
    People use mmapped files all the time in performance critical Java. Typically using the unsafe packages. In fact, if you want to communicate with C/assembler level things, this is the way people who do fast Java do it because JNI is very slow. If you want to use an abstraction around ByteBuffers that feels like value types take a look at the javalution structs. As a counter to your argument, C# has had value types for quite a while and has not achieved Java levels of performance, so those in and of themselves aren't enough. Mostly thats because if you are doing any allocation in fast code you are doing it wrong regardless of language. Even in C object pools and arena allocation are standard for performance critical work. The gap between Java and C (or really C++) right now is almost entirely around control of the memory model, not allocation (that said, I'd love the JVM to have value types and am glad that go started with them). If anything will allow go to achieve better performance than Java its that it will be able to incorporate the lessons learned from Java without the support burden. I do think the constrained nature of go will give it a very good chance at impressive performance.
    - tracker1 12y ago
      I don't think C# vs Java performance can always be directly compared.. C#'s SuppressUnmanagedCodeSecurity attribute for PInvoke interfaces, when you can use it, reduces C#'s call overhead to something like 20% over using a Managed C++ library as a bridge. If you don't need to pass complex numbers (doubles) it isn't bad at all. Just depends on your needs. Then again, depending on your needs, being able to scale out or up is far more important than raw performance characteristics... just depends on your needs.
- perturbation 12y ago
  Doesn't Go have the potential to be faster than Java since it's compiled to native code (rather than compiled to byte code)?
  - ori_b 12y ago
    Java is also compiled to native code, but only after it runs for a bit.
    - kryptiskt 12y ago
      Android does AOT compiling with ART, it became the default in Lollipop.
      - Roboprog 12y ago
        Android does the right thing for a transient GUI client program on a device with little memory. HotSpot JIT does the right thing for a (long running) server program on big, dedicated, hardware.
  - vardump 12y ago
    JIT has potential to be faster than compiled native code. Better runtime information. Compiled code needs to cover all potential options and this means more instructions to execute. Say a variable value is set through a command line option to be a certain value. Compiled native code has to assume the value to be dynamic, but a JIT can optimize it away, effectively hardcoding it for that particular invocation. Same applies to more complicated type of software. Some configuration and invocation parameters tend to be effectively static during that particular invocation. JITs can capitalize on this fact. JITs have also better chance to adapt to exact hardware it's running on. Compiled code is forced to make one or a limited number of assumptions of available CPU hardware configuration. In the end, both options are running compiled native code. JIT just does it a bit before running. Of course current reality is the opposite, but the key word here is potential.
    - perturbation 12y ago

eternalban 12y ago

You jest? Go is GC'd just like JVM. The only possible benefit -- even if Go catches up at runtime -- is the compact form of memory objects in Go vs Java object. But then again, if you are writing such systems (in either language) you are very likely to spend quite a lot of time in 'unsafe' land.

enneff 12y ago
> if you are writing such systems (in either language) you are very likely to spend quite a lot of time in 'unsafe' land. I don't think that's necessarily true. Go does a much better job than Java at letting you manage your allocations and re-use memory. You can write tight, performance-critical code in Go without resorting to 'unsafe'; it just requires care, as it does in any language.
- tylertreat 12y ago
  I don't know if I completely agree with this. To write truly performance-critical Go, you end up throwing away many of the language's qualities. Channels are slow, defers add overhead, interfaces add overhead (e.g. I2T), etc. Don't get me wrong, I like Go, but in my (and others I work with) experience, it's not the right choice for performance-critical systems.
  - enneff 12y ago
    I disagree with "channels are slow". They may be faster than you think. Defers needn't add as much overhead as they do today; this can be fixed. I think it really depends on your definition of "performance-critical". I agree Go isn't suitable for all performance-critical tasks, but it covers a vast swathe of them quite comfortably.
    - eternalban 12y ago
      > this can be fixed. I believe this and am counting on it. Go2.0 and beyond should be a solid choice. You should also note that it is entirely understood that more mature tech e.g. JVM have had the benefit of multibillion Dollar investment by SUN, IBM, Oracle, etc. I feel it is regrettable that (imo valid and reasonable) criticism of what is currently not up to par with this tech always seemingly requires a disclaimer that "I love Go". I have been using this language since the day it was released. I know it fairly well. I like it. But excessive hype and sensitivity around it is frankly somewhat irritating. peace out and happy v. day Go <3
      - enneff 12y ago
        Valentines day was yesterday for me. You wouldn't see me spending my time responding here if it were today!
  - eternalban 12y ago
    Not an uncommon view: https://twitter.com/kellabyte/status/564531804837654528 https://twitter.com/kellabyte/status/564531804837654528 (That's not me, to be clear.) We all like Go. Just wish we could discuss these matters without unduly raising temperatures. "It's just code".
    - twotwotwo 12y ago
      Sommers is an absolute beast, does totally astounding work and is an asset to the community. But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire (https://twitter.com/kellabyte/status/547063455048404992 https://twitter.com/kellabyte/status/547063455048404992). In other words, pushing the hardware absolutely to its limits is another game (Sommers' game :) ) from simply getting a tricky part of your app performing at the level you need. And, confusingly, "high-performance" or "performance-critical" are terms you might use to describe either task. For context, the person whose presentation triggered Kelly's comment, MIT scholar Neha Narula, was experimenting with an 80 core machine, and she replied "it's kind of amazing I could push Go that far," and "to be fair they weren't really optimizing for my use case :)" -- her whole presentation is on YouTube at https://www.youtube.com/watch?v=Mbg1COjhsJU https://www.youtube.com/watch?v=Mbg1COjhsJU (some wild stuff--she got improvements for >48-core machines pushed into the Go GC) and those replies I quoted are at https://twitter.com/neha/status/564569903219634176 https://twitter.com/neha/status/564569903219634176 .
      - danieldk 12y ago
        But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire We are not all writing web apps. Some of use are in machine learning, NLP, signal processing, etc. where squeezing out as much performance as possible does matter. In those fields Go is still weak. No autovectorization, no OpenMP, no direct CUDA integration, GC overhead, etc. Luckily, this can often be worked around since cgo is so good. One can write performance-intensive parts in C or C++, compile with the latest gcc or clang and link it with the Go code to drive it. This is often an understated advantage of Go compared to Java, where JNI calls are expensive. But the Go camp always advocate for porting everything to Go (because fast compile times).
  - twotwotwo 12y ago
    I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com). CloudFlare has it sitting in the middle of every request for some sites (in the form of Railgun, their delta compression for pages). I'm not exactly a perf ninja and got some Go code packing Wikipedia history dumps at >100 MB/s. Dropbox uses Go for, their words, "our performance-critical backends" (https://blogs.dropbox.com/tech/2014/07/open-sourcing-our-go-libraries/ https://blogs.dropbox.com/tech/2014/07/open-sourcing-our-go-...). So folks manage to heavy lifting with it. I see "I couldn't possibly use X for performance reasons" here a lot, as an almost immediate response about a wide range of different tools, and two things come up in my mind: 1) you can always set the standard arbitrarily high. If you're doing AAA shoot-'em-up games or something, yes, use something else. 2) you're often not as good at tools that're new or different. It's possible you could go further with just a few more tricks about profiling or using free pools or whatever, or a little more info about how your code executes or what the runtime does. FWIW, if you hit a specific wall that's a problem for your app, folks on golang-nuts (or StackOverflow, where I've hung out sometimes) are often happy to try and help. Hope this is more helpful than fussy. Mostly just don't want folks to be discouraged into thinking certain things are impossible when in some cases they're really being done in production out there.
    - coldtea 12y ago
      >I mean, Go is (or was) in the loop whenever YouTube hit its MySQL clusters (vitess) and on on the server providing downloads of Chrome and so forth (dl.google.com). Not as impressive as it sounds. Most of it is cached in memory anyway...
    - tylertreat 12y ago
      Agreed with setting the standard arbitrarily high. It depends on what problem you're trying to solve—just like deciding when to use any other tool. Context matters. Where I work, we've been trying to build some super-fast systems and things like the GC and even calling interface methods matter. We open sourced some data structures that show just how optimized we're trying to get (https://github.com/Workiva/go-datastructures https://github.com/Workiva/go-datastructures), but in hindsight, Go might not have been the right tool for the problem.
- eropple 12y ago
  I don't believe that this is true. You use the exact same patterns and tricks in both for perf-critical code: off-GC primitives and object pools rule the day. What are you considering a "much better" option available in Go? (I considered stack-allocated structs in Go, but honestly that doesn't strike me as a particularly major thing; it may be slightly more terse, but fundamentally the same behavior.)
  - enneff 12y ago
    Values ("off-GC primitives?") are first class-citizens in Go. Stack-allocated structs have much more value than you think.
    - eropple 12y ago
      More value than I think? I'm pretty sure that I know exactly their value. I'll maintain that stack-allocated structs are a nice-to-have on top of the sort of stack-allocated primitives that Java provides; I'd like to have those structs as a convenience when I write Java, which is why I alluded to terseness in my prior post. But there's very little (or should be, I suppose, depending on implementation) performance difference to speak of between a struct and parallel declaration of the same values.
      - enneff 12y ago
        You said: > I considered stack-allocated structs in Go, but honestly that doesn't strike me as a particularly major thing; But for me they're one of the major ways in which I control memory use in Go programs. So, yeah, I think you underestimate them. No condescension implied. Apologies if it came across that way. I think we probably agree more than we disagree.
        eropple 12y ago
        Yeah, I edited it out because after I took a breath I figured you didn't mean it that way. =) Hugs all around. But I really don't think it's a major thing, because there's no meaningful difference, in terms of performance, between "struct { int x; int y; } A" and "int Ax; int Ay;". I'm suspicious of claims that Go, as fundamentally a not that different language to a JVM language, is going to yield significant performance benefits. I'm not saying they aren't a nice convenience (value types are one of the things I love about C# when I'm writing code under MonoGame), I'm saying that I can't think of a reasonable way, barring bugs or short-sighted implementations, that they made for faster code.
justinsb 12y ago
If you're mmaping a file and processing the majority of it, you really are typically doing that in C today if you care about performance. You can get acceptable performance in Java with ByteBuffers, but because of the lack of value types it doesn't feel like Java any more. Go should be able to get much closer to C's performance, while still being closer to idiomatic Go code. And Go can plug in small pieces of C code / assembler for the really performance critical stuff. All this can be done with JNI, but JNI is so un-fun that I can see Go making big inroads here.
- kasey_junk 12y ago
  People use mmapped files all the time in performance critical Java. Typically using the unsafe packages. In fact, if you want to communicate with C/assembler level things, this is the way people who do fast Java do it because JNI is very slow. If you want to use an abstraction around ByteBuffers that feels like value types take a look at the javalution structs. As a counter to your argument, C# has had value types for quite a while and has not achieved Java levels of performance, so those in and of themselves aren't enough. Mostly thats because if you are doing any allocation in fast code you are doing it wrong regardless of language. Even in C object pools and arena allocation are standard for performance critical work. The gap between Java and C (or really C++) right now is almost entirely around control of the memory model, not allocation (that said, I'd love the JVM to have value types and am glad that go started with them). If anything will allow go to achieve better performance than Java its that it will be able to incorporate the lessons learned from Java without the support burden. I do think the constrained nature of go will give it a very good chance at impressive performance.
  - tracker1 12y ago
    I don't think C# vs Java performance can always be directly compared.. C#'s SuppressUnmanagedCodeSecurity attribute for PInvoke interfaces, when you can use it, reduces C#'s call overhead to something like 20% over using a Managed C++ library as a bridge. If you don't need to pass complex numbers (doubles) it isn't bad at all. Just depends on your needs. Then again, depending on your needs, being able to scale out or up is far more important than raw performance characteristics... just depends on your needs.
perturbation 12y ago
Doesn't Go have the potential to be faster than Java since it's compiled to native code (rather than compiled to byte code)?
- ori_b 12y ago
  Java is also compiled to native code, but only after it runs for a bit.
  - kryptiskt 12y ago
    Android does AOT compiling with ART, it became the default in Lollipop.
    - Roboprog 12y ago
      Android does the right thing for a transient GUI client program on a device with little memory. HotSpot JIT does the right thing for a (long running) server program on big, dedicated, hardware.
- vardump 12y ago
  JIT has potential to be faster than compiled native code. Better runtime information. Compiled code needs to cover all potential options and this means more instructions to execute. Say a variable value is set through a command line option to be a certain value. Compiled native code has to assume the value to be dynamic, but a JIT can optimize it away, effectively hardcoding it for that particular invocation. Same applies to more complicated type of software. Some configuration and invocation parameters tend to be effectively static during that particular invocation. JITs can capitalize on this fact. JITs have also better chance to adapt to exact hardware it's running on. Compiled code is forced to make one or a limited number of assumptions of available CPU hardware configuration. In the end, both options are running compiled native code. JIT just does it a bit before running. Of course current reality is the opposite, but the key word here is potential.
  - perturbation 12y ago