12 ms·

Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it - Go seems to have filled the space that Java was too fat fo

by ypcx 7y ago

Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it - Go seems to have filled the space that Java was too fat for, JavaScript too light for, and C/C++ too difficult/fun-less to use for (which is everything). After 20 years of coding, and having just recently discovered LISP and learned Clojure, I believe the perfect language will be a Clojure compiled to native with the ease of Go (e.g. w/ similar import system). One such direction seems to be Carp (https://github.com/carp-lang/Carp https://github.com/carp-lang/Carp), but I haven't tried it yet.

amelius 7y ago
> C++ too difficult/fun-less C++ would be a lot more fun without the obligation to maintain header files all the time. This is one thing that modern languages get right.
- pjmlp 7y ago
  Modules are in C++20 and you can already play around with them.
- treypitt 7y ago
  aren't header files quite useful for yourself and especially other devs as a quick summary? I've found myself wishing my IDE would automatically make a header file equivalent for my python code (VS Code actually does this, but gives a lot of other info that's extraneous in my opinion)
otterley 7y ago
Kubernetes was transpiled to Go from Java, so I wouldn’t use it as an example in support of your point. (Read the source code; it’s pretty obvious — and it’s been confirmed elsewhere.). Some components such as etcd, however, are pure Go.
- threeseed 7y ago
  And is etcd significantly better than Zookeeper ? Not really. So not a great example for Go's supremacy. Also with GraalVM you can equally support low memory, fast startup use cases.
  - otterley 7y ago
    I’m not sure that people are arguing about “supremacy” as much as they are arguing that Go was an attractive choice for the authors vis-a-vis other languages. Getting into language or tool superiority arguments without deeply analyzing the environmental factors and use cases seems like a pointless exercise to me, and ought to be discouraged here IMO.
  - jen20 7y ago
    Etcd may not be, but Consul is in my experience substantially easier() to run in production than ZK, and is also written in Go. I’d say this is nothing to do with implementation language and much more to do with a more understandable consensus model. ( based on having spoken to hundreds of operators of both over the last few years, while (disclaimer) working for HashiCorp and other distributed system vendors, but also having personally run both at serious scale)
  - zzzcpan 7y ago
    Sure, Go didn't turn out to be well suited for distributed systems either. But neither are JVM languages.
    - erik_seaberg 7y ago
      Sandboxed network classloaders were a big win for Hadoop and Spark.
      - ptx 7y ago
        Is the JVM's sandboxing still supported, patched and believed to be secure? I had assumed they gave up on it when they removed applet and webstart support.
zzzcpan 7y ago
> Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime. Things could have been different if it focused on a single threaded runtime or provided control over OS threads, but it did neither. Also neither of those projects were enabled by Go. One is a pretty low quality but strategic push into the cloud and other one is a pivot from an attempt to solve real problems of building a PaaS hosting platform.
- fpoling 7y ago
  Modern Go solved the problem of control over native threads threads. But it is puzzling indeed that it took almost 10 years despite popularity with containers.
  - cpuguy83 7y ago
    Only partially... what's available at least prevents leaking the context unexpectedly to other goroutines (due to thread re-use), however anything that spins up a new goroutine from the locked goroutine will end up in a new thread in a totally different context. Even stdlib spins up new goroutines.
- otterley 7y ago
  > Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime. Can you please elaborate on this? My experience with Go is that it is able to fork/exec programs and manage cgroups via sysfs on Linux at least as well as any other programming language can.
  - cpuguy83 7y ago
    You pretty much need to do everything from a separate process, which is the main issue. Certain things like setns(mountns) will straight up fail in a multithreaded program, so there are some nasty hacks in runc to handle such things in C prior to the go runtime firing up.
coldtea 7y ago
>Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it Let's name it. I wouldn't call either of these a "software renaissance". Docker is a wrapper on top of Linux kernel containers, and Kubernetes is a 10000 pound gorilla on top (and port of a C++ app, some say even through a Java rewrite/transpile to Go). Go was just at the peak of being fashionable at the time (and had some basic features not semantics/syntax related like easy static compilation and cross platform-ness) so got adopted by these projects, the same way many new projects now use Rust.
- geofft 7y ago
  Docker is a wrapper on top of kernel containers in the same way that Emacs is a wrapper on top of the kernel filesystem API. I've written a 150-line container manager in shell before (because we had a use case where both Docker was not the right technical fit out of the box and the organization had barely any operational experience with even normal use of Docker), and it's great that containers are built into the kernel and you can just use unshare and a bunch of bind mounts if you'd like, but Docker is its own quite substantial product and brings its own approach to building systems (Dockerfiles, container networking, etc.) that aren't implied by or required by the kernel API.
- twic 7y ago
  Docker is a bad design. It is monstrously overcomplicated for what it does. In historico-aesthetic analogies, it's baroque, not renaissance.
  - cpuguy83 7y ago
    This sounds like dogma. While I do agree there are some pieces which are over complicated or just plain bad code, on the whole it's more that the API's aren't quite right specifically for people who want to integrate with it rather than use it. containerd exists pretty much because of this observation.
  - lkrubner 7y ago
    And what does it actually give that is new? What does Docker give that you can't get with a normal VM, with Terraform to spin up however many instances that you need? Put an app on an AMI, save it, if you need one instance, or a 1,000 instances, spin them up with Terraform. You get to stick with normal operating systems, such as Linux and Windows. You don't have to learn a bunch of new technologies. http://www.smashcompany.com/technology/docker-is-a-dangerous-gamble-which-we-will-regret http://www.smashcompany.com/technology/docker-is-a-dangerous...
pjmlp 7y ago
Kubernetes was written in Java, only got re-written in Go as another team took over and they were into Go bandwagon.
snowAbstraction 7y ago
I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects. I know there are some but I couldn't find many that were both active and to my taste. I was looking to learn and hopefully contribute. I was mostly looking for something mature for either mathematical optimization (stuff like linear, quadratic or integer programming) and (non-DL) machine learning. For C++, Java, Scala, Python and Julia (less sure about this), there seems to be much more. You can compare Go's gonum to Python's numpy. For Java's Weka, Scala's MLlib, Python's scikit-learn, C++'s Dlib to what Go is offering. Maybe Go leans more on its C-interop for these sorts of things which is a bit like numpy. Am I missing something? Will this change?
- ChrisRackauckas 7y ago
  What are you not sure of with Julia here? There are places in the ecosystem to not be sure of, but this isn't one. Julia has probably the most advanced mathematical optimization right now with JuMP (http://www.juliaopt.org/JuMP.jl/v0.19.2/ http://www.juliaopt.org/JuMP.jl/v0.19.2/) and some of the most advanced post-DL machine learning with the full language differentiable programming tools (Zygote, Tracker, ForwardDiff) which have showcased applications like quantum machine learning and neural stochastic differential equations (https://arxiv.org/abs/1907.07587 https://arxiv.org/abs/1907.07587). Some of it is still in flux, but in terms of ecosystem there's a lot of stuff there that you won't find in other languages.
  - snowAbstraction 7y ago
    I meant personally I am less sure about Julia, that my knowledge is less sure. I've neither used it myself nor spent a few hours browser project source code unlike most of other stuff I mentioned. Since I am not finding what I am looking for in Go then maybe I should ought to try out Julia. Thanks for the JuMP reference. That looks cool.
- Rotareti 7y ago
  > I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects. I think this has to do with golangs poor FFI performance.
delusional 7y ago
> Java was too fat for I think it should always be mentioned when people call java fat, that the core language is usually fast enough. What makes java "fat" is the mentality of "Frameworks". For me, Go seems like java, but without the fucking frameworks.
- saganus 7y ago
  I have been wondering for some time now, when exactly is a language "fat" or "light"? The idea seems to be there and "obvious" in a way, but haven't been able to pinpoint exactly what it is. Is it a feeling or an actual metric? e.g. When I work with Java in IntelliJ, things don't feel significantly slower than developing Node in WebStorm. Is it perhaps the way the code looks, in terms of verbosity? I do agree that frameworks contribute to this feeling of Java, but again, is it because a Spring Boot webserver boots slower than a Node one? Or maybe, say a Java cli program vs a C++ one. Is it the speed at which things run? Could it be that, because we know Java uses a VM then it "should" be slower than compiled code and thus it's fatter? What is that "thing" that makes us determine that a language is fat or light?
  - ypcx 7y ago
    The one area where Java feels fat is the memory usage overhead due to 64-bit pointers being used a lot. So if you end up holding and processing a lot of data in RAM, the process allocation and the garbage collector metrics won't look awesome. That said, recently Openj9 improves on this literally by miles. I agree with all the other points - Java and its tooling are very powerful and by that nature they are enabling us to put that much more load on it, so yes, that's where it will be predominantly visible. Edit: that said, Java was created when we didn't have containers. (Just as Clojure was created before Java had lambdas and aggregate streams). So it may be the time to come back closer to the metal again, but as you point out, we need to maintain a clear mind while contemplating that.
    - pjmlp 7y ago
      Yes we did, HP-UX vault and Tru64 sandboxes.
mrbonner 7y ago
Considering it took you that long to be a Lisp convert, I like to know more on how you came to the see the light. I’ve been doing coding for over 15 years, too. And, every time I tried to pick up a Lisp dialect (Clojure was the last) I couldn’t help but wonder what with all the buzz about this. I really wish that I could see the light someday, though.
- lkrubner 7y ago
  Depends where you are coming from. What languages do you work in now? If the language you are working in now has excellent support for concurrency, then that part of Clojure is not going to impress you. Or if you currently work with a language that makes it easy to write DSLs, then that part of Clojure won't impress you.
  - mrbonner 7y ago
    Mostly in Java and now picking up Rust in my spare time.
- ypcx 7y ago
  I wish I had a good answer to this. I suppose I've never looked at anything else than what I was working with (C, C++, Java, JavaScript and much later CoffeeScript) out of "religious" reasons. I guess being really good at what you do can do that to you. I hold CoffeeScript in a very high regard. It has freed me from two things - the spaghetti verbosity of JavaScript, and the abrupt syntax changes of ECMA spec introduced in the recent versions, ones I don't particularly see as a positive development of the language, _especially_ compared to the timeless cleanliness of CoffeeScript. So and I think that CoffeeScript may have thawed me a little towards LISP-like syntax and behavior. I don't know. I think at some point, you realize that you have written (and managed) enough of C/Java-like verbose code and that it's time to try something completely different. So and then it hits you - programming should be data-centric, not language-construct-centric. And that a too big codebase must be a result of ill-specified project scope, more often than not.

by ypcx 7y ago

amelius 7y ago

> C++ too difficult/fun-less C++ would be a lot more fun without the obligation to maintain header files all the time. This is one thing that modern languages get right.

pjmlp 7y ago
Modules are in C++20 and you can already play around with them.
treypitt 7y ago
aren't header files quite useful for yourself and especially other devs as a quick summary? I've found myself wishing my IDE would automatically make a header file equivalent for my python code (VS Code actually does this, but gives a lot of other info that's extraneous in my opinion)

otterley 7y ago

Kubernetes was transpiled to Go from Java, so I wouldn’t use it as an example in support of your point. (Read the source code; it’s pretty obvious — and it’s been confirmed elsewhere.). Some components such as etcd, however, are pure Go.

threeseed 7y ago
And is etcd significantly better than Zookeeper ? Not really. So not a great example for Go's supremacy. Also with GraalVM you can equally support low memory, fast startup use cases.
- otterley 7y ago
  I’m not sure that people are arguing about “supremacy” as much as they are arguing that Go was an attractive choice for the authors vis-a-vis other languages. Getting into language or tool superiority arguments without deeply analyzing the environmental factors and use cases seems like a pointless exercise to me, and ought to be discouraged here IMO.
- jen20 7y ago
  Etcd may not be, but Consul is in my experience substantially easier() to run in production than ZK, and is also written in Go. I’d say this is nothing to do with implementation language and much more to do with a more understandable consensus model. ( based on having spoken to hundreds of operators of both over the last few years, while (disclaimer) working for HashiCorp and other distributed system vendors, but also having personally run both at serious scale)
- zzzcpan 7y ago
  Sure, Go didn't turn out to be well suited for distributed systems either. But neither are JVM languages.
  - erik_seaberg 7y ago
    Sandboxed network classloaders were a big win for Hadoop and Spark.
    - ptx 7y ago
      Is the JVM's sandboxing still supported, patched and believed to be secure? I had assumed they gave up on it when they removed applet and webstart support.

zzzcpan 7y ago

> Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime. Things could have been different if it focused on a single threaded runtime or provided control over OS threads, but it did neither. Also neither of those projects were enabled by Go. One is a pretty low quality but strategic push into the cloud and other one is a pivot from an attempt to solve real problems of building a PaaS hosting platform.

fpoling 7y ago
Modern Go solved the problem of control over native threads threads. But it is puzzling indeed that it took almost 10 years despite popularity with containers.
- cpuguy83 7y ago
  Only partially... what's available at least prevents leaking the context unexpectedly to other goroutines (due to thread re-use), however anything that spins up a new goroutine from the locked goroutine will end up in a new thread in a totally different context. Even stdlib spins up new goroutines.
otterley 7y ago
> Go turned out to be a pretty bad choice for container tech due to its awful multithreaded runtime. Can you please elaborate on this? My experience with Go is that it is able to fork/exec programs and manage cgroups via sysfs on Linux at least as well as any other programming language can.
- cpuguy83 7y ago
  You pretty much need to do everything from a separate process, which is the main issue. Certain things like setns(mountns) will straight up fail in a multithreaded program, so there are some nasty hacks in runc to handle such things in C prior to the go runtime firing up.

coldtea 7y ago

>Well, Go seems to have enabled a new wave of software renaissance - Kubernetes, Docker, you name it Let's name it. I wouldn't call either of these a "software renaissance". Docker is a wrapper on top of Linux kernel containers, and Kubernetes is a 10000 pound gorilla on top (and port of a C++ app, some say even through a Java rewrite/transpile to Go). Go was just at the peak of being fashionable at the time (and had some basic features not semantics/syntax related like easy static compilation and cross platform-ness) so got adopted by these projects, the same way many new projects now use Rust.

geofft 7y ago
Docker is a wrapper on top of kernel containers in the same way that Emacs is a wrapper on top of the kernel filesystem API. I've written a 150-line container manager in shell before (because we had a use case where both Docker was not the right technical fit out of the box and the organization had barely any operational experience with even normal use of Docker), and it's great that containers are built into the kernel and you can just use unshare and a bunch of bind mounts if you'd like, but Docker is its own quite substantial product and brings its own approach to building systems (Dockerfiles, container networking, etc.) that aren't implied by or required by the kernel API.
twic 7y ago
Docker is a bad design. It is monstrously overcomplicated for what it does. In historico-aesthetic analogies, it's baroque, not renaissance.
- cpuguy83 7y ago
  This sounds like dogma. While I do agree there are some pieces which are over complicated or just plain bad code, on the whole it's more that the API's aren't quite right specifically for people who want to integrate with it rather than use it. containerd exists pretty much because of this observation.
- lkrubner 7y ago
  And what does it actually give that is new? What does Docker give that you can't get with a normal VM, with Terraform to spin up however many instances that you need? Put an app on an AMI, save it, if you need one instance, or a 1,000 instances, spin them up with Terraform. You get to stick with normal operating systems, such as Linux and Windows. You don't have to learn a bunch of new technologies. http://www.smashcompany.com/technology/docker-is-a-dangerous-gamble-which-we-will-regret http://www.smashcompany.com/technology/docker-is-a-dangerous...

pjmlp 7y ago

Kubernetes was written in Java, only got re-written in Go as another team took over and they were into Go bandwagon.

snowAbstraction 7y ago

I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects. I know there are some but I couldn't find many that were both active and to my taste. I was looking to learn and hopefully contribute. I was mostly looking for something mature for either mathematical optimization (stuff like linear, quadratic or integer programming) and (non-DL) machine learning. For C++, Java, Scala, Python and Julia (less sure about this), there seems to be much more. You can compare Go's gonum to Python's numpy. For Java's Weka, Scala's MLlib, Python's scikit-learn, C++'s Dlib to what Go is offering. Maybe Go leans more on its C-interop for these sorts of things which is a bit like numpy. Am I missing something? Will this change?

ChrisRackauckas 7y ago
What are you not sure of with Julia here? There are places in the ecosystem to not be sure of, but this isn't one. Julia has probably the most advanced mathematical optimization right now with JuMP (http://www.juliaopt.org/JuMP.jl/v0.19.2/ http://www.juliaopt.org/JuMP.jl/v0.19.2/) and some of the most advanced post-DL machine learning with the full language differentiable programming tools (Zygote, Tracker, ForwardDiff) which have showcased applications like quantum machine learning and neural stochastic differential equations (https://arxiv.org/abs/1907.07587 https://arxiv.org/abs/1907.07587). Some of it is still in flux, but in terms of ecosystem there's a lot of stuff there that you won't find in other languages.
- snowAbstraction 7y ago
  I meant personally I am less sure about Julia, that my knowledge is less sure. I've neither used it myself nor spent a few hours browser project source code unlike most of other stuff I mentioned. Since I am not finding what I am looking for in Go then maybe I should ought to try out Julia. Thanks for the JuMP reference. That looks cool.
Rotareti 7y ago
> I've been learning a bit of Go and I like what I've understood so far but one thing that seems somewhat lacking in the Go Eco system is mathy projects. I think this has to do with golangs poor FFI performance.

delusional 7y ago

> Java was too fat for I think it should always be mentioned when people call java fat, that the core language is usually fast enough. What makes java "fat" is the mentality of "Frameworks". For me, Go seems like java, but without the fucking frameworks.

saganus 7y ago
I have been wondering for some time now, when exactly is a language "fat" or "light"? The idea seems to be there and "obvious" in a way, but haven't been able to pinpoint exactly what it is. Is it a feeling or an actual metric? e.g. When I work with Java in IntelliJ, things don't feel significantly slower than developing Node in WebStorm. Is it perhaps the way the code looks, in terms of verbosity? I do agree that frameworks contribute to this feeling of Java, but again, is it because a Spring Boot webserver boots slower than a Node one? Or maybe, say a Java cli program vs a C++ one. Is it the speed at which things run? Could it be that, because we know Java uses a VM then it "should" be slower than compiled code and thus it's fatter? What is that "thing" that makes us determine that a language is fat or light?
- ypcx 7y ago
  The one area where Java feels fat is the memory usage overhead due to 64-bit pointers being used a lot. So if you end up holding and processing a lot of data in RAM, the process allocation and the garbage collector metrics won't look awesome. That said, recently Openj9 improves on this literally by miles. I agree with all the other points - Java and its tooling are very powerful and by that nature they are enabling us to put that much more load on it, so yes, that's where it will be predominantly visible. Edit: that said, Java was created when we didn't have containers. (Just as Clojure was created before Java had lambdas and aggregate streams). So it may be the time to come back closer to the metal again, but as you point out, we need to maintain a clear mind while contemplating that.
  - pjmlp 7y ago
    Yes we did, HP-UX vault and Tru64 sandboxes.

mrbonner 7y ago

Considering it took you that long to be a Lisp convert, I like to know more on how you came to the see the light. I’ve been doing coding for over 15 years, too. And, every time I tried to pick up a Lisp dialect (Clojure was the last) I couldn’t help but wonder what with all the buzz about this. I really wish that I could see the light someday, though.

lkrubner 7y ago
Depends where you are coming from. What languages do you work in now? If the language you are working in now has excellent support for concurrency, then that part of Clojure is not going to impress you. Or if you currently work with a language that makes it easy to write DSLs, then that part of Clojure won't impress you.
- mrbonner 7y ago
  Mostly in Java and now picking up Rust in my spare time.
ypcx 7y ago
I wish I had a good answer to this. I suppose I've never looked at anything else than what I was working with (C, C++, Java, JavaScript and much later CoffeeScript) out of "religious" reasons. I guess being really good at what you do can do that to you. I hold CoffeeScript in a very high regard. It has freed me from two things - the spaghetti verbosity of JavaScript, and the abrupt syntax changes of ECMA spec introduced in the recent versions, ones I don't particularly see as a positive development of the language, _especially_ compared to the timeless cleanliness of CoffeeScript. So and I think that CoffeeScript may have thawed me a little towards LISP-like syntax and behavior. I don't know. I think at some point, you realize that you have written (and managed) enough of C/Java-like verbose code and that it's time to try something completely different. So and then it hits you - programming should be data-centric, not language-construct-centric. And that a too big codebase must be a result of ill-specified project scope, more often than not.