6 ms·
Go 1.21 will (probably) download newer toolchains on demand by default
- sigg3 3y agoIt is customizable: > Not everyone is going to be happy with the idea of the 'go' command downloading and running binaries from the Internet. If this is the case for you, you need to set GOTOOLCHAIN to either 'path', if you're happy to search $PATH for the right toolchain, or 'local', if you want to turn off this behavior entirely and have 'go' just fail.
- timmytokyo 3y agoIt's customizable, but it downloads by default. Anyone who cares about controlling what they download from the Internet would prefer opt-in, not opt-out.
- bushbaba 3y agoTend to agree here. Feels like an overstep
- codazoda 3y agoI’m a veteran dev but I’m new to Go… The following quotes seem to indicate that “auto” is not the default. At least, not if you haven’t run some command to set it at a default (and I’m not seeing one). > If the $GOROOT/go.env file is missing or does not set a default, the go command assumes GOTOOLCHAIN=local. > When GOTOOLCHAIN is set to local, the go command always runs the bundled Go toolchain. In the intro, however, it does say: > The default GOTOOLCHAIN setting is auto, which enables the toolchain switching described earlier. I guess I’m also confused. :/ Note that go.work is only created by the `go work init` command when you intend to use multiple modules in one directory. I’m guessing that doing this is relatively unusual. This could be how a “default” gets set for newer versions (unconfirmed conjecture on my part). I don’t see any command that creates a `go.env` and therefore might set GOTOOLCHAIN in that file. The only other thing I can think of is that `go mod init` could set this to “auto” in the `go.mod` of a new project. But, go.env is probably missing in this case. Ref: https://go.dev/doc/toolchain https://go.dev/doc/toolchain
- silisili 3y agoSeems crazy to me. I want to see an error if the version in my PATH is too old, so I can address. I don't want my toolchain to do -anything- implicitly, including searching PATH. If my PATH is wrong, I'll fix it. And downloading is even weirder. What happens if the download fails? Am I presented a random network error, or a Go too old error? Who decides which is even more correct?
- maccard 3y agoAnd people who don't care won't change the default settings. The best case scenario is opt-out. That way people who care have a choice, and people who don't care are updated. People will complain either way.
- ilyt 3y agoThat's terrible default. Reminds me on GOPRIVATE bullshit where since some go version private repositories outright didn't work without setting env variables, because go team decided to not add fallback if something is not available via their shitty proxy
- sidewndr46 3y agoThis is going to be hilarious when every docker container starts updating itself on every single invocation!
- ttymck 3y agoI really don't understand. Maybe I'm stupid. But I don't see the issue. I work with go daily, but I'm pretty sure our binary is pegged to a specific go version, using go.mod I don't see in what scenario I would add a dependency and it would unexpectedly upgrade my go toolchain? Isn't my toolchain version pegged by a go.mod file? This article may raise a valid concern, but it doesn't propose a clear alternative or solution. I'm not a go truther but there seems to be lot of pessimism around it (error handling, generics, and now toolchain) and not enough optimism.
- philosopher1234 3y agoIIRC the go version in go.mod is a minimum go toolchain version. In the future if you pull in a dep with a higher minimum toolchain version, it would be unbuildable without upgrading your toolchain. I expect there is an env variable to disable this, though.
- yobert 3y agoI think the part they take issue with that might surprise you is if you decide to start using a dependency, and that dependency declares in it's go.mod file (not yours) that it needs a newer version than yours, and that triggers a download and toolchain swap. I think it's... Iffy. I'd prefer it to fail and for it to be up to me to upgrade, personally.
- ttymck 3y agoOK, if I understand correctly: I might have a git repo, and nowhere in it does anything say "go1.2x" My go.mod might say go1.26 But one of the dependencies, in it's source (not anywhere to be found in my source), says go1. 29 So when you build my source, it will build with go1.29 If that's the case, then yes, I'm appalled, AND this article did a terrible job of informing me of the situation
- arccy 3y agoYour go.mod would be considered untidy > A module’s go line must declare a version greater than or equal to the go version declared by each of the modules listed in require statements.
- KingOfCoders 3y agoAt last Go got rustup.
- kramerger 3y agoYou say it as it was a good thing. Not having full control over the compiler is generally bad if your product needs to be supported for a long time or has to undergo some type of certification. Rustup has slowed down rust adaptation in certain industries. Yes, it is technically possible to download a fixed version and stick to it but it will be challenging when the rest of the ecosystem is on nightly...
- weinzierl 3y agoI don't understand that. If anything rustup has made it easier because I can pull any old version on a whim, or not?
- JoshTriplett 3y ago> Not having full control over the compiler is generally bad if your product needs to be supported for a long time or has to undergo some type of certification. rustup, and this mechanism in Go, don't take away your control over the compiler. You can always choose to build your software with whichever version you like. > Rustup has slowed down rust adaptation in certain industries. Yes, it is technically possible to download a fixed version and stick to it but it will be challenging when the rest of the ecosystem is on nightly... The vast majority of the Rust ecosystem is on stable, not nightly. Crates that require nightly tend to get limited adoption.
- yjftsjthsd-h 3y ago> rustup, and this mechanism in Go, don't take away your control over the compiler. You can always choose to build your software with whichever version you like. It certainly overrules your choice by default. Before, you could choose whatever version you wanted and it would work; now, you have to run the version you want, and also set an option to tell it to not just ignore the version you installed.
- plorkyeran 3y agoI don’t see downloading a new copy of go which matches a dependency’s version requirement as being fundamentally different from downloading that dependency’s other dependencies. It’s surprising due to being a new thing, but in five years I suspect people will consider it completely obvious and normal.
- 2h 3y ago> I don’t see downloading a new copy of go which matches a dependency’s version requirement as being fundamentally different Well, you'd be wrong then. If you went into a car dealership and asked for a new rearview mirror, and they gave you (and charged you for) a new car, you'd be rightly surprised. Or better yet, if you ask apt get to update a package, you damn sure don't expect it to update itself. Nothing else works like this, and shouldn't. It's just not what most people need or expect, or even want.
- rcme 3y ago> Or better yet, if you ask apt get to update a package, you damn sure don't expect it to update itself. This might be true for apt, because that’s how it always worked, but is this true of modern package managers? Like if I do `brew install X` I expect the latest version of X, which might require updating brew. I definitely don’t want the latest version of X my local brew install happens to know about. As an example, doesn’t pretty much every Dockerfile installing dependencies from apt call apt-get update -y?
- ben-schaaf 3y ago> Or better yet, if you ask apt get to update a package, you damn sure don't expect it to update itself. If a package requires a newer version of apt and said version is available then I would absolutely expect apt to do so. apt is after all just another package.
- wereHamster 3y agoThat's kindof a bad analogy. `go build` is a very high-level command. I'd compare it to going to a car dealership and asking them to 'make the car usable'. You think only a small part is broken but the dealership determines that it's incompatible with the current law and you need to get a new one. If you want to build go project in a hermetically sealed environment (where it doesn't fetch anything unknown or unexpected from the internet), use for example Nix. When building software with Nix, the sandbox doesn't have access to the internet and all project dependencies must be specified via the Nix language.
- gjvc 3y agoIf this happens, this will be a case of "jumping the shark". https://en.wikipedia.org/wiki/Jumping_the_shark https://en.wikipedia.org/wiki/Jumping_the_shark
- gjvc 3y agodownvotes are likely because people are resentful of the npm'isation of the platform
- 015a 3y agoI'm... really not sure I agree with this, from a philosophical point of view. It feels like this is making "eh, we'll just upgrade our Go version next quarter" too easy; ultimately some responsibility toward updating your application's Go version to work with what new dependencies require should fall on Us, the application developers. Sure, we're bad at it. Everyone's lived through running years-old versions of some toolchain. But I think this just makes the problem worse, not better. Its compounded by the problem that, when you're setting up a new library, the `go` directive in the mod file defaults to your current toolchain; most likely a very current one. It would take a not-insignificant effort on the library author's part to change that to assert the true-minimum version of Go required, based on libraries and language features and such. That's an effort most devs won't take on. I'd also guess that many developers, up-to this point if not indefinitely because education is hard, interpreted that `go` directive to mean more-of "the version of go this was built with"; not necessarily "the version of go minimally required". There are really major libraries (kubernetes/client-go [1]) which assert a minimum go version of 1.20; the latest version (see, for comparison, the aws-sdk, which specifies a more reasonable go1.11 [2]). I haven't, you know, fully audited these libraries, but 1.20 wasn't exactly a major release with huge language and library changes; do they really need 1.20? If devs haven't traditionally operated in this world where keeping this value super-current results in actually significant downstream costs in network bandwidth (go1.20 is 100mb!) and CI runtime, do we have confidence that the community will adapt? There's millions of Go packages out there. Or, will a future version of Go patch a security update, not backport it more than one version or so, and libraries have to specify the newest `go` directive version, because manifest security scanning and policy and whatever? Like, yeah, I get the rosy worldview of "your minimum version encodes required language and library features", but its not obvious to me that this is how this field is, or even will be, used. Just a LOT of tertiary costs to this change which I hope the team has thought through. [1] https://github.com/kubernetes/client-go/blob/master/go.mod#L5 https://github.com/kubernetes/client-go/blob/master/go.mod#L... [2] https://github.com/aws/aws-sdk-go/blob/main/go.mod https://github.com/aws/aws-sdk-go/blob/main/go.mod
- arccy 3y agoGo already only supports/backports to the 2 latest versions.
- jrockway 3y agoGood idea! Go has always impressed me on how every single new release works perfectly with zero effort on my part. There is no reason I can think of that someone would want to be on an older version of Go. I understand the objections here. I upgrade software a lot. Upgrade Emacs, your .emacs fails to load. Upgrade Linux, your system fails to boot. That's because they make backwards incompatible changes and release if the unit tests pass. I don't know what Go is doing, but this simply never happens. Every 6 months, you get a performance improvement and new features for free. For free! It's truly amazing. (While I'm here, I'll also point out that PostgreSQL is similar here. Have you ever upgraded and had your app broke? Absolutely not. Every version is a free performance and feature win.) At the risk of being downvoted, I have to say that if you don't believe this, you've been Stockholm Syndrome'd into your programming language of choice. I work primarily in Go, but occasionally have to do frontend stuff. The pattern that I run into is that if I use the version of Node in Debian, the code doesn't compile. If I go to nodejs.org and download the latest version, it doesn't compile. Only if I install 20000 lines of bash that inspect some files and switches 300 environment variables to put the right version of node.js in my path can I even begin to ... run a script that concatenates 300,000 lines of code and translate it to a version of Javascript that Chrome and Firefox both have? Guys, it's crazy. I love a good user interface, but if I was trapped on a desert island with node.js I would instantly kill myself. No questions asked. Go isn't like that. If your users can handle a CLI, you are one "tar xzvf" way from sanity. It's great. I digressed, but this is a great choice. The Go team always seems to know how to improve the lives of end users. I appreciate that.
- c7DJTLrn 3y agoI'm all in on Go and enjoy how the toolchain simplifies things so you can just think about writing code. But I am not a fan of this. I don't want my compiler to execute remote code uninstructed.
- srcreigh 3y agoOne workplace I had used GCP. There was a Postgres upgrade and one of our services had a major performance degradation, some people worked for days to mitigate the problem. Sadly I don’t have the version number or any details as I wasn’t working on the issue. It was sometime in mid 2022.
- leighmcculloch 3y agoGo downloading a toolchain is not any different to dustup downloading the version of rust that is in a toolchain toml file. At least with Go it will presumably only download officially released versions, and not gotip/nightly.
- isanjay 3y agorustup ?
- veber-alex 3y agoIt is different. The rustup toolchain toml file is under your control, dependencies have no affect on it. This Go feature is like the rust-version key in Cargo.toml where if a dependency specifies in it's own Cargo.toml file that it needs a rust version higher than you what you have you will get an error and your code won't compile (there is no option to auto upgrade the toolchain in rust).
- GeneralMayhem 3y agoThis is incorrect afaict. The Go docs specify that the toolchain directive is only considered in the "main" module. If your dependencies set it, it has no effect (unless they're binary tool dependencies that you go install).
- veber-alex 3y agoAccording to the docs[0] you can't declare a go version in the main module or workspace that is lower then the go version of one of your dependencies. A module’s go line must declare a version greater than or equal to the go version declared by each of the modules listed in require statements. A workspace’s go line must declare a version greater than or equal to the go version declared by each of the modules listed in use statements. For example, if module M requires a dependency D with a go.mod that declares go 1.22.0, then M’s go.mod cannot say go 1.21.3. so the "go" directive is like "rust-version" in Cargo.toml and the "toolchain" directive is like the rust version in a rust-toolchain file. [0]: https://tip.golang.org/doc/toolchain https://tip.golang.org/doc/toolchain
- JetSetIlly 3y agoIt sounds to me like this is downloading the toolchain as though it were any other Go package and compiling it. I'm not sure it's downloading binaries, as the article suggests. From the Go Toolchains documentation: "When using GOTOOLCHAIN=auto or GOTOOLCHAIN=<name>+auto, the Go command downloads newer toolchains as needed. These toolchains are packaged as special modules with module path golang.org/toolchain and version v0.0.1-goVERSION.GOOS-GOARCH. Toolchains are downloaded like any other module, meaning that toolchain downloads can be proxied by setting GOPROXY and have their checksums checked by the Go checksum database." I don't know for sure though.
- agwa 3y agoIt would indeed be downloading binaries. See the design doc, which talks about the need to set the execute bit on files in the module cache: https://go.googlesource.com/proposal/+/master/design/57001-gotoolchain.md#downloading-distributions https://go.googlesource.com/proposal/+/master/design/57001-g...
- JetSetIlly 3y agoThanks. I'm now not sure how I feel about this.
- agwa 3y agoCurious what your concerns are? As of Go 1.21 the toolchain will be fully reproducible [https://github.com/golang/go/issues/57120 https://github.com/golang/go/issues/57120] and since the binaries will be distributed through the module system it'll be possible to verify that everyone is getting the same binaries. So you can be pretty confident you'll end up with the same bytes as if you downloaded the source and compiled yourself.
- JetSetIlly 3y agoIn principle, I like the idea of it. Whether or not it's a binary or source to be compiled locally is irrelevant, I suppose. My concern is that it's not clear to me how this will look on my computer. Currently, I download the source tarball and build with the latest Go toolchain I have previously installed. I then change the path to point to the newly built toolchain. So, in the future, what is it I'm downloading? How does the toolchain get updated exactly? Where does the new executable reside? What happens to the toolchain that I explicitly built and installed? How does Go1.21 change my path? As I say, I like the idea in principle but there are details that I think are important, that don't seem to have been explained anywhere.
- SuperSandro2000 3y agoAs a distro maintainer: we are probably needing to disable that by default.
- amarshall 3y agoYes, I agree, especially considering the distro :)
- mananaysiempre 3y agoThe Arch Linux maintainers, on the other hand, already tried and gave up on isolating Go and Rust builds from the network, from what I’ve read. (That was for package dependencies and not the toolchain itself.)
- ilyt 3y agoDefault smells of same ignorance like proxy and GOPRIVATE introduction, where after introduction they forced using proxy so no private repositories for dependencies worked any more without fucking with env
- moondev 3y agoIs it possible to fully embed the "go sdk" into your own golang program? The motivation is shipping my single binary program which would include go itself. Then the program could proxy the "go" command to build and run code without needing "golang" installed or downloaded if airgapped.
- Philip-J-Fry 3y agoYou can embed data into the application. So in theory you could embed a whole Go root directory, extract it to disk and then just run the executables.
- kn100 3y agoSeeing a lot of discourse here talking about how Go releases are so stable and don't break shit. As someone who also loves Go, I mostly agree with these takes, but let's not forget how much of a mess dependency management used to be in the bad old days. I remember Glide, Dep, Godep, etc. I think given the Go teams attitude towards not breaking backwards compatibility we will probably see less breaking upgrades like we see in frontend-land for example, but things were not always this nice and stable.
- LinuxBender 3y agoGo 1.21 will (probably) download newer toolchains on demand by default Will it fail gracefully and quickly when firewall rules do not permit Go to make outbound connections? If I give it a tcp-reset will it stop trying? Can I set an environment variable that says to never try in the first place? Why would I want this? Simple. I want to know that a set of applications and libraries at specific versions have gone through Dev -> Security Audit / Code Review -> QA -> Load Testing -> Staging -> Production. Bug fixed? No problem, rinse and repeat the QA process.