10 ms·
Go 1.18 will embed source version information into binaries
- de_keyboard 5y agoCan you ever really trust this info? I think that the deployment manifest is the only thing you can really trust.
- Dobbs 5y agoThis is really useful. Almost every project I’ve worked on has had to have a build script in part to inject this info so it being a first class feature makes life just that much easier.
- pvtmert 5y agoAt the same time, I was thinking how much of internal structure can be exposed. :)
- LysPJ 5y agoIf there are projects where you don't want to reveal this info in the binary, you can turn the feature off with: -buildinfo=false
- pvtmert 5y agoYes, but it is turned on by default. And 99% of people will go and copy-paste existing tutorial code to their projects. I especially found out this is really useful for extracting information form non-stripped firmware binaries. They lay around on some S3 buckets, where cheap IoT manufacturers think it's safe to expose them online for updating purposes...
- deleted 5y ago[deleted]
- gwbas1c 5y agoNot much: > the currently checked-out revision and a flag indicating whether edited or untracked files are present Basically, it's including a git commit ID, which is a hash, and flags that indicate if there are local changes. It's pretty common to include build hashes in shipping code, logs, ect.
- LysPJ 5y agoAgreed. Anything that simplifies the build process and leads to less custom tooling is very welcome.
- mytailorisrich 5y agoThe build version, embedded in the build, should match with a label or tag in source control so that this information should allow you to immediately get everything you need out of source control. In general you don't want more than that embedded in builds.
- usr1106 5y agoThey call it "uncommitted". What does that mean? Myself I have used "git describe --dirty" to store version information (not in Go binaries, but in general). "Dirty" will indicate changes to files under version control. But it will miss any new file that should be under version control but isn't.
- buro9 5y agoSomething I have been wondering is whether the Go bootstrap, wherein Go is itself built from Go 1.4, also needs to be updated so that Go bootstraps from Go 1.17 or 1.18. The reason for this is the change in syntax for build conditionals. Go 1.17 being compatible with the old and new, but Go 1.18 will only be compatible with the new. By which point Go 1.18 will only use the new and Go 1.4 would no longer be able to build later versions of Go.
- dtech 5y agoiirc correctly their bootstrap always has been to bootstrap a Go 1.n version, that can build Go 1.n+1 etc.
- Vogtinator 5y agoNot always, there are gaps allowed.
- vvern 5y agoVersions after go1.4 up to 1.17 need to be built with 1.4. From 1.18 on, they need to be built with 1.16. See https://github.com/golang/go/issues/44505 https://github.com/golang/go/issues/44505.
- vvern 5y agoupdate: they missed implementing this and now go1.17 will be required to bootstrap go1.19+. https://github.com/golang/go/issues/44505#issuecomment-942593422 https://github.com/golang/go/issues/44505#issuecomment-94259...
- hbbio 5y agoGreat! This is my current build script: BUILD_TIME=$(date +"%Y%m%d.%H%M%S") CommitHash=$(git describe | cut -d- -f3) GoVersion=$(go version | cut -c 14- | cut -d' ' -f1) GitTag=$(git describe | cut -d- -f1) FLAG="-X $TRG_PKG.BuildTime=$BUILD_TIME" FLAG="$FLAG -X $TRG_PKG.CommitHash=$CommitHash" FLAG="$FLAG -X $TRG_PKG.GoVersion=$GoVersion" FLAG="$FLAG -X $TRG_PKG.GitTag=$GitTag" go build -v -ldflags "$FLAG"
- rollcat 5y agoPlease consider omitting build time from your artifacts. It makes build reproducibility a nightmare; people are coming up with frameworks to lie to build systems about the current time. https://reproducible-builds.org https://reproducible-builds.org
- ainar-g 5y agoAlternatively, just make it configurable: buildtime="${BUILD_TIME:-$( date -u +%FT%TZ%z )}" # Use $buildtime in your go build command. This will only set the variable buildtime if BUILD_TIME isn't set already. Making reproducing a build easy: just define BUILD_TIME to be equal to the one of the binary you want to reproduce.
- Foxboron 5y agoPlease read the link. Reproducible Builds has standardized "SOURCE_DATE_EPOCH" for exactly this usecase. Introducing other variable names i just going to cause more problems. https://reproducible-builds.org/docs/source-date-epoch/ https://reproducible-builds.org/docs/source-date-epoch/
- IshKebab 5y agoThis sounds useful, but this is a bad attitude: > People who distribute binaries widely and build them doing funny things may now have more of those funny things be somewhat visible, though. Well, until they turn these features off with those documented flags. Yeah only if they know to check! This is pretty much the unsafe-by-default anti-pattern. You know, mysql_real_escape(), yaml.load(disable_gaping_security_holes=True) and so on. That said, the actual information looks fairly innocuous and file paths get accidentally embedded in C++ binaries all the time with no issues.
- xaduha 5y agoOnly tangentially related, but go:embed is pretty neat https://blog.carlmjohnson.net/post/2021/how-to-use-go-embed/ https://blog.carlmjohnson.net/post/2021/how-to-use-go-embed/
- JyB 5y agoCan you access that auto-embed information at runtime with some stdlib calls?
- jmcnulty 5y agoThere used to be a lot more information in binaries in general. These days everything is stripped. Giving my age away here :)
- userbinator 5y agoYour comment indicates you didn't experience the era where binaries had nothing more than the machine instructions and data necessary to execute. If anything, I think binaries now are more bloated than ever.
- drdaeman 5y agoI remember different. It used to be controversial when compilers of old had embedded some identifiers into the binaries they produced - like the infamous MSVC-added "Rich" header or I think Borland compilers used to have something like that too. Some people used to strip or blank out all sort of identifiers for various reasons (IIRC, primarily to complicate analysis with automated tools).
- alexfromapex 5y agoNot familiar enough with Go to know, would this aid hacking?
- vladak 5y agoI wonder what other languages/ecosystems embed the versioning into the binaries. This reminds me of the mcs(1) command (Manipulate Comment Section) used for some Solaris 10 binaries to display the patch level information: # /usr/ccs/bin/mcs -p /kernel/crypto/sparcv9/aes /kernel/crypto/sparcv9/aes: @(#)SunOS 5.10 Generic 150400-64 Dec 2018
- tandr 5y agoHmm... so many questions 1. Go binaries are bloated already. Do they plan to compress this (and other textual info, like full type names) in the binary? 2. I saw enough of build pipelines that do not clone repositories, but do a shallow copy for a tag, and have no .git folder. How much info will be added in this case? 3. Moreover, pretty much every serious build has source generation steps (protobuf, jooq, etc - you name it). If resulting executable will have a very particular info about these, the sheer amount of information added will probably make executables considerable larger.
- capableweb 5y ago1. It seems to be very small amount of information, if you compare to what already being included in a typical Golang binary. What's 1kb (or even 10kb) when your binary size is already huge thanks to Golang? If binary size is very important for you, then you would surely use something else than Golang in the first place.
- pqb 5y ago1) A lot of CI pipelines already do it by utilizing ldflags in go-build command. I don't think if few kilobytes more will matter much. 2): It will not be added (based on information provided in the article [0]). 3): Is it valid question? By my basic understanding of article the case of adding more bytes to binary will require having the main package [0]. Also, generation (e.g. gogo-proto, go:generate) will create only business logic code in new package, no additional metadata will be added. [0] Quote from article: > Version control information is embedded if the go command is invoked in a directory within a Git or Mercurial repository, and the main package and its containing main module are in the same repository.
- tempodox 5y agoNice to see compiler support for something like that. We've been doing stuff like this in C(++) for decades, extractable with the `what` command. It's more fiddly without direct compiler support, but on the plus side you can adapt it to any VCS.
- johnisgood 5y agoOK, all I remember is that many people have complaints about Go executables being huge. How does this affect it?