11 ms·
Add opt-in transparent telemetry to Go toolchain
- wageslave99 3y agoTangential topic, but if Golang had any way to have constant pointers and frozen variables, the language would be my favorite.
- pc86 3y agoThat sounds more unrelated than tangential.
- bryanlarsen 3y agoIOW, it's a 90 degree tangent?
- Conscat 3y agoA language without operator overloading is non-starter.
- catlifeonmars 3y agoFor which use case?
- omginternets 3y agoFunny, for me it’s the other way around. Ain’t no way I’m going back to the land of C++ operator abuse, where ‘+’ doesn’t even mean ‘+’ anymore.
- Conscat 3y agoWithout operator overloading, you cannot define arithmetic data types that have a homogenous API with fundamental types. Since math is.. basically all that computers do aside from some IO here and there, that's means a language without operator overloading is a domain-specific language. Curiously, none of these languages that lack operator overloading come close to covering all of the common use cases for math, except perhaps Odin.
- keybored 3y agoA language without user-defined operators is a non-starter. ;)
- bpicolo 3y agoWhen reading the title, I was hoping it would mean better abstractions than context for passing around OpenTelemetry info in a golang codebase
- xyzzy_plugh 3y agoIt's a real shame libraries like OpenTelemetry encourage using the context to pass information around. In my programs I prefer to be explicit. Sadly this breaks so many libraries which also depends on having a magically configured context to function properly. It's thread local storage all over again.
- XorNot 3y agoI like explicit but I don't want to do a whole lot of extra typing either. What I'd really like to see if the type system automatically infer the type of something like the context object implicitly.
- omginternets 3y agoHmm, my editor is able to autocomplete this. Not yours?
- jerf 3y agoI assume you mean that your editor is able to autocomplete that something is a context.Context. I assume what the GP means is to infer a strong type for a struct that has all the data a context has in it. This is much harder, for many reasons. A context that comes in from one path may have a RequestSource in it, but another may not; the resulting type of the function is rather complicated to infer. We prefer not to have two different functions as a result. There's also the problem that such inferences end up strongly tying types across many functions together, such that up in some middleware for your web site you add a new value into the context, and if there was a strong type for that context, that strong type would cascade throughout everything that could someday possibly touch that context. The result is much like checked exceptions in Java. This is perhaps the hardest practical type problem I know. It seems to me to be very related to a similar problem, which is that of trying to strongly type errors. The sort of cross-scope type inference we envision in our heads is extremely unwieldy in practice, if you take the time to try to scope out what that would convert to in a real program with many nested scopes and arbitrarily complicated paths in to those scopes, plus arbitrarily complicated closures being passed around that further impact the types.
- parabyl 3y agoIt'll be interesting to see how this is accepted by the community at large. I think explicit opt-in, combined with having the discussion about which metrics to collect in public, would be enough assurance for most that this isn't a bad idea - but doesn't neccesarily mean that most will opt-in as a result.
- JohnFen 3y agoThat they made it opt-in means that I will no longer completely rule out using golang. I don't know if I'd actually opt in, though. I haven't evaluated that issue.
- laputan_machine 3y agoGoogle doing more spying, unsurprising. At least it's opt-in (for now)
- jeroenhd 3y agoAs long as they keep this opt-in, I see no reason to accuse them if anything. The telemetry collection design is clearly made to be as privacy preserving as possible. There's always the risk that they'll roll out the telemetry setup now as an opt-in feature and then switch it to opt-out down the line, but I don't think this is the current team's intention.
- davidw 3y agoWhen did logging and reporting become "telemetry"?
- mrwnmonm 3y agoWhat does telemetry mean?
- davidw 3y agoIt's accurate in that it's the "remote collection and transmission of data", but it's a term traditionally associated with aerospace. Software has been doing this for a long time, but only recently have people started calling it "telemetry".
- EscapeFromNY 3y agoIt's not just an aerospace term. It's a general term dating back centuries across dozens of different industries. https://en.wikipedia.org/wiki/Telemetry https://en.wikipedia.org/wiki/Telemetry Look at that list under Applications. What makes software so different that it shouldn't be included?
- yamtaddle 3y agoFor my part: it didn't used to be called that, and feels like software folks trying to feel fancier than they are (and/or whitewashing bad practices by pushing new-to-the-industry terminology that makes spying sound scientific) Though at this point it's been widespread long enough that I suppose it's just a normal term that's "always been used", to some developers, not a new, alien-feeling part of the software lexicon.
- EscapeFromNY 3y agoI think the word telemetry is fine. It's concise, accurate, and not emotionally loaded in either direction. Microsoft OTOH calls their data collection the "Customer Experience Improvement Program" - now THAT is whitewashing. It's only been a decade or so now that you could rely on most computers always having an internet connection. That's probably why the term feels new - it only started being used when the practice became technically feasible. Maybe others remember things differently. /shrug
- bsaul 3y agoVery relieved that they chose to back off the initial opt-out proposal. It’s always refreshing to see a language listening to its user base. This was the kind of decision that could instantly change the reputation of a PL, for purely political / psychological reasons.
- varispeed 3y agoIt's a classic case of Overton Window They get users accustomed to the idea that now telemetry is in the toolchain. Next step will be to "accidentally" turn it on for everyone. Then it will be "oopsie daisy" everyone is okay? See. Nothing happened, so we'll leave it opt out.
- rob74 3y agoI very much doubt that everyone would be okay with them "accidentally" turning telemetry on by default while continuing to say that it's opt-in. In fact, it would be very damaging for the Go project's reputation, especially since everyone already associates Google with "spying on users"...
- candiddevmike 3y agoThe vast majority of folks downvoted the initial proposal and they're still moving forward with it (albeit as opt-in now), so I don't think everyone's opinion really matters here. Go isn't a democracy, for better or for worse.
- falcolas 3y agoOpt-in is a huge concession, especially since it means they will lose a lot of the information they would have gotten had they kept it opt-in. It makes built-in telemetry more palatable, because it gives the users a choice. In contrast with how Homebrew implemented their opt-out telemetry, opinions obviously did matter to Go. Yes, I'm still salty about that project being so tone deaf.
- jeroenhd 3y agoI like that they changed their approach to opt-in. I also like how much effort they've put into making the data collection as anonymous as possible despite being a Google project. Well done, golang team. Other companies with supposedly open languages (looking at you, Microsoft) can learn a thing or two from you.
- varispeed 3y ago> IP addresses exposed by the HTTP session that uploads the report are not recorded with the reports. Like pinky promise, trust me bro. This whole thing looks delusional. I hope someone is going to create a fork as Golang team has lost their marbles.
- tgv 3y agoThere aren't many orgs and people I would trust over this, but the go team gets the benefit of the doubt. They've proven that they're serious.
- ericpauley 3y agoWhat is the point of a fork? Just don't opt in.
- varispeed 3y agoThat answer it akin to "just use VPN" every time there is announcement of some kind of censorship of the web. The point is, this is just introducing another security threat to worry about. You now have to be aware that the toolchain can call home and ensure it doesn't happen. Someone might misconfigure it or you don't know if a release comes that has a "bug" and it sends telemetry regardless of settings. This is just completely wrong and should be nipped in the bud. The mere fact that they are pressing ahead with this is sinister. These things are never about "oh just don't opt in". If someone really feels the need to send telemetry to Google and be spied on, they should use a completely separate tool that is not included in the toolchain.
- remram 3y agoEvery program can call home, if it doesn't do what its author says it does. This is not specific to compiler toolchains, or this toolchain, and is not new.
- varispeed 3y agoaka "everything can happen" straw man. There is a difference between anything might happen in the future vs something specific is going to happen.
- liampulles 3y agoGiven how happy people seem to be sending large parts of their codebases to LLMs these days, privacy concerns over telemetry logging look quaint in comparison.
- carapace 3y ago(Maybe I'm just cranky this morning, so please ignore this comment if it fails to please you, but uh... I think the "real WTF" is getting into a position where you need telemetry at all. It's hard to describe what I mean, it seems like most folks who think about these things at all either get it or don't, and the two positions are so self-evident to those that hold them that it blinds us to the rationale of the "other side". In any event, "compiler does not access network" Works For Me. Sorry for the noise.)
- hedora 3y agoYeah; I can’t imagine how terrible golang’s test infrastructure would have to be for this to provide them with any practical benefit. Their motivating example was something like “golang stopped working on clean macos, and no one noticed for months”, which kind of proves my point.
- stanleydrew 3y agoI don't think they need telemetry. Go is clearly a successful language and community without it. The whole point of the proposal is that things could be improved with some telemetry. Now of course you could weigh the tradeoffs and say the potential for improvement doesn't outweigh the risk of misuse, but it seems clear to me that reasonable people can disagree.
- deleted 3y ago[deleted]
- carapace 3y agoI really don't want to argue about this (so I don't know why I'm arguing about it. Like I said, I think I'm just cranky this morning.) > I don't think they need telemetry. Go is clearly a successful language and community without it. Right! So why "pee in the soup"? > things could be improved with some telemetry Or they could not do those things? (I read "Why Telemetry?" and remain unmoved.) > you could weigh the tradeoffs I have. > the potential for improvement doesn't outweigh the risk of misuse That's not my argument. It's not even misuse that I care about here (I do care about that, but that's a separate concern, and one that doesn't arise if you don't collect data in the first place, eh?), I care about use. I don't want my compiler to make network connections to Google or anybody, for any reason. > reasonable people can disagree That's what I said.
- dudeous20230517 3y ago[flagged]
- ape4 3y agoIs it in the toolchain or in the applications made with Go?
- kitd 3y agoToolchain
- cratermoon 3y agoJust the toolchain. https://github.com/golang/go/issues/58894 https://github.com/golang/go/issues/58894
- kklisura 3y agoCan someone explain how we managed to have programming languages and toolchain development for half a century without using telemetry, but somehow we need it today in our tools? Their Why Telemetry? blog, just doesn't cut it for me [1]. [1] https://research.swtch.com/telemetry-intro https://research.swtch.com/telemetry-intro
- shadowgovt 3y agoIt'll be interesting to see what effect telemetry has on the ongoing development of the tools and language, since we don't have another tool / language chain to compare it to.
- metaltyphoon 3y agoYou can somewhat compare to dotnet where it has opt-out telemetry.
- arp242 3y agoYou can say "we managed to do X without Y" for a lot of values of X and Y. I think that's the wrong way to go about things; instead it's more useful to ask "will this be useful?" There's a long list of real-world use cases in part 3 of that blog series. I miss telemetry in my app sometimes too; there's some features where I wonder if anyone actually uses this, and I also don't really know what kind of things people run in to. Simply "ask people" is tricky, as I don't really have a way to contact everyone who cloned my git repo, and in general most people tend to be conservative in reporting feedback. I have found this a problem even in a company setting with internal software: people would tell me issues they've been frustrated at for months over beers in the pub, when this was sometimes just a simple 5 minute tweak that I would be happy to make. Can I make my app without telemetry? Obviously, yes. And I have no plans to ever add it. But that doesn't mean it's not useful.
- jjav 3y ago> instead it's more useful to ask "will this be useful?" Well, that's also the wrong way to look at it. Because everything, no matter how broadly bad it might be, is useful to someone somewhere. Of course telemetry can be useful to the developer of the application (if they look at the data and act on it). But at the same time it violates the privacy of all its users, who vastly outnumber (at least for most projects) its developers. For any argument we need to look at pros & cons, not just the pros.
- TDiblik 3y agoI might be wrong, but isn't dotnet telemetry opt-out by default?
- omginternets 3y agoI was one of the more vocal opponents of opt-in telemetry in the original discussion. One thing that seems clear to me coming out of that exchange is that the Go devs are quite anxious to get good-quality data, and are concerned that not enough people will opt in. I for one plan to enable it, and I hope others will do the same.
- jackmott42 3y agoHow is telemetry so valuable that language maintainers feel the need to introduce it even when it is so controversial?
- Patrickmi 3y agoWhen you’re in organization with strict goals and no resource for public tests implementation you’ll have to make it up for correct information
- 0cf8612b2e1e 3y agoIn what world is Google strapped for resources?
- Patrickmi 3y agoI think you’re overestimating the level of endurance ( google’s money) have on open source. Firstly Google as an Organization needs revenue to support go’s team expenses ( and I wouldn’t be surprised to them if they cut some cash because of OpenAI and MSC threat), open source teams especially in big corporations IMO have limits to resources dedicated especially if the project can’t be placed at the “platform level”, example take android source projects ideally I see no difference how both teams project their will in the direction on where the project will go but google must have a greater influence on android projects because they’re dependent on it seriously (basically 75% of revenue comes there), so what does google that depends on Go?, am pretty sure Go’s codebase hasn’t reached 7% or 10% at most, discontinue Go ?, fine they dedicate few team specific for their use case or just Rewrite it
- cesarb 3y agoI think adding telemetry to a compiler goes in the opposite direction of the current "minimal privileges, sandbox all the things" trend. For instance, it would make sense to create a set of selinux rules (or something like it) to make sure that a compiler cannot do anything other than reading its input files (and system headers/libraries/etc) and writing to its output directory, even if for instance a buffer overflow triggered by a malicious source code file led to running shell code within the compiler. Having to allow access to the network for the telemetry would require weakening these rules. It reminds me of the classic "confused deputy" article (https://css.csail.mit.edu/6.858/2015/readings/confused-deputy.html https://css.csail.mit.edu/6.858/2015/readings/confused-deput...), which coincidentally also involved a compiler tracking statistics about its usage.
- arp242 3y agoThat would actually already be difficult with the current go tool, since it's more than "just" a compiler but also fetches dependencies. If all dependencies are already in place it won't hit the network, so there are options, but you'd have to find another way to retrieve those. The telemetry is opt-in,and failing to send them won't fail the compile (it won't even run on every compile). It's not really preventing you from applying your SELinux policy if you want, even if it would have been opt-out.
- duped 3y ago> to make sure that a compiler cannot do anything other than reading its input files (and system headers/libraries/etc) Define "input files." Tools have to do a combination of reading, parsing, and sometimes even version unification/downloads just to get the complete set of inputs to feed to the compiler. Of course you can define the compiler as the tool that parses text and writes machine code, but then you're just shoveling dirty water around.
- aatd86 3y agoI will be the one overweighted in the report because I keep typing go mod init and go work init... I don't know I never actually remember the commands. :o)
- tjpnz 3y agoI'm fairly accustomed now to disabling this shite, but if it's only going to become more prevalent I could see it creating real toil. Are we also going to need a PiHole-like solution for servers and dev boxes?
- remram 3y agoIt's opt-in.
- pachico 3y agoIn general, I have no objections to well explained opt-in anything.
- JohnFen 3y agoGood on them! That's how you do telemetry without it becoming spying.
- Patrickmi 3y agoEvery Problem have a Solution, telemetry is good but abuse of anything is bad, if this follows strictly inline to what they have planned just like what they have been doing, I see no problem. Not everyone must be satisfied