5 ms·

Can someone explain how we managed to have programming languages and toolchain development for half a century without using telemetry, but somehow we need it to

Can someone explain how we managed to have programming languages and toolchain development for half a century without using telemetry, but somehow we need it today in our tools? Their Why Telemetry? blog, just doesn't cut it for me [1]. [1] https://research.swtch.com/telemetry-intro https://research.swtch.com/telemetry-intro

shadowgovt 3y ago
It'll be interesting to see what effect telemetry has on the ongoing development of the tools and language, since we don't have another tool / language chain to compare it to.
- metaltyphoon 3y ago
  You can somewhat compare to dotnet where it has opt-out telemetry.
arp242 3y ago
You can say "we managed to do X without Y" for a lot of values of X and Y. I think that's the wrong way to go about things; instead it's more useful to ask "will this be useful?" There's a long list of real-world use cases in part 3 of that blog series. I miss telemetry in my app sometimes too; there's some features where I wonder if anyone actually uses this, and I also don't really know what kind of things people run in to. Simply "ask people" is tricky, as I don't really have a way to contact everyone who cloned my git repo, and in general most people tend to be conservative in reporting feedback. I have found this a problem even in a company setting with internal software: people would tell me issues they've been frustrated at for months over beers in the pub, when this was sometimes just a simple 5 minute tweak that I would be happy to make. Can I make my app without telemetry? Obviously, yes. And I have no plans to ever add it. But that doesn't mean it's not useful.
- jjav 3y ago
  > instead it's more useful to ask "will this be useful?" Well, that's also the wrong way to look at it. Because everything, no matter how broadly bad it might be, is useful to someone somewhere. Of course telemetry can be useful to the developer of the application (if they look at the data and act on it). But at the same time it violates the privacy of all its users, who vastly outnumber (at least for most projects) its developers. For any argument we need to look at pros & cons, not just the pros.
  - arp242 3y ago
    I think that should go without saying, but yes, obviously you're correct that "useful" involves arguments about advantages and disadvantage. The previous poster wasn't talking about trade-offs though.
  - revelio 3y ago
    The vast, vast majority of people simply do not care about this type of privacy (from absolute strangers with no details about their personal lives). It's a nerd canard that this type of data sharing matters. Android, iOS, Windows, macOS, Chrome, games consoles, Docker, VS Code, IntelliJ etc. They all collect telemetry and stats on how they are used. That didn't stop them becoming monster success stories even amongst the developer population because nobody cares. The Go team are right to do this. Really, we should all be following their lead. Software telemetry is essentially pure win with no downsides for end users, which is why everyone has adopted it. It can also be done in better ways than we do now, like by recording stats in human readable form in files that sit around for a while before they get uploaded, so uploads can be turned to manual mode for inspection by the 0.1% of people who do seriously care.
- deleted 3y ago
  [deleted]
- JohnFen 3y ago
  > I think that's the wrong way to go about things; instead it's more useful to ask "will this be useful?" I think that's the wrong question as well. The right question is "does this provide benefits in excess of the costs"?
  - mrguyorama 3y ago
    Every new feature starts at -100 points and must have at least 100 points to be added to the list of features. You should only be adding telemetry if you can verifiably prove it will give you information you can't get otherwise, that you will actually use that info (what features hit the most bugs doesn't freaking matter if you are spending 95% of ever sprint doing completely different things) and only if you can find a way to legally guarantee that info is used for NOTHING else. Microsoft did not have broad telemetry in Windows in the 90s, and yet Raymend Chen had no trouble getting popular software and running it to find out what problems it ran in to. When vista had basic telemetry, all they found out was that nVidia makes crash prone drivers (50% of all Vista BSODs) but that didn't help them at all. People still blamed Vista for all the problems not caused by Vista, and nVidia was not getting that telemetry. Telemetry is a weird crutch that people keep latching onto before they even break their legs.
gjulianm 3y ago
You can manage to have things without telemetry, but having telemetry is incredibly useful. I think the example of "how much of our user base actually uses these features" is a very good one, specially in a compiler where maintaining old features could be adding a lot of complexity to the code base. And, as they also explain, a lot of bugs and undesired behaviors are things that the users won't know they have to report and just accept as part of the normal behavior. Things like cache misses, slowed down compilation times in certain situations, sporadic crashes... All of those things could be improved if the developers knew about them.
- estebank 3y ago
  A common concern I see is people being worried that the reaction to having "how much of our user base actually uses these features" answered will be to remove the feature entirely, when in practice it is more of "we think implementation A of the feature is no longer in use and we have migrated everyone to implementation B, is that the case?" or "we have implementation A produce user visible effects, while we have implementation B run parallel to it to detect divergences, have any been detected in the wild in the past X months?". In Rust we've had large migrations like that (the "new" borrow checker comes to mind) and we had a really long periods of time where they were tested against the latest crate versions on crates.io until crater came back clean, and even then we had bug reports about regressions in the wild only after it was released on stable. For me personally there's one big blind spot when testing only against published code, no matter how big the corpus is: humans are excellent fuzzers and the malformed code they write and try to compile is hard to replicate. Having visibility into uncommon cases that are only visible on users machines would be incredibly useful. An example of this could be the "botched" 1.52.0 Rust release[1], where invalid incremental compilations were changed from silent to visible Internal Compiler Errors in nightly for several releases to the point where the team felt all of the outstanding incr comp bugs related to them had been addressed, but when turned on in stable immediately hit users in the real world, making it necessary to do an emergency dot-release reverting that change. If we had telemetry on stable compilers, instead of turning on the silent to ICE change we could have added a metric for the silent error and know ahead of time that the feature wasn't ready for prime time. With more telemetry we could have known what was causing this. Snooping over a user's shoulder while they hit the case could make identifying the bug trivial, but of course then that user would then turn around and ask me in unfriendly terms "who are you and how did you get in?" Arguments can be made against even implementing telemetry, and they can be compelling enough to elect against it, but shouting down the conversation from even happening is not helpful. [1]: https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html
eternalban 3y ago
Two things happened in '00s in relation to this question. One, on demand computing infrastructure (cloud), and two, the scaling requirements of a new breed of networked services. The germinal change in response to these shifts was that processes replaced components, and system boundaries spanned devices. When your code is running on application servers and your applications are composed of components, all the tools were already there, in the OS and as add ons, like dtrace, and in whatever monitoring tools came with your application server. Today, instead of components, we compose systems out of (lightweight) processes, and processes can be created on any device, and the replacement for the application server is the whole gamut of k8, terraform, elastic, ..., etc. Nothing has changed in the abstract structure of our systems, its just that the current approach has the beast dismembered and spread out and loosely connected via protocols, instead of a linker or a dispatch mechanism of a platform.
estebank 3y ago
Humans managed to live for most of history without penicillin, or even boiling water, at the cost of most humans dying before making it to adolescence. People managed to have global communications with only steam ships and telegraph, at the cost of slower pace of information dissemination. NASA managed to make it to the moon with less computing power than the cellphone in your pocket, at high resource, monetary, human and time costs. Cars managed to work with more rudimentary design than today's, without any computers, at the cost of lower life-spans, lower efficiency and higher pollution. You can make many arguments for and against telemetry in developer tools. Not acknowledging that telemetry helps with visibility into how those tools actually work in the wild, which in turn helps lower the incidence of bugs and speed up development, is disingenuous. You can arrive to the conclusion that even inert, opt-in telemetry is not worth it, but don't disregard out of hand the utility of it in helping their development as if it were some crazy idea.
- hinkley 3y ago
  And yet when NASA went to the moon, they had telemetry data.
  - deleted 3y ago
    [deleted]
  - tredre3 3y ago
    > And yet when NASA went to the moon, they had telemetry data. NASA couldn't obtain that same data locally, they had to know how the vehicle and software behaved in the real situation. NASA's telemetry also ran on their own hardware, not on arbitrary users'. Contrast that to Go, which is used in real situations internally at Google. They don't need to spy on their users, they have first hand experience using the tool.
    - Patrickmi 3y ago
      > Contrast that to Go, which is used in real situations internally at Google. They don't need to spy on their users, they have first hand experience using the tool. And that’s a big disadvantage if I were to scale the usage of Go then I would say startups and middle orgs occupies like 80% if the Go team work according to logic at google scale, Go wouldn’t be this successful
jjav 3y ago
> Can someone explain how we managed to have programming languages and toolchain development for half a century without using telemetry, but somehow we need it today in our tools? Back in the 90s in companies I worked for, attempting to add any kind of phone-home code was a fireable offense. Or at least, would get you a very stiff talking to from a few very high up people in the organization. You'd never even think of doing that again. Customer trust and privacy was paramount. As we all know, spyware slowly started creeping into end user apps and later became a flood. Now it's difficult to find any consumer app that doesn't continuously leak everything the user does. It's become so normalized that now even developers tools seem to think it's somehow ok to leak user data.
JohnFen 3y ago
I think there is a whole generation of developers who have no experience with how to do these things in the absence of telemetry, so they genuinely believe it's not possible.

shadowgovt 3y ago

It'll be interesting to see what effect telemetry has on the ongoing development of the tools and language, since we don't have another tool / language chain to compare it to.

metaltyphoon 3y ago
You can somewhat compare to dotnet where it has opt-out telemetry.

arp242 3y ago

You can say "we managed to do X without Y" for a lot of values of X and Y. I think that's the wrong way to go about things; instead it's more useful to ask "will this be useful?" There's a long list of real-world use cases in part 3 of that blog series. I miss telemetry in my app sometimes too; there's some features where I wonder if anyone actually uses this, and I also don't really know what kind of things people run in to. Simply "ask people" is tricky, as I don't really have a way to contact everyone who cloned my git repo, and in general most people tend to be conservative in reporting feedback. I have found this a problem even in a company setting with internal software: people would tell me issues they've been frustrated at for months over beers in the pub, when this was sometimes just a simple 5 minute tweak that I would be happy to make. Can I make my app without telemetry? Obviously, yes. And I have no plans to ever add it. But that doesn't mean it's not useful.

jjav 3y ago
> instead it's more useful to ask "will this be useful?" Well, that's also the wrong way to look at it. Because everything, no matter how broadly bad it might be, is useful to someone somewhere. Of course telemetry can be useful to the developer of the application (if they look at the data and act on it). But at the same time it violates the privacy of all its users, who vastly outnumber (at least for most projects) its developers. For any argument we need to look at pros & cons, not just the pros.
- arp242 3y ago
  I think that should go without saying, but yes, obviously you're correct that "useful" involves arguments about advantages and disadvantage. The previous poster wasn't talking about trade-offs though.
- revelio 3y ago
  The vast, vast majority of people simply do not care about this type of privacy (from absolute strangers with no details about their personal lives). It's a nerd canard that this type of data sharing matters. Android, iOS, Windows, macOS, Chrome, games consoles, Docker, VS Code, IntelliJ etc. They all collect telemetry and stats on how they are used. That didn't stop them becoming monster success stories even amongst the developer population because nobody cares. The Go team are right to do this. Really, we should all be following their lead. Software telemetry is essentially pure win with no downsides for end users, which is why everyone has adopted it. It can also be done in better ways than we do now, like by recording stats in human readable form in files that sit around for a while before they get uploaded, so uploads can be turned to manual mode for inspection by the 0.1% of people who do seriously care.
deleted 3y ago
[deleted]
JohnFen 3y ago
> I think that's the wrong way to go about things; instead it's more useful to ask "will this be useful?" I think that's the wrong question as well. The right question is "does this provide benefits in excess of the costs"?
- mrguyorama 3y ago
  Every new feature starts at -100 points and must have at least 100 points to be added to the list of features. You should only be adding telemetry if you can verifiably prove it will give you information you can't get otherwise, that you will actually use that info (what features hit the most bugs doesn't freaking matter if you are spending 95% of ever sprint doing completely different things) and only if you can find a way to legally guarantee that info is used for NOTHING else. Microsoft did not have broad telemetry in Windows in the 90s, and yet Raymend Chen had no trouble getting popular software and running it to find out what problems it ran in to. When vista had basic telemetry, all they found out was that nVidia makes crash prone drivers (50% of all Vista BSODs) but that didn't help them at all. People still blamed Vista for all the problems not caused by Vista, and nVidia was not getting that telemetry. Telemetry is a weird crutch that people keep latching onto before they even break their legs.

gjulianm 3y ago

You can manage to have things without telemetry, but having telemetry is incredibly useful. I think the example of "how much of our user base actually uses these features" is a very good one, specially in a compiler where maintaining old features could be adding a lot of complexity to the code base. And, as they also explain, a lot of bugs and undesired behaviors are things that the users won't know they have to report and just accept as part of the normal behavior. Things like cache misses, slowed down compilation times in certain situations, sporadic crashes... All of those things could be improved if the developers knew about them.

estebank 3y ago
A common concern I see is people being worried that the reaction to having "how much of our user base actually uses these features" answered will be to remove the feature entirely, when in practice it is more of "we think implementation A of the feature is no longer in use and we have migrated everyone to implementation B, is that the case?" or "we have implementation A produce user visible effects, while we have implementation B run parallel to it to detect divergences, have any been detected in the wild in the past X months?". In Rust we've had large migrations like that (the "new" borrow checker comes to mind) and we had a really long periods of time where they were tested against the latest crate versions on crates.io until crater came back clean, and even then we had bug reports about regressions in the wild only after it was released on stable. For me personally there's one big blind spot when testing only against published code, no matter how big the corpus is: humans are excellent fuzzers and the malformed code they write and try to compile is hard to replicate. Having visibility into uncommon cases that are only visible on users machines would be incredibly useful. An example of this could be the "botched" 1.52.0 Rust release[1], where invalid incremental compilations were changed from silent to visible Internal Compiler Errors in nightly for several releases to the point where the team felt all of the outstanding incr comp bugs related to them had been addressed, but when turned on in stable immediately hit users in the real world, making it necessary to do an emergency dot-release reverting that change. If we had telemetry on stable compilers, instead of turning on the silent to ICE change we could have added a metric for the silent error and know ahead of time that the feature wasn't ready for prime time. With more telemetry we could have known what was causing this. Snooping over a user's shoulder while they hit the case could make identifying the bug trivial, but of course then that user would then turn around and ask me in unfriendly terms "who are you and how did you get in?" Arguments can be made against even implementing telemetry, and they can be compelling enough to elect against it, but shouting down the conversation from even happening is not helpful. [1]: https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html https://blog.rust-lang.org/2021/05/10/Rust-1.52.1.html

eternalban 3y ago

Two things happened in '00s in relation to this question. One, on demand computing infrastructure (cloud), and two, the scaling requirements of a new breed of networked services. The germinal change in response to these shifts was that processes replaced components, and system boundaries spanned devices. When your code is running on application servers and your applications are composed of components, all the tools were already there, in the OS and as add ons, like dtrace, and in whatever monitoring tools came with your application server. Today, instead of components, we compose systems out of (lightweight) processes, and processes can be created on any device, and the replacement for the application server is the whole gamut of k8, terraform, elastic, ..., etc. Nothing has changed in the abstract structure of our systems, its just that the current approach has the beast dismembered and spread out and loosely connected via protocols, instead of a linker or a dispatch mechanism of a platform.

estebank 3y ago

Humans managed to live for most of history without penicillin, or even boiling water, at the cost of most humans dying before making it to adolescence. People managed to have global communications with only steam ships and telegraph, at the cost of slower pace of information dissemination. NASA managed to make it to the moon with less computing power than the cellphone in your pocket, at high resource, monetary, human and time costs. Cars managed to work with more rudimentary design than today's, without any computers, at the cost of lower life-spans, lower efficiency and higher pollution. You can make many arguments for and against telemetry in developer tools. Not acknowledging that telemetry helps with visibility into how those tools actually work in the wild, which in turn helps lower the incidence of bugs and speed up development, is disingenuous. You can arrive to the conclusion that even inert, opt-in telemetry is not worth it, but don't disregard out of hand the utility of it in helping their development as if it were some crazy idea.

hinkley 3y ago
And yet when NASA went to the moon, they had telemetry data.
- deleted 3y ago
  [deleted]
- tredre3 3y ago
  > And yet when NASA went to the moon, they had telemetry data. NASA couldn't obtain that same data locally, they had to know how the vehicle and software behaved in the real situation. NASA's telemetry also ran on their own hardware, not on arbitrary users'. Contrast that to Go, which is used in real situations internally at Google. They don't need to spy on their users, they have first hand experience using the tool.
  - Patrickmi 3y ago
    > Contrast that to Go, which is used in real situations internally at Google. They don't need to spy on their users, they have first hand experience using the tool. And that’s a big disadvantage if I were to scale the usage of Go then I would say startups and middle orgs occupies like 80% if the Go team work according to logic at google scale, Go wouldn’t be this successful

jjav 3y ago

> Can someone explain how we managed to have programming languages and toolchain development for half a century without using telemetry, but somehow we need it today in our tools? Back in the 90s in companies I worked for, attempting to add any kind of phone-home code was a fireable offense. Or at least, would get you a very stiff talking to from a few very high up people in the organization. You'd never even think of doing that again. Customer trust and privacy was paramount. As we all know, spyware slowly started creeping into end user apps and later became a flood. Now it's difficult to find any consumer app that doesn't continuously leak everything the user does. It's become so normalized that now even developers tools seem to think it's somehow ok to leak user data.

JohnFen 3y ago

I think there is a whole generation of developers who have no experience with how to do these things in the absence of telemetry, so they genuinely believe it's not possible.