6 ms·

Telemetry is godsend for developers, since it lets them track down actual product usage and issues. Telemetry is a privacy nightmare for users, since it sends

Telemetry is godsend for developers, since it lets them track down actual product usage and issues. Telemetry is a privacy nightmare for users, since it sends data to the developers outside of an average user's control - data which is probably easily associated with an individual and kept for eons (disk space is cheap after all). I think telemetry collection is starting to become a professional ethics issue for developers, and should be talked about more.

phantomathkg 4y ago
As a developer tool, I think it is absolutely possible for Microsoft and extension developers to write a very deep and detail explanation to the users and explain why do they need to collect what telemetry data. As a user and engineer, I am more than happy to share the usage and crash data with them, but at the same time, I would appreciate if they can be fair to us the user.
- zerkten 4y ago
  They publish research updates like https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/articles/measurably-improve-your-product-by-combining-qualitative-and-quantitative-methods/ https://www.microsoft.com/en-us/research/group/experimentati....
bentcorner 4y ago
What do you think about regulation like the GDPR?
- falcolas 4y ago
  I like them. I don't really understand how it would apply to telemetry - it's not something I've read into - and I don't think many other folks are looking into GDPR compliance with telemetry either.
  - zerkten 4y ago
    All of the big companies have armies of lawyers deeply familiar with these laws and regulations. They make their engineering teams implement processes to mitigate the issues with storage and processing. Examples: replacing/removing external IDs, ensuring data can't be linked to users, aggregating data, not retaining data, only processing data within data centers (no data extracted to laptops), deleting old data, etc. This is why it's slow and painful to work at big companies compared to startups. It doesn't mean that accidents don't happen, but it's pretty well thought out. The same applies to how they do qualitative research. It's got reasonably high standards which are fairly consistently applied. It's not perfect, but nothing is. There is a lot of job security and motivation for people to make it better. Smaller companies have almost zero understanding of how to deal with this due to lack of resources and expertise. The only way they get compliant is by using good tools which force the user into a safer, more compliant posture.
pseudalopex 4y ago
Collecting data without informed consent was an ethics issue always.
- shadowgovt 4y ago
  Not hardly. In the early days of the Internet it was seen as tit-for-tat among peers; you're accessing my server for data I have, of course I'm allowed to log and analyze your request. That was the exchange. The exchange was rarely made explicit in any kind of formalized consent sense.
  - uoaei 4y ago
    Informal consent (and lack thereof) is still consent (and lack thereof).
  - yencabulator 4y ago
    If I am typing into an editor running on my computer, I am not consciously accessing your server for data you have.
    - shadowgovt 4y ago
      I spend most of my time in Google Docs as the editor I'm typing in. And when I'm coding in emacs, I'm using Tramp to puppeteer a server somewhere else. The veil is thin between the internet and offline computing these days.
marginalia_nu 4y ago
> Telemetry is godsend for developers, since it lets them track down actual product usage This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so. Look at it this way: Your average fire extinguisher sits mounted on a wall its entire life and is never actually used. You still rather have it and not need it than need it and not have it.
- falcolas 4y ago
  > This is is actually a terrible trap Agreed. > You still rather have it and not need it than need it and not have it. And there's the trap. Data is a liability, but until a company gets burned for losing it, they won't do anything about it. Well, being real, even if they do get burned it's just the "cost of doing business".
  - lcnPylGDnU4H9OF 4y ago
    I get what you're saying and I agree but I also think you misinterpreted the parent comment. > You still rather have it and not need it than need it and not have it. Given the context of the fire extinguisher, "it" in this case is a useful (possibly critical), not-commonly-needed feature, rather than a collection of data.
- ajross 4y ago
  > This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so. This seems exactly backwards to me. No one who has ever managed a deployed product with good failure telemetry would say that it tells you "next to nothing". Telemetry detects failures that mere testing simply never can and never will. And no one who lived through earlier ages of consumer software would ever say that it's "turned to shit". Software is getting better. It's getting better steadily and inexorably, and it's been doing that for half a century. It may not be doing what you want, but it's doing it with higher quality than people in the 90's or 00's would have ever dreamed. When was the last time[1] you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? This was a daily (or worse) event for most of software history! And certainly robust telemetry practice is one of the big drivers of this quality revolution we've seen. [1] Heh, cue the peanut gallery of everyone wanting to post crash reports. That's on me for poor framing, I guess. But seriously, everyone: Both anecdotally and via actual studies, consumer-visible software is getting much better.
rkagerer 4y ago
This is not a nuanced argument or compex issue. Include an OFF switch. It's that simple. Personally I refuse to work with or consult for companies that don't get this. It's a matter of professional pride. If I wouldn't use it myself, I'm not going to foist it on my users.
- shadowgovt 4y ago
  One of the challenges is defining what counts as telemetry. So they include an OFF switch. Does it also switch off checking for new versions of VSCode? Does it turn off consulting online sources for dictionary updates? Does it turn off viewing the extension library? Those requests are technically also telemetry but if you disable them they look like broken features. The license agreement is written with open language to say "There are some things you can't turn off" because MS doesn't know a priori whether a court would consider "updating the extension library" telemetry until someone drags them into court and presses the issue legally, and they want their asses covered either way. And the OFF switch doesn't switch off any extensions because the API is open enough that extensions can do their own network access independent of VSCode. In practice, it turns out to be very hard to build one guaranteed-to-work OFF switch. Not that it isn't a laudable goal. But in general: if it has online features, assume telemetry == true.
  - EVa5I7bHFq9mnYK 4y ago
    Those examples are not telemetry because they only pull data from internet, without sending any specific information. You could check for new versions of VSCode or browse extensions library using just a browser.
    - shadowgovt 4y ago
      That's definitely an argument one could make in a court of law. And it might carry the day. Corporations demand more certainty than "might carry the day" from their legal departments when drafting terms of service.
shadowgovt 4y ago
> Telemetry is godsend for developers, since it lets them track down actual product usage and issues. Extremely true. Data gleaned from actual usage instrumentation is radically higher quality than self-reported. > Telemetry is a privacy nightmare for users, since it sends data to the developers outside of an average user's control - data which is probably easily associated with an individual and kept for eons None of that is usually true. In the common case, users don't care (after all, MS has the telemetry to estimate how many users disable telemetry). And product improvement data is pseudonymized and usually worthless after a version iteration. I think the folks that have an anxiety about product use data collection don't realize how worthless individual data points are. The data is pseudonymized out of necessity if for no other reason; it comes in as such a firehose that it has to be immediately analytically bucketed or it overwhelms storage at the volume these tools get used. However, the telemetry options get defaulted to "on" because volumes of data are invaluable for concrete analysis of how the product is used, counts on feature access frequency, understanding pain points in usage flow, etc. I've done some work in this space (not for MS) and am happy to answer any questions people might have that aren't "Who did you work for?"
- OrvalWintermute 4y ago
  >The data is pseudonymized out of necessity if for no other reason; it comes in as such a firehose that it has to be immediately analytically bucketed or it overwhelms storage at the volume these tools get used My 20TB drives say otherwise
JohnFen 4y ago
> Telemetry is godsend for developers, since it lets them track down actual product usage I have come to think that telemetry used to track actual product usage is a bad thing. It encourages shallow thinking about how users really use the product, and often leads to terrible product decisions.

phantomathkg 4y ago

As a developer tool, I think it is absolutely possible for Microsoft and extension developers to write a very deep and detail explanation to the users and explain why do they need to collect what telemetry data. As a user and engineer, I am more than happy to share the usage and crash data with them, but at the same time, I would appreciate if they can be fair to us the user.

zerkten 4y ago
They publish research updates like https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/articles/measurably-improve-your-product-by-combining-qualitative-and-quantitative-methods/ https://www.microsoft.com/en-us/research/group/experimentati....

bentcorner 4y ago

What do you think about regulation like the GDPR?

falcolas 4y ago
I like them. I don't really understand how it would apply to telemetry - it's not something I've read into - and I don't think many other folks are looking into GDPR compliance with telemetry either.
- zerkten 4y ago
  All of the big companies have armies of lawyers deeply familiar with these laws and regulations. They make their engineering teams implement processes to mitigate the issues with storage and processing. Examples: replacing/removing external IDs, ensuring data can't be linked to users, aggregating data, not retaining data, only processing data within data centers (no data extracted to laptops), deleting old data, etc. This is why it's slow and painful to work at big companies compared to startups. It doesn't mean that accidents don't happen, but it's pretty well thought out. The same applies to how they do qualitative research. It's got reasonably high standards which are fairly consistently applied. It's not perfect, but nothing is. There is a lot of job security and motivation for people to make it better. Smaller companies have almost zero understanding of how to deal with this due to lack of resources and expertise. The only way they get compliant is by using good tools which force the user into a safer, more compliant posture.

pseudalopex 4y ago

Collecting data without informed consent was an ethics issue always.

shadowgovt 4y ago
Not hardly. In the early days of the Internet it was seen as tit-for-tat among peers; you're accessing my server for data I have, of course I'm allowed to log and analyze your request. That was the exchange. The exchange was rarely made explicit in any kind of formalized consent sense.
- uoaei 4y ago
  Informal consent (and lack thereof) is still consent (and lack thereof).
- yencabulator 4y ago
  If I am typing into an editor running on my computer, I am not consciously accessing your server for data you have.
  - shadowgovt 4y ago
    I spend most of my time in Google Docs as the editor I'm typing in. And when I'm coding in emacs, I'm using Tramp to puppeteer a server somewhere else. The veil is thin between the internet and offline computing these days.

marginalia_nu 4y ago

> Telemetry is godsend for developers, since it lets them track down actual product usage This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so. Look at it this way: Your average fire extinguisher sits mounted on a wall its entire life and is never actually used. You still rather have it and not need it than need it and not have it.

falcolas 4y ago
> This is is actually a terrible trap Agreed. > You still rather have it and not need it than need it and not have it. And there's the trap. Data is a liability, but until a company gets burned for losing it, they won't do anything about it. Well, being real, even if they do get burned it's just the "cost of doing business".
- lcnPylGDnU4H9OF 4y ago
  I get what you're saying and I agree but I also think you misinterpreted the parent comment. > You still rather have it and not need it than need it and not have it. Given the context of the fire extinguisher, "it" in this case is a useful (possibly critical), not-commonly-needed feature, rather than a collection of data.
ajross 4y ago
> This is is actually a terrible trap, as product usage tells you next to nothing about which features are useful. I suspect it' a big part of why a lot of software has turned to shit last decade or so. This seems exactly backwards to me. No one who has ever managed a deployed product with good failure telemetry would say that it tells you "next to nothing". Telemetry detects failures that mere testing simply never can and never will. And no one who lived through earlier ages of consumer software would ever say that it's "turned to shit". Software is getting better. It's getting better steadily and inexorably, and it's been doing that for half a century. It may not be doing what you want, but it's doing it with higher quality than people in the 90's or 00's would have ever dreamed. When was the last time[1] you saw a major consumer application (the TikTok or Instagram client, or your web browser, or VSCode) crash hard and fail on you? This was a daily (or worse) event for most of software history! And certainly robust telemetry practice is one of the big drivers of this quality revolution we've seen. [1] Heh, cue the peanut gallery of everyone wanting to post crash reports. That's on me for poor framing, I guess. But seriously, everyone: Both anecdotally and via actual studies, consumer-visible software is getting much better.

rkagerer 4y ago

This is not a nuanced argument or compex issue. Include an OFF switch. It's that simple. Personally I refuse to work with or consult for companies that don't get this. It's a matter of professional pride. If I wouldn't use it myself, I'm not going to foist it on my users.

shadowgovt 4y ago
One of the challenges is defining what counts as telemetry. So they include an OFF switch. Does it also switch off checking for new versions of VSCode? Does it turn off consulting online sources for dictionary updates? Does it turn off viewing the extension library? Those requests are technically also telemetry but if you disable them they look like broken features. The license agreement is written with open language to say "There are some things you can't turn off" because MS doesn't know a priori whether a court would consider "updating the extension library" telemetry until someone drags them into court and presses the issue legally, and they want their asses covered either way. And the OFF switch doesn't switch off any extensions because the API is open enough that extensions can do their own network access independent of VSCode. In practice, it turns out to be very hard to build one guaranteed-to-work OFF switch. Not that it isn't a laudable goal. But in general: if it has online features, assume telemetry == true.
- EVa5I7bHFq9mnYK 4y ago
  Those examples are not telemetry because they only pull data from internet, without sending any specific information. You could check for new versions of VSCode or browse extensions library using just a browser.
  - shadowgovt 4y ago
    That's definitely an argument one could make in a court of law. And it might carry the day. Corporations demand more certainty than "might carry the day" from their legal departments when drafting terms of service.

shadowgovt 4y ago

> Telemetry is godsend for developers, since it lets them track down actual product usage and issues. Extremely true. Data gleaned from actual usage instrumentation is radically higher quality than self-reported. > Telemetry is a privacy nightmare for users, since it sends data to the developers outside of an average user's control - data which is probably easily associated with an individual and kept for eons None of that is usually true. In the common case, users don't care (after all, MS has the telemetry to estimate how many users disable telemetry). And product improvement data is pseudonymized and usually worthless after a version iteration. I think the folks that have an anxiety about product use data collection don't realize how worthless individual data points are. The data is pseudonymized out of necessity if for no other reason; it comes in as such a firehose that it has to be immediately analytically bucketed or it overwhelms storage at the volume these tools get used. However, the telemetry options get defaulted to "on" because volumes of data are invaluable for concrete analysis of how the product is used, counts on feature access frequency, understanding pain points in usage flow, etc. I've done some work in this space (not for MS) and am happy to answer any questions people might have that aren't "Who did you work for?"

OrvalWintermute 4y ago
>The data is pseudonymized out of necessity if for no other reason; it comes in as such a firehose that it has to be immediately analytically bucketed or it overwhelms storage at the volume these tools get used My 20TB drives say otherwise

JohnFen 4y ago

> Telemetry is godsend for developers, since it lets them track down actual product usage I have come to think that telemetry used to track actual product usage is a bad thing. It encourages shallow thinking about how users really use the product, and often leads to terrible product decisions.