6 ms·

It boggles the mind that a bunch of very smart people at Microsoft thought this was a good idea. Say what you will about Oracle but Java has never been malware

by dnomad 8y ago

It boggles the mind that a bunch of very smart people at Microsoft thought this was a good idea. Say what you will about Oracle but Java has never been malware that continuously spies on users. Unfortunately the GDPR explicitly allows this kind of IoT-style ubiquitous anonymized monitoring: as long as the information cannot be tied back to an individual person (meaning no IP addresses but GUIDs are fine) and the business can claim a legitimate business interest they're likely to get away with it. This sort of ubiquitous monitoring is only going to get more popular. In Hong Kong we're seeing new Condo apartments that have full-fledged "internal sensor grids" a la Star Trek (as it was described to me). In theory though the data collection firm (which is not the same firm that owns the building) has no way of correlating it back to a specific person/apartment number.

manigandham 8y ago
Like all things, there must be a sense of context and proportionality. This issue is specifically about the .NET Core SDK command line tools. Getting data about which commands are used most, if they're successful or throwing errors, possible typos, unclear command flags, etc, are useful when shipping software frameworks used by 10s of millions to build products that run the entire Fortune 500 and thousands of other businesses. This is not ubiquitous spyware, and is about as generic as it gets while still remaining useful. Every software company (many of which are featured here on HN) does the same analytical tracking to figure out how their products are being used and it results in better features. What exactly is the danger here, especially with the lack of personal info and an opt-out method? They even make the data public along with insights into how it helped: https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-weve-learned-from-net-core-sdk-telemetry/ https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-weve...
- 0x0 8y ago
  Collecting command line arguments is a HUGE invasion of privacy. Imagine I ran a command "make.exe my-secret-project-name" or "make.exe process_gdrp_removal_request SOME_SSN". And as the linked page admits, they really do collect all command line arguments, not just a white list of valid commands. Perhaps someone accidentally pastes an AWS API Secret into the wrong terminal window and it ended up as "dotnet.exe MY_AWS_SECRET". ("You will notice misspellings, like “bulid”. That’s what the user typed. It’s information").
  - philliphaydon 8y ago
    They don't collect command line arguments according to the github issue.
    - 0x0 8y ago
      They do collect at least the first argument, how else do they end up with "bulid"? Accidents happen and someone could paste or tab-complete a confidential thing in that spot...
      - dingo_bat 8y ago
        So if you push your api key to github by accident is github responsible?
        0x0 8y ago
        Invalid comparison, in my humble opinion. Pushing a secret API key to github, even accidentally, does not happen covertly or as an unexpected side effect of some other tool written by github, and you'll soon realize what happens because you'll see your key sitting there. You also have a general idea of what you are playing with when you start using github, whether it's private repos or public repos. Having a local CLI tool submit your secret API key as part of telemetry collecting command line arguments is totally unexpected and invisible for the user, and you won't even see it has happened.
      - philliphaydon 8y ago
        I Guess they whitelist it. If someone types a valid command send the command. If someone types an invalid command then it prob sends something that isn’t descriptive. (Assumptions I’m making)
    - opencl 8y ago
      According to this PR they do collect command line arguments. https://github.com/dotnet/cli/pull/2145 https://github.com/dotnet/cli/pull/2145 "My understanding is that this was recently discussed with our privacy team and we concluded that collecting the arguments themselves (hashed or not) is not acceptable per our privacy policies. Not sure whether the code already reflects that, but it's being worked on." Somebody in the thread for the issue said it's "not acceptable per our privacy policies" and "being worked on" two years ago, but did they ever actually stop collecting command line arguments?
- setquk 8y ago
  Does this justify hashing user identification for later correlation with other products? https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1c6bddf20167f166/src/dotnet/Telemetry/Sha256Hasher.cs#L12 https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1...
- ptx 8y ago
  > Like all things, there must be a sense of context and proportionality. This issue is specifically about the .NET Core SDK command line tools. Exactly. Command line tools don't usually collect your data and send it off somewhere without asking. You wouldn't expect ls, cat and grep to spy on you. That's why it's so surprising. But even big interactive applications like IDEs don't normally do this without asking. IntelliJ had a dialog pop up just a few days ago to ask me nicely to participate, defaulting to non-participation (!). Visual Studio and Office used to do the same thing, as I recall. (Additionally, once you figure out how to opt-out in .NET Core, does it tell you somehow that you did so successfully? I think I've set the environment variable correctly, but how can I be sure?)
  - josteink 8y ago
    > Exactly. Command line tools don't usually collect your data and send it off somewhere without asking. You wouldn't expect ls, cat and grep to spy on you. That's why it's so surprising. Except in Ubuntu Linux where apt-get it has done so for ages without anyone complaining: https://askubuntu.com/questions/57808/what-is-the-popularity-contest-package-for#57811 https://askubuntu.com/questions/57808/what-is-the-popularity...
    - ptx 8y ago
      Has it? It says right on the page you're linking to that it's disabled by default, so you would have to explicitly opt in. Thus nobody complaining, as those who don't want it won't enable it. I think it comes from Debian[1] actually, where it's also opt-in – my Debian machine doesn't have it installed. Like it says in the Debian package description: "Vote for your favourite packages automatically". It's voting, not spying – strictly voluntary. Those of us who don't want to vote don't have to. [1] https://popcon.debian.org/ https://popcon.debian.org/
      - josteink 8y ago
        > it's disabled by default Fair enough, but it wasn’t always like that. > “Vote for your favourite packages automatically". It's voting, not spying That’s just playing with words. What it does is exactly the same: provide the upstream author statistics about what is being used and not. Surely you can’t mean that this is a serious argument.
- OnlyRepliesToBS 8y ago
  ~"you're just overreacting" ~"good for the user" ~"just the tip" ~"look how honest they were" Absolute bullshit. It was like pulling teeth just trying to get that information from Microsoft and that's not an order-of-magnitude all of it.
0x0 8y ago
Java has been pushing "sponsored offers" (i.e. unwanted spyware like browser toolbars and similar user-hostile software) in their installers for years, so they're not exactly a shining beacon of light.
- setquk 8y ago
  Yes Java has been a perpetural problem, at least the official JRE and JDK. The JRE ships with junk and the JDK forces you to forge an agreement cookie to download it. OpenJDK is fine however.
- ionised 8y ago
  I haven't had a problem with that for a while. The installers I've been getting directly from the Oracle JDK pages haven't installed anything extra, or asked to.
  - viraptor 8y ago
    Are you sure? It's been there since 2013 (https://www.computerworld.com/article/2494794/malware-vulnerabilities/oracle-will-continue-to-bundle--crapware--with-java.html https://www.computerworld.com/article/2494794/malware-vulner...) I don't think there's been any change since. Java 8 still had a page about disabling the offers: https://www.java.com/en/download/faq/disable_offers.xml https://www.java.com/en/download/faq/disable_offers.xml
    - 0x0 8y ago
      The option to suppress sponsor offers is still there in the control panel for Java 10.0.1. It's of course possible that some users don't experience these "sponsored offers" due to geotargeting, A/B-testing, lack of available "offers" at a specific time, or any other reason controlled by the server side of the installer.
pjc50 8y ago
If it is genuinely anonymous, then it's far less of a problem and not actually spying on individual users.
- worldsayshi 8y ago
  Yeah, I kind of agree here. If it is possible to track down a users identity in a systematic way when using these methods it shouldn't be too hard to prove that in a whitepaper that would stand up in court?
- dnomad 8y ago
  The problem is that deanonymization [1] is a thing. It's quite likely that that the Microsoft dataset is itself is anonymous but when it's correlated with other datasets it becomes possible to surveil people. The danger here is that governments are happy to let businesses collect tons of anonymous data and then, when the moment is right, swoop in and correlate this data. This happens so often with stuff like IP addresses that they have to be considered personal data even though in theory the same IP address can be recycled. [1] https://en.wikipedia.org/wiki/De-anonymization https://en.wikipedia.org/wiki/De-anonymization
- Cakez0r 8y ago
  I went down the rabbit hole on this one a little. They're sending a SHA256 hash of the user's mac address with every telemetry event (which can be trivially reversed. I leave it to the reader to decide whether MAC addresses are personal identifiers). And also for what it's worth, it appears that they are using this ID to correlate what users are doing across their products - as indicated by this comment: "// The hashed mac address needs to be the same hashed value as produced by the other distinct sources given the same input. (e.g. VsCode)" (https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1c6bddf20167f166/src/dotnet/Telemetry/Sha256Hasher.cs#L12 https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1...)
  - setquk 8y ago
    That's a blatant violation then.
thierryzoller 8y ago
>but GUIDs are fine In general terms, I think you are mistaken by that - as long as information, including pseudonymous, can be traced back directly or indirectly to an individual it is to be considered personal data.
ubernostrum 8y ago
There seems to have been some sort of recent push at Microsoft on this. A little while back, an update to Office for Mac caused each application to pop a dialog on startup to share "diagnostic data" with Microsoft. The only options were "Full" and "Basic" data sharing. The buttons in the dialog were "Accept" (once you'd clicked an option) and "Learn More". No "Don't Share Data" option. Many users complained in the support forums about this, saying they wanted a way to completely opt out. Worse: if you closed the dialog without selecting any option, you were automatically "opted in" to "Full" data sharing (as you could see if you then went into settings and looked at the privacy tab). This week they rolled out another update. Which sneakily looks like it fixed this; now the dialog has buttons "No" and "Yes", but if you read it the question it's asking is "Can Office send enhanced error reporting?" There still is not a "do not share data" option.
- detaro 8y ago
  I'm surprised they haven't gotten more flak from large companies and governments about their telemetry.
MaxBarraclough 8y ago
This stuff is Free and Open Source, right? And the phone-home stuff is done with ordinary network connections, right? So the telemetry could be stopped either using firewall rules, or by modifying the code. Edit looks like it can be disabled with the "DOTNET_CLI_TELEMETRY_OPTOUT" environmental variable, following https://github.com/dotnet/cli/issues/3093#issuecomment-229976758 https://github.com/dotnet/cli/issues/3093#issuecomment-22997...

by dnomad 8y ago

manigandham 8y ago

Like all things, there must be a sense of context and proportionality. This issue is specifically about the .NET Core SDK command line tools. Getting data about which commands are used most, if they're successful or throwing errors, possible typos, unclear command flags, etc, are useful when shipping software frameworks used by 10s of millions to build products that run the entire Fortune 500 and thousands of other businesses. This is not ubiquitous spyware, and is about as generic as it gets while still remaining useful. Every software company (many of which are featured here on HN) does the same analytical tracking to figure out how their products are being used and it results in better features. What exactly is the danger here, especially with the lack of personal info and an opt-out method? They even make the data public along with insights into how it helped: https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-weve-learned-from-net-core-sdk-telemetry/ https://blogs.msdn.microsoft.com/dotnet/2017/07/21/what-weve...

0x0 8y ago
Collecting command line arguments is a HUGE invasion of privacy. Imagine I ran a command "make.exe my-secret-project-name" or "make.exe process_gdrp_removal_request SOME_SSN". And as the linked page admits, they really do collect all command line arguments, not just a white list of valid commands. Perhaps someone accidentally pastes an AWS API Secret into the wrong terminal window and it ended up as "dotnet.exe MY_AWS_SECRET". ("You will notice misspellings, like “bulid”. That’s what the user typed. It’s information").
- philliphaydon 8y ago
  They don't collect command line arguments according to the github issue.
  - 0x0 8y ago
    They do collect at least the first argument, how else do they end up with "bulid"? Accidents happen and someone could paste or tab-complete a confidential thing in that spot...
    - dingo_bat 8y ago
      So if you push your api key to github by accident is github responsible?
      - 0x0 8y ago
        Invalid comparison, in my humble opinion. Pushing a secret API key to github, even accidentally, does not happen covertly or as an unexpected side effect of some other tool written by github, and you'll soon realize what happens because you'll see your key sitting there. You also have a general idea of what you are playing with when you start using github, whether it's private repos or public repos. Having a local CLI tool submit your secret API key as part of telemetry collecting command line arguments is totally unexpected and invisible for the user, and you won't even see it has happened.
    - philliphaydon 8y ago
      I Guess they whitelist it. If someone types a valid command send the command. If someone types an invalid command then it prob sends something that isn’t descriptive. (Assumptions I’m making)
  - opencl 8y ago
    According to this PR they do collect command line arguments. https://github.com/dotnet/cli/pull/2145 https://github.com/dotnet/cli/pull/2145 "My understanding is that this was recently discussed with our privacy team and we concluded that collecting the arguments themselves (hashed or not) is not acceptable per our privacy policies. Not sure whether the code already reflects that, but it's being worked on." Somebody in the thread for the issue said it's "not acceptable per our privacy policies" and "being worked on" two years ago, but did they ever actually stop collecting command line arguments?
setquk 8y ago
Does this justify hashing user identification for later correlation with other products? https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1c6bddf20167f166/src/dotnet/Telemetry/Sha256Hasher.cs#L12 https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1...
ptx 8y ago
> Like all things, there must be a sense of context and proportionality. This issue is specifically about the .NET Core SDK command line tools. Exactly. Command line tools don't usually collect your data and send it off somewhere without asking. You wouldn't expect ls, cat and grep to spy on you. That's why it's so surprising. But even big interactive applications like IDEs don't normally do this without asking. IntelliJ had a dialog pop up just a few days ago to ask me nicely to participate, defaulting to non-participation (!). Visual Studio and Office used to do the same thing, as I recall. (Additionally, once you figure out how to opt-out in .NET Core, does it tell you somehow that you did so successfully? I think I've set the environment variable correctly, but how can I be sure?)
- josteink 8y ago
  > Exactly. Command line tools don't usually collect your data and send it off somewhere without asking. You wouldn't expect ls, cat and grep to spy on you. That's why it's so surprising. Except in Ubuntu Linux where apt-get it has done so for ages without anyone complaining: https://askubuntu.com/questions/57808/what-is-the-popularity-contest-package-for#57811 https://askubuntu.com/questions/57808/what-is-the-popularity...
  - ptx 8y ago
    Has it? It says right on the page you're linking to that it's disabled by default, so you would have to explicitly opt in. Thus nobody complaining, as those who don't want it won't enable it. I think it comes from Debian[1] actually, where it's also opt-in – my Debian machine doesn't have it installed. Like it says in the Debian package description: "Vote for your favourite packages automatically". It's voting, not spying – strictly voluntary. Those of us who don't want to vote don't have to. [1] https://popcon.debian.org/ https://popcon.debian.org/
    - josteink 8y ago
      > it's disabled by default Fair enough, but it wasn’t always like that. > “Vote for your favourite packages automatically". It's voting, not spying That’s just playing with words. What it does is exactly the same: provide the upstream author statistics about what is being used and not. Surely you can’t mean that this is a serious argument.
OnlyRepliesToBS 8y ago
~"you're just overreacting" ~"good for the user" ~"just the tip" ~"look how honest they were" Absolute bullshit. It was like pulling teeth just trying to get that information from Microsoft and that's not an order-of-magnitude all of it.

0x0 8y ago

Java has been pushing "sponsored offers" (i.e. unwanted spyware like browser toolbars and similar user-hostile software) in their installers for years, so they're not exactly a shining beacon of light.

setquk 8y ago
Yes Java has been a perpetural problem, at least the official JRE and JDK. The JRE ships with junk and the JDK forces you to forge an agreement cookie to download it. OpenJDK is fine however.
ionised 8y ago
I haven't had a problem with that for a while. The installers I've been getting directly from the Oracle JDK pages haven't installed anything extra, or asked to.
- viraptor 8y ago
  Are you sure? It's been there since 2013 (https://www.computerworld.com/article/2494794/malware-vulnerabilities/oracle-will-continue-to-bundle--crapware--with-java.html https://www.computerworld.com/article/2494794/malware-vulner...) I don't think there's been any change since. Java 8 still had a page about disabling the offers: https://www.java.com/en/download/faq/disable_offers.xml https://www.java.com/en/download/faq/disable_offers.xml
  - 0x0 8y ago
    The option to suppress sponsor offers is still there in the control panel for Java 10.0.1. It's of course possible that some users don't experience these "sponsored offers" due to geotargeting, A/B-testing, lack of available "offers" at a specific time, or any other reason controlled by the server side of the installer.

pjc50 8y ago

If it is genuinely anonymous, then it's far less of a problem and not actually spying on individual users.

worldsayshi 8y ago
Yeah, I kind of agree here. If it is possible to track down a users identity in a systematic way when using these methods it shouldn't be too hard to prove that in a whitepaper that would stand up in court?
dnomad 8y ago
The problem is that deanonymization [1] is a thing. It's quite likely that that the Microsoft dataset is itself is anonymous but when it's correlated with other datasets it becomes possible to surveil people. The danger here is that governments are happy to let businesses collect tons of anonymous data and then, when the moment is right, swoop in and correlate this data. This happens so often with stuff like IP addresses that they have to be considered personal data even though in theory the same IP address can be recycled. [1] https://en.wikipedia.org/wiki/De-anonymization https://en.wikipedia.org/wiki/De-anonymization
Cakez0r 8y ago
I went down the rabbit hole on this one a little. They're sending a SHA256 hash of the user's mac address with every telemetry event (which can be trivially reversed. I leave it to the reader to decide whether MAC addresses are personal identifiers). And also for what it's worth, it appears that they are using this ID to correlate what users are doing across their products - as indicated by this comment: "// The hashed mac address needs to be the same hashed value as produced by the other distinct sources given the same input. (e.g. VsCode)" (https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1c6bddf20167f166/src/dotnet/Telemetry/Sha256Hasher.cs#L12 https://github.com/dotnet/cli/blob/b45f1fb439b36872c249b07f1...)
- setquk 8y ago
  That's a blatant violation then.

thierryzoller 8y ago

>but GUIDs are fine In general terms, I think you are mistaken by that - as long as information, including pseudonymous, can be traced back directly or indirectly to an individual it is to be considered personal data.

ubernostrum 8y ago

There seems to have been some sort of recent push at Microsoft on this. A little while back, an update to Office for Mac caused each application to pop a dialog on startup to share "diagnostic data" with Microsoft. The only options were "Full" and "Basic" data sharing. The buttons in the dialog were "Accept" (once you'd clicked an option) and "Learn More". No "Don't Share Data" option. Many users complained in the support forums about this, saying they wanted a way to completely opt out. Worse: if you closed the dialog without selecting any option, you were automatically "opted in" to "Full" data sharing (as you could see if you then went into settings and looked at the privacy tab). This week they rolled out another update. Which sneakily looks like it fixed this; now the dialog has buttons "No" and "Yes", but if you read it the question it's asking is "Can Office send enhanced error reporting?" There still is not a "do not share data" option.

detaro 8y ago
I'm surprised they haven't gotten more flak from large companies and governments about their telemetry.

MaxBarraclough 8y ago

This stuff is Free and Open Source, right? And the phone-home stuff is done with ordinary network connections, right? So the telemetry could be stopped either using firewall rules, or by modifying the code. Edit looks like it can be disabled with the "DOTNET_CLI_TELEMETRY_OPTOUT" environmental variable, following https://github.com/dotnet/cli/issues/3093#issuecomment-229976758 https://github.com/dotnet/cli/issues/3093#issuecomment-22997...