7 ms·

I wish we could add telemetry to the rr debugger without risking this sort of blowback. We have no idea how many people are using rr, and how much, so when we t

by roca 5y ago

I wish we could add telemetry to the rr debugger without risking this sort of blowback. We have no idea how many people are using rr, and how much, so when we talk to hardware vendors and other groups we depend on, we have no leverage. This is a real problem. It would be a big boost to free software if some org like Freedesktop had a standard telemetry library, data collection policy, data collection and publication service, and a distro-wide master switch to opt into data collection, and it was socially acceptable to use that framework.

Animats 5y ago
You can check downloads. No, we do not need a "standard telemetry library". A standard crash reporter might be OK. It should ask the user if you want to submit a crash report, and give you a ticket number which allows you to visit a web site and see what's been done about it.
- jeffgreco 5y ago
  Downloads are an incredibly non-useful statistic. Appreciate your disinterest in providing information but it’s disingenuous to say this is a worthwhile piece of data.
- roca 5y ago
  You can get rr in many ways: distro packages, Github release downloads, cloning the repo and building it yourself. It's difficult to track all that. Also it matters how much people use rr. If a lot of people download it, use it once and never use it again that's important information (especially if you also report when it worked and when it didn't work).
manquer 5y ago
You could make it optional ? Mozilla does it . Nobody is forking Firefox over just that, I don't think everyone is against it. If it opt out and part of the install process and uses know what kind of data will be used it should be fine. Audacity seems a exception here. They are broadly following this kind of rules , however there is blowback which other projects don't seem to be getting. Perhaps there are underlying issues in the community ?
- cyral 5y ago
  It even seems Audacity is asking for permission based on this screenshot: https://github.com/audacity/audacity/pull/835 https://github.com/audacity/audacity/pull/835
  - monetus 5y ago
    That is the opt out for crash reporting, the version checking doesn't show a dialog before it phones home. Version checking is configurable in the settings, but it will run once I guess.
  - btown 5y ago
    As far as I can tell (and correct me if I'm wrong) while other open-source projects might be able to get away with opt-in telemetry, Muse Group has simultaneously (a) raised the ire of other open-source developers with the bungled response to e.g. https://news.ycombinator.com/item?id=27740550; https://news.ycombinator.com/item?id=27740550; (b) had technically correct but really optically horrid legal language about potential uses of the Audacity data, which implied a blanket granting of any metadata to any buyer of the company to use as they see fit, even if GDPR wouldn't let them do that; (c) didn't make it clear from the get-go that telemetry would be opt-in, nor make any sort of foundational commitment that it would only ever be opt-in; and (d) hasn't yet reacted quickly to describe their ownership structure in the interest of transparency. It's a perfect storm of mistrust and crisis mismanagement. And the sad thing is that there are talented folks like Tantacrul, Head of Design at Muse Group, who could really benefit the community with opt-in telemetry guiding their product decisions, who have now been thrust into a crisis management role by this botched rollout, and are unprepared to give the transparency now required. Did nobody else at the company learn from WhatsApp's debacle on this? It's especially sad because while few would shed a tear for WhatsApp's missteps, Muse Group's holdings in MuseScore and Audacity are literally the lifeblood of hobbyist and student music creators. Mistrust in that software can easily lead to "you know what, I heard bad things about Muse stuff, I won't pick it up" and that would rob the world of so, so much creativity. Muse Group, rein in your lawyers, swallow your pride, and act like the stewards of the future of music that you are.
X6S1x6Okd1st 5y ago
Here's an example of a smaller project that added telemetry without suffering a fork: https://github.com/com-lihaoyi/Ammonite/issues/607 https://github.com/com-lihaoyi/Ammonite/issues/607
- IshKebab 5y ago
  It doesn't sound like they've done anything different. They're just small enough that nobody can be bothered and people haven't written poorly informed criticisms of their decisions all over Reddit and Hacker News.
zbrozek 5y ago
Maybe so, but as a user I have no incentive to want to allow it. And there's no checks or balances, since it's completely unilateral on the part of the software vendor. My experience has been that there's continuous creeping growth on what gets collected and stored for eternity. Any nonvolatile bit is a public bit. It's only a matter of time.
- jldugger 5y ago
  > Maybe so, but as a user I have no incentive to want to allow it. I mean, you kind of do -- to the extent that the data is used to prioritize fixing features you use, you benefit from being part of the survey sample.
  - zbrozek 5y ago
    That's roundabout and not compelling to a user. They have no idea what is gathered, how it's used, or any idea if their issues will be solved or when. It's like buying lottery tickets with payoff in bug fixes, except you don't know what they cost. Bad deal. The software house always wins.
swiley 5y ago
If you have online documentation a great measure of the popularity of your project might be unique IPs connecting to the server for it.
bsder 5y ago
I see "rr" as a little different because it is a developer tool. There is going to be a little more sensitivity about accidentally sending proprietary information. Audacity is an end application and really isn't likely to accidentally send something sensitive short of packing up entire audio files (which would be stupidly obvious if it were being done).
- jononor 5y ago
  Filenames/filepaths can be quite sensitive sometimes, and are occationally collected by telemetry (I do not know if it is in this case). For example, I know that Audacity is used for forensics, where that would be considered quite problematic.
- mixmastamyk 5y ago
  Except everything readable on my machine by my user.
ezyang 5y ago
It's especially bad for rr, since it doesn't otherwise have any reason to talk to the Internet (I see people mentioning Firefox telemetry, but you know, Firefox is a browser, you expect it to talk to the net). The best I can think of is to incentivize it other ways; e.g., telemetry only for bug reporting, or a "you ping us, we give you a nice hat" or something.
ta988 5y ago
Just ask the first time it is run: "Hello, we would like to know how many people are using our software. you can either click on that link using xyz analytics, or send us an email at xyx@xyz.com, thanks a lot for your support."
- schoen 5y ago
  This reminds me of the Linux Counter project. When you installed Slackware, you automatically had an e-mail from Patrick Volkerding or Harald Alvestrand asking you to e-mail Harald to explain how you used Linux ("I use Linux at home" / "I use Linux at work" / "I use Linux at school"). This personalized request always felt so warm and casual to me, and I appreciated the thought that Harald Alvestrand cared to know that I, personally, was using Linux.
- rendaw 5y ago
  And add that you need it for leverage with hardware vendors.
- roca 5y ago
  That might work well.
  - piaste 5y ago
    If there's no reason for the user to share the data, 99% won't, if only because it's the quickest option to get it out of the way. My suggestion would be to give a reason, however small, to send that email. For example, "Would you like to vote on which of these four issues from our roadmap you would like us to prioritize?" (best asked after a bit of usage, rather than on first installation).
wolverine876 5y ago
If you engineered a way to do it while maintaining user privacy, it would be a service to lots of FOSS developers. Edit: How about routing it via something people trust, such as a trusted proxy like Mozilla (if they would host such a thing) and over the Tor network (if that could be bundled easily).
elteto 5y ago
Just a couple of silly ideas, I’m no expert: 1. Collect the telemetry to a local file. The file must not be encrypted, be human-readable and contain only anonymized data. I should also be able to confirm all this by just reading the file. 2. Bug the user every time rr finishes or something like that. Ask them to send the telemetry by running a curl command that you will print out right there. 3. Have a sensible log collection policy on the telemetry servers and explain that too.
- warkdarrior 5y ago
  > Bug the user every time rr finishes or something like that. Oooh, we're redescovering nagware (https://en.wikipedia.org/wiki/Shareware#Nagware https://en.wikipedia.org/wiki/Shareware#Nagware ), but now for privacy instead of money!
ximm 5y ago
https://popcon.debian.org/ https://popcon.debian.org/ is an opt-in mechanism to measure the number of installs. Of course that is not the same as actual usage, but pretty close.
- acdha 5y ago
  > Of course that is not the same as actual usage, but pretty close. How would you know that? It seems quite plausible that there are statistically significant differences between the subset of users who enable it and everyone else.
mixmastamyk 5y ago
As seen, forcing or tricking others into fulfilling your desires is a violation of trust and rightly condemned. Best you could ask them to install a separate package to enable it.
tgsovlerkhgsel 5y ago
To minimize blowback, you need to show that you respect user's privacy: - Communicate clearly and discuss openly before writing code, asking for feedback from the community. - Make very clear from the beginning that this is opt-in (Audacity failed this, IMO) - Give specific examples of the actual benefits of telemetry in concrete terms (e.g. "leverage with hardware vendors"), not just "make $product better". - avoid dark patterns (the screenshot Audacity has added now to the pull request has a highlighted "yes" button - that already will make some people hate you, anything worse than that will rightfully make most privacy conscious people hate you) - Avoid controversial third party providers (this likely amplified the outrage in the Audacity case a lot) - If you screw up and/or get unlucky and a shitstorm starts, back down. Even if Audacity actually fixes the issues to a point where everyone would consider it reasonable had it been like that from the start, trust has been lost to the point where any telemetry feature will be controversial. Audacity screwed up in multiple ways: - poor communication in general - Still doesn't intend to make the data collection opt-in. (The update check sends OS and version information, and that part is supposed to be opt-out.) The privacy policy also makes clear that it wasn't supposed to be opt-in (uses legitimate interest instead of consent, tells people who are under 13 to not use the app). - introduced this shortly after other controversial changes (change of ownership, CLA requirement) - collecting data "For legal enforcement" - "If you are under 13 years old, please do not use the App." (following from COPPA limitations - it should be "please disable analytics" instead) - used controversial third party providers
aasasd 5y ago
IIRC Debian have attempted to track installations of packages, and likewise discovered that people don't like it. (Pretty much 100% predictable by now: the issue is hot, and complaining is easy—so it will be done.)
ddtaylor 5y ago
Add a `--enable-telemetry` option.
notyere 5y ago
Is this sarcasm? Really can't tell

by roca 5y ago

Animats 5y ago

You can check downloads. No, we do not need a "standard telemetry library". A standard crash reporter might be OK. It should ask the user if you want to submit a crash report, and give you a ticket number which allows you to visit a web site and see what's been done about it.

jeffgreco 5y ago
Downloads are an incredibly non-useful statistic. Appreciate your disinterest in providing information but it’s disingenuous to say this is a worthwhile piece of data.
roca 5y ago
You can get rr in many ways: distro packages, Github release downloads, cloning the repo and building it yourself. It's difficult to track all that. Also it matters how much people use rr. If a lot of people download it, use it once and never use it again that's important information (especially if you also report when it worked and when it didn't work).

manquer 5y ago

You could make it optional ? Mozilla does it . Nobody is forking Firefox over just that, I don't think everyone is against it. If it opt out and part of the install process and uses know what kind of data will be used it should be fine. Audacity seems a exception here. They are broadly following this kind of rules , however there is blowback which other projects don't seem to be getting. Perhaps there are underlying issues in the community ?

cyral 5y ago
It even seems Audacity is asking for permission based on this screenshot: https://github.com/audacity/audacity/pull/835 https://github.com/audacity/audacity/pull/835
- monetus 5y ago
  That is the opt out for crash reporting, the version checking doesn't show a dialog before it phones home. Version checking is configurable in the settings, but it will run once I guess.
- btown 5y ago
  As far as I can tell (and correct me if I'm wrong) while other open-source projects might be able to get away with opt-in telemetry, Muse Group has simultaneously (a) raised the ire of other open-source developers with the bungled response to e.g. https://news.ycombinator.com/item?id=27740550; https://news.ycombinator.com/item?id=27740550; (b) had technically correct but really optically horrid legal language about potential uses of the Audacity data, which implied a blanket granting of any metadata to any buyer of the company to use as they see fit, even if GDPR wouldn't let them do that; (c) didn't make it clear from the get-go that telemetry would be opt-in, nor make any sort of foundational commitment that it would only ever be opt-in; and (d) hasn't yet reacted quickly to describe their ownership structure in the interest of transparency. It's a perfect storm of mistrust and crisis mismanagement. And the sad thing is that there are talented folks like Tantacrul, Head of Design at Muse Group, who could really benefit the community with opt-in telemetry guiding their product decisions, who have now been thrust into a crisis management role by this botched rollout, and are unprepared to give the transparency now required. Did nobody else at the company learn from WhatsApp's debacle on this? It's especially sad because while few would shed a tear for WhatsApp's missteps, Muse Group's holdings in MuseScore and Audacity are literally the lifeblood of hobbyist and student music creators. Mistrust in that software can easily lead to "you know what, I heard bad things about Muse stuff, I won't pick it up" and that would rob the world of so, so much creativity. Muse Group, rein in your lawyers, swallow your pride, and act like the stewards of the future of music that you are.

X6S1x6Okd1st 5y ago

Here's an example of a smaller project that added telemetry without suffering a fork: https://github.com/com-lihaoyi/Ammonite/issues/607 https://github.com/com-lihaoyi/Ammonite/issues/607

IshKebab 5y ago
It doesn't sound like they've done anything different. They're just small enough that nobody can be bothered and people haven't written poorly informed criticisms of their decisions all over Reddit and Hacker News.

zbrozek 5y ago

Maybe so, but as a user I have no incentive to want to allow it. And there's no checks or balances, since it's completely unilateral on the part of the software vendor. My experience has been that there's continuous creeping growth on what gets collected and stored for eternity. Any nonvolatile bit is a public bit. It's only a matter of time.

jldugger 5y ago
> Maybe so, but as a user I have no incentive to want to allow it. I mean, you kind of do -- to the extent that the data is used to prioritize fixing features you use, you benefit from being part of the survey sample.
- zbrozek 5y ago
  That's roundabout and not compelling to a user. They have no idea what is gathered, how it's used, or any idea if their issues will be solved or when. It's like buying lottery tickets with payoff in bug fixes, except you don't know what they cost. Bad deal. The software house always wins.

swiley 5y ago

If you have online documentation a great measure of the popularity of your project might be unique IPs connecting to the server for it.

bsder 5y ago

I see "rr" as a little different because it is a developer tool. There is going to be a little more sensitivity about accidentally sending proprietary information. Audacity is an end application and really isn't likely to accidentally send something sensitive short of packing up entire audio files (which would be stupidly obvious if it were being done).

jononor 5y ago
Filenames/filepaths can be quite sensitive sometimes, and are occationally collected by telemetry (I do not know if it is in this case). For example, I know that Audacity is used for forensics, where that would be considered quite problematic.
mixmastamyk 5y ago
Except everything readable on my machine by my user.

ezyang 5y ago

It's especially bad for rr, since it doesn't otherwise have any reason to talk to the Internet (I see people mentioning Firefox telemetry, but you know, Firefox is a browser, you expect it to talk to the net). The best I can think of is to incentivize it other ways; e.g., telemetry only for bug reporting, or a "you ping us, we give you a nice hat" or something.

ta988 5y ago

Just ask the first time it is run: "Hello, we would like to know how many people are using our software. you can either click on that link using xyz analytics, or send us an email at xyx@xyz.com, thanks a lot for your support."

schoen 5y ago
This reminds me of the Linux Counter project. When you installed Slackware, you automatically had an e-mail from Patrick Volkerding or Harald Alvestrand asking you to e-mail Harald to explain how you used Linux ("I use Linux at home" / "I use Linux at work" / "I use Linux at school"). This personalized request always felt so warm and casual to me, and I appreciated the thought that Harald Alvestrand cared to know that I, personally, was using Linux.
rendaw 5y ago
And add that you need it for leverage with hardware vendors.
roca 5y ago
That might work well.
- piaste 5y ago
  If there's no reason for the user to share the data, 99% won't, if only because it's the quickest option to get it out of the way. My suggestion would be to give a reason, however small, to send that email. For example, "Would you like to vote on which of these four issues from our roadmap you would like us to prioritize?" (best asked after a bit of usage, rather than on first installation).

wolverine876 5y ago

If you engineered a way to do it while maintaining user privacy, it would be a service to lots of FOSS developers. Edit: How about routing it via something people trust, such as a trusted proxy like Mozilla (if they would host such a thing) and over the Tor network (if that could be bundled easily).

elteto 5y ago

Just a couple of silly ideas, I’m no expert: 1. Collect the telemetry to a local file. The file must not be encrypted, be human-readable and contain only anonymized data. I should also be able to confirm all this by just reading the file. 2. Bug the user every time rr finishes or something like that. Ask them to send the telemetry by running a curl command that you will print out right there. 3. Have a sensible log collection policy on the telemetry servers and explain that too.

warkdarrior 5y ago
> Bug the user every time rr finishes or something like that. Oooh, we're redescovering nagware (https://en.wikipedia.org/wiki/Shareware#Nagware https://en.wikipedia.org/wiki/Shareware#Nagware ), but now for privacy instead of money!

ximm 5y ago

https://popcon.debian.org/ https://popcon.debian.org/ is an opt-in mechanism to measure the number of installs. Of course that is not the same as actual usage, but pretty close.

acdha 5y ago
> Of course that is not the same as actual usage, but pretty close. How would you know that? It seems quite plausible that there are statistically significant differences between the subset of users who enable it and everyone else.

mixmastamyk 5y ago

As seen, forcing or tricking others into fulfilling your desires is a violation of trust and rightly condemned. Best you could ask them to install a separate package to enable it.

tgsovlerkhgsel 5y ago

To minimize blowback, you need to show that you respect user's privacy: - Communicate clearly and discuss openly before writing code, asking for feedback from the community. - Make very clear from the beginning that this is opt-in (Audacity failed this, IMO) - Give specific examples of the actual benefits of telemetry in concrete terms (e.g. "leverage with hardware vendors"), not just "make $product better". - avoid dark patterns (the screenshot Audacity has added now to the pull request has a highlighted "yes" button - that already will make some people hate you, anything worse than that will rightfully make most privacy conscious people hate you) - Avoid controversial third party providers (this likely amplified the outrage in the Audacity case a lot) - If you screw up and/or get unlucky and a shitstorm starts, back down. Even if Audacity actually fixes the issues to a point where everyone would consider it reasonable had it been like that from the start, trust has been lost to the point where any telemetry feature will be controversial. Audacity screwed up in multiple ways: - poor communication in general - Still doesn't intend to make the data collection opt-in. (The update check sends OS and version information, and that part is supposed to be opt-out.) The privacy policy also makes clear that it wasn't supposed to be opt-in (uses legitimate interest instead of consent, tells people who are under 13 to not use the app). - introduced this shortly after other controversial changes (change of ownership, CLA requirement) - collecting data "For legal enforcement" - "If you are under 13 years old, please do not use the App." (following from COPPA limitations - it should be "please disable analytics" instead) - used controversial third party providers

aasasd 5y ago

IIRC Debian have attempted to track installations of packages, and likewise discovered that people don't like it. (Pretty much 100% predictable by now: the issue is hot, and complaining is easy—so it will be done.)

ddtaylor 5y ago

Add a `--enable-telemetry` option.

notyere 5y ago

Is this sarcasm? Really can't tell