3 ms·

My projects could genuinely benefit from telemetry as I have no idea about usage patterns and my community (mainly artists) is not famous for maintaining a clos

by andybak 5mo ago

My projects could genuinely benefit from telemetry as I have no idea about usage patterns and my community (mainly artists) is not famous for maintaining a close dialogue with software developers. I haven't bothered because a) opt-out risks a backlash and b) opt-in affects the data so much it becomes useless (much smaller sample and probably self-selecting a certain type of user) Skimming the comments here, it seems everybody assumes telemetry is always nefarious. I get the distrust of large corporations and other obvious bad actors - but the blanket cynicism for all telemetry here is kinda surprising. Have none of the developers here ever had a need for it themselves?

deleted 5mo ago
[deleted]
dahart 5mo ago
I’m sympathetic to both the default distrust and to devs like you who want telemetry to improve their software and won’t use the data for anything else, but it is because of bad actors and enough dark ad patterns that we just can’t trust companies to play nice, and it’s too difficult to expect people to scrutinize each and every app or site individually. So I get why the default assumption is nefarious behavior. But you’re totally right - telemetry & crash dumps & analytics are helpful & great for devs who care about the customer UX and don’t use the data for advertising or anything other than fixing & writing good software, so it’s a real kind of tragedy of the commons that we can’t have safe, trustworthy, and pro-consumer telemetry. I went from building a web app that used Google Analytics and some other kinds of anonymous telemetry (and using that data only for identifying functional software & site issues), to building driver software that absolutely cannot send data out, and I wish for telemetry all the time. Not only is it difficult to understand what users are doing, they usually don’t even know themselves and can’t tell me what happened when things crash. The result is that turnaround times for critical issues are in months, when it could be days or hours if we had crash dumps and analytics, the lack of automated reporting hurts users. I’m not sure there’s a way to separate the good from the bad, to designate some kinds of telemetry as safe and to be able to trust it while disallowing the stuff we don’t want. If that were somehow possible, if anyone has ideas, I would love to help figure out how to make it a reality.
- kuschku 5mo ago
  The best way is to collect logs & crash reports locally, and if the app crashes you offer an option to send the report directly to you. That's what I do in my apps. And it turns out, that actually increased the quality of the bug reports I got, because users were more invested and willing to cooperate.
marshray 5mo ago
I feel that I have been a victim of "good telemetry" too, as when advanced product features were removed which were probably not popular but that I personally relied on.
deanishe 5mo ago
> I get the distrust of large corporations and other obvious bad actors […] > the blanket cynicism for all telemetry here is kinda surprising Who's providing the telemetry/analytics if not one of same large corporations? Many devs say they care about user privacy, but very few seem to care enough not to farm surveillance out to a 3rd-party they have no control over.
- andybak 5mo ago
  > Who's providing the telemetry/analytics if not one of same large corporations? Erm. It would be me? The idea was that the app (not a web app btw) would send back data about which features were being used (to a server I control) so I could build up a picture of how often various features were being used relative to other features. Nothing remotely personally identifiable.
ragall 5mo ago
Telemetry only tells you what users do, not why and doesn't explain their mental models. Try asking directly: open a discussion board (for example Github's Discussions) and encourage them to post about aspects of the software they found puzzling/annoying/inefficient. Take 15 minutes a week to go through the posts to see if anything attracts your attention.
- theshrike79 5mo ago
  Normal users don't register on a discussion board to tell about what went well during a normal day. People only bother when something has made them really angry about something and need to vent. This is why default analytics is the correct option. It gets the average people who don't care about forums and usually won't even bother to change many of the settings. The crowd who doesn't open HN first thing in the morning.
1vuio0pswjnm7 5mo ago
Opt-in data is "useless" That's one I have not heard before Useless for what Targeting a certain "type of user" perhaps "I get the distrust of large corporations and other obvious bad actors - but the blanket cycnicism for all telemetry here is kind surprising" There is effectively no way for a user to determine whether an actor is "bad" or "good" and that definition may vary depending on the user The user cannot verify how the data might be used or where it might be transferred. As such, there is almost zero incentive for the data collector not to engage in malfeasance (as the user defines that term); deterrents are lacking Perhaps there is irony in criticising "blanket" cynicism whilst arguing for "default" telemetry. Both suffer from the same "one size fits all" error
- 1vuio0pswjnm7 5mo ago
  Begs the question: Why isn't opt-out "useless" Perhaps because only way to get large sample size is to target users who are unaware of "defaults", i.e., remove choice Perhaps when forced to make a choice ("opt-in"), users will not choose to share data (unless the developer uses dark patterns to manipulate the choice) Why is that
- LordDragonfang 5mo ago
  > That's one I have not heard before No offense, but if that's the case, you are very new to the discussion. It's been pretty well-documented that opt-out provides orders of magnitude more useful reports than opt-in. For the best example: Factorio, a game with an almost-exclusively-technical playerbase and extremely well-regarded and community-friendly dev team, which already had a ton of people writing good bug reports on the forums, [fixed 12 crash-causing bugs](https://factorio.com/blog/post/fff-231 https://factorio.com/blog/post/fff-231) within two days after making crash reports automatic and opt-in. And if it has that much impact for Factorio, you can imagine how much bigger the impact is for non-technical software.
- 1vuio0pswjnm7 5mo ago
  "Another idea was to have a prompt asking if you wish to upload the log or not, after every crash. Apart from the extra implementation time, players will still often click "Don't send", either because that's what they have been doing for many years or because they just want to quickly get back in the game or because they feel that the crash was somehow their fault." https://factorio.com/blog/post/fff-231 https://factorio.com/blog/post/fff-231 Here, the developer is trying to infer user reasoning and intent Curiously, he omits the possibility that users would prefer not to send the data When in fact this is exactly what users indicate they prefer He pretends that "Don't send" is ambiguous, for example, that despite clicking "Don't send" users actually don't care if data is sent But there is nothing here that indicates users wanted to send data or that they do not care Software developers can obviously do whatever they want and they can act against the interests of users This includes ignoring or explaining away the preferences of their users ("Don't send") and engaging in speculation about user reasoning and intent The developer here seems dismissive of users' reasons for clicking "Don't send", even though he does not know the reasons and can only speculate. At the same time he expects readers to take his reasons for collecting crash logs as justified. Then he unilaterally decides to remove user choice (the "Don't send" button) and substitute his own choice (send data) as a default Perhaps lack of developer pre-release testing and quality control is relevant to this discussion. Alas, the problem is framed as one of data collection and user consent where the "solution" is making data collection surreptitious and making "consent" uninformed, implied

by andybak 5mo ago

deleted 5mo ago

[deleted]

dahart 5mo ago

I’m sympathetic to both the default distrust and to devs like you who want telemetry to improve their software and won’t use the data for anything else, but it is because of bad actors and enough dark ad patterns that we just can’t trust companies to play nice, and it’s too difficult to expect people to scrutinize each and every app or site individually. So I get why the default assumption is nefarious behavior. But you’re totally right - telemetry & crash dumps & analytics are helpful & great for devs who care about the customer UX and don’t use the data for advertising or anything other than fixing & writing good software, so it’s a real kind of tragedy of the commons that we can’t have safe, trustworthy, and pro-consumer telemetry. I went from building a web app that used Google Analytics and some other kinds of anonymous telemetry (and using that data only for identifying functional software & site issues), to building driver software that absolutely cannot send data out, and I wish for telemetry all the time. Not only is it difficult to understand what users are doing, they usually don’t even know themselves and can’t tell me what happened when things crash. The result is that turnaround times for critical issues are in months, when it could be days or hours if we had crash dumps and analytics, the lack of automated reporting hurts users. I’m not sure there’s a way to separate the good from the bad, to designate some kinds of telemetry as safe and to be able to trust it while disallowing the stuff we don’t want. If that were somehow possible, if anyone has ideas, I would love to help figure out how to make it a reality.

kuschku 5mo ago
The best way is to collect logs & crash reports locally, and if the app crashes you offer an option to send the report directly to you. That's what I do in my apps. And it turns out, that actually increased the quality of the bug reports I got, because users were more invested and willing to cooperate.

marshray 5mo ago

I feel that I have been a victim of "good telemetry" too, as when advanced product features were removed which were probably not popular but that I personally relied on.

deanishe 5mo ago

> I get the distrust of large corporations and other obvious bad actors […] > the blanket cynicism for all telemetry here is kinda surprising Who's providing the telemetry/analytics if not one of same large corporations? Many devs say they care about user privacy, but very few seem to care enough not to farm surveillance out to a 3rd-party they have no control over.

andybak 5mo ago
> Who's providing the telemetry/analytics if not one of same large corporations? Erm. It would be me? The idea was that the app (not a web app btw) would send back data about which features were being used (to a server I control) so I could build up a picture of how often various features were being used relative to other features. Nothing remotely personally identifiable.

ragall 5mo ago

Telemetry only tells you what users do, not why and doesn't explain their mental models. Try asking directly: open a discussion board (for example Github's Discussions) and encourage them to post about aspects of the software they found puzzling/annoying/inefficient. Take 15 minutes a week to go through the posts to see if anything attracts your attention.

theshrike79 5mo ago
Normal users don't register on a discussion board to tell about what went well during a normal day. People only bother when something has made them really angry about something and need to vent. This is why default analytics is the correct option. It gets the average people who don't care about forums and usually won't even bother to change many of the settings. The crowd who doesn't open HN first thing in the morning.

1vuio0pswjnm7 5mo ago

Opt-in data is "useless" That's one I have not heard before Useless for what Targeting a certain "type of user" perhaps "I get the distrust of large corporations and other obvious bad actors - but the blanket cycnicism for all telemetry here is kind surprising" There is effectively no way for a user to determine whether an actor is "bad" or "good" and that definition may vary depending on the user The user cannot verify how the data might be used or where it might be transferred. As such, there is almost zero incentive for the data collector not to engage in malfeasance (as the user defines that term); deterrents are lacking Perhaps there is irony in criticising "blanket" cynicism whilst arguing for "default" telemetry. Both suffer from the same "one size fits all" error

1vuio0pswjnm7 5mo ago
Begs the question: Why isn't opt-out "useless" Perhaps because only way to get large sample size is to target users who are unaware of "defaults", i.e., remove choice Perhaps when forced to make a choice ("opt-in"), users will not choose to share data (unless the developer uses dark patterns to manipulate the choice) Why is that
LordDragonfang 5mo ago
> That's one I have not heard before No offense, but if that's the case, you are very new to the discussion. It's been pretty well-documented that opt-out provides orders of magnitude more useful reports than opt-in. For the best example: Factorio, a game with an almost-exclusively-technical playerbase and extremely well-regarded and community-friendly dev team, which already had a ton of people writing good bug reports on the forums, [fixed 12 crash-causing bugs](https://factorio.com/blog/post/fff-231 https://factorio.com/blog/post/fff-231) within two days after making crash reports automatic and opt-in. And if it has that much impact for Factorio, you can imagine how much bigger the impact is for non-technical software.
1vuio0pswjnm7 5mo ago
"Another idea was to have a prompt asking if you wish to upload the log or not, after every crash. Apart from the extra implementation time, players will still often click "Don't send", either because that's what they have been doing for many years or because they just want to quickly get back in the game or because they feel that the crash was somehow their fault." https://factorio.com/blog/post/fff-231 https://factorio.com/blog/post/fff-231 Here, the developer is trying to infer user reasoning and intent Curiously, he omits the possibility that users would prefer not to send the data When in fact this is exactly what users indicate they prefer He pretends that "Don't send" is ambiguous, for example, that despite clicking "Don't send" users actually don't care if data is sent But there is nothing here that indicates users wanted to send data or that they do not care Software developers can obviously do whatever they want and they can act against the interests of users This includes ignoring or explaining away the preferences of their users ("Don't send") and engaging in speculation about user reasoning and intent The developer here seems dismissive of users' reasons for clicking "Don't send", even though he does not know the reasons and can only speculate. At the same time he expects readers to take his reasons for collecting crash logs as justified. Then he unilaterally decides to remove user choice (the "Don't send" button) and substitute his own choice (send data) as a default Perhaps lack of developer pre-release testing and quality control is relevant to this discussion. Alas, the problem is framed as one of data collection and user consent where the "solution" is making data collection surreptitious and making "consent" uninformed, implied