5 ms·

Anyone here want to speak up for opt-out telemetry to help understand product usage patterns? Surely "opt-in" destroys any validity the data would have (self s

Anyone here want to speak up for opt-out telemetry to help understand product usage patterns? Surely "opt-in" destroys any validity the data would have (self selected groups are usually a poor proxy for typical users) even aside from the massive drop-off in participation.

maweki 5y ago
Considering the people who would give out their zip-code in the supermarket at checkout, I would hazard the guess that the less technical crowd would opt in and that their feedback or telemetry on what makes a simple UI is more valuable.
- ziml77 5y ago
  But in the case of a checkout where you give info letting them track your purchases, you get discounts in return. Audacity isn't offering anything in return for opting in to telemetry.
  - svrtknst 5y ago
    What you're getting in return is improved UX and performance, albeit it is a more abstract and collective gain than discounts. You also get (unless you are a contributor, I guess) a very capable and transparent music editing software for free.
  - maweki 5y ago
    In Germany I have never been offered a discount in that manner. Coupons are the only way to do this here.
    - majewsky 5y ago
      I'm also German. Once, when I was asked for my zip code at checkout, I asked what they use it for. The cashier said it's being used to decide where to distribute physical advertisements (those store-branded magazines that land in your mailbox every week which list all the items that are on sale next week). I can't know how much truth there is to it, so take it with a grain of salt.
- hyperman1 5y ago
  One of our local supermarkets could not check out until we gave them one, so we found some to give them. For Belgium: https://www.bpost2.be/zipcodes/files/zipcodes_num_nl_new.html https://www.bpost2.be/zipcodes/files/zipcodes_num_nl_new.htm... Fun ones are 0612 (Saint Nicholas) or 1110 (NATO). We got great mileage out of 3 suisses until they went bankrupt. Further random fact: The state and the postal services in Belgium don't always agree on the zip code. We found 1 location in Ghent where a monastery used to be, but it had been torn down and houses were built. The first inherited the address of the monastery, and hence had - on its own - a whole postal code, no street and no house number. But only for the postal services, not for the state (or the other way around, I dont remember). This location actually crashed the server for one geolocation provider. Just to say, we are hackers, we're good at these games. They want a zip code? Find one for your country and feed it to them.
  - latexr 5y ago
    I remember a situation when after checkout an employee asked if I’d provide my zip code for statistical purposes. I politely declined and she froze, as if her brain didn’t know how to handle the exception.
worble 5y ago
> Surely "opt-in" destroys any validity the data would have (self selected groups are usually a poor proxy for typical users) Is there actually any data to back this up? Personally I see no reason why an opted-in group would be using an application that differently from the masses - your hot paths are still hot paths, common errors and annoyances will still be triggered. Really the only major thing I see that could be different is that a certain hard to find feature might be used more than it would otherwise. But disregarding that, the argument is not "opt-in" vs "opt-out". Opt-out is not be on the table at all - silently collecting information from users without their consent or any affirmative action on their part is immoral, and damages user trust. So when you take that into account, the options are opt-in data vs no data whatsoever, and opt-in, taking into account any potential bias you think this data might have, is still more infinitely useful then getting nothing whatsoever.
- andybak 5y ago
  Putting discussions of deanonymization to one side for a second - how is collecting aggregate usage patterns "immoral"? If your issue is about the potential for deanonymization then that's understandable but that's a discussion where we would need to talk about specifics.
  - inetknght 5y ago
    > how is collecting aggregate usage patterns "immoral"? What I do alone in my own home should be known only to me. Do I read books? It doesn't matter if I do nor what books. Do I listen to music? It doesn't matter if I do nor what songs. Do I watch tv? It doesn't matter if I do nor what videos. Do I sleep? It doesn't matter if I do nor for how long. Do I play games? It doesn't matter if I do nor which games. Do I use a computer? It doesn't matter if I do nor what software. What I do in my own home is private. Anyone looking at that without my explicit consent is violating my privacy. Usage patterns of what I do in my own home are immoral whether in aggregate or not.
    - andybak 5y ago
      Nobody is talking about what books you read. We're talking about "What percentage of users clicked this button or used this audio filter?".
542458 5y ago
I don’t think anybody is aiming for statistical-research-publication level rigour on the data here. Yeah, it’s not an ideal proxy for genpop, but it’s far better than nothing. The goal here isn’t to be able to say 1.454% of users try feature X, rather it’s to be able to say “only a few users have tried feature X, is it correctly explained in the UI?”.
johannes1234321 5y ago
I agree that it is bad for feature decisions (a filter rarely used can't be removed, maybe the few times it is being used, it is important) However telemetry reporting crashes with context information can be useful to identify bugs. And the combination can be useful "each time a filter is being used it crashes" vs "the filter is used often, but crashes rarely"
Narishma 5y ago
Wouldn't "opt-out" just self select in the other direction?

maweki 5y ago

Considering the people who would give out their zip-code in the supermarket at checkout, I would hazard the guess that the less technical crowd would opt in and that their feedback or telemetry on what makes a simple UI is more valuable.

ziml77 5y ago
But in the case of a checkout where you give info letting them track your purchases, you get discounts in return. Audacity isn't offering anything in return for opting in to telemetry.
- svrtknst 5y ago
  What you're getting in return is improved UX and performance, albeit it is a more abstract and collective gain than discounts. You also get (unless you are a contributor, I guess) a very capable and transparent music editing software for free.
- maweki 5y ago
  In Germany I have never been offered a discount in that manner. Coupons are the only way to do this here.
  - majewsky 5y ago
    I'm also German. Once, when I was asked for my zip code at checkout, I asked what they use it for. The cashier said it's being used to decide where to distribute physical advertisements (those store-branded magazines that land in your mailbox every week which list all the items that are on sale next week). I can't know how much truth there is to it, so take it with a grain of salt.
hyperman1 5y ago
One of our local supermarkets could not check out until we gave them one, so we found some to give them. For Belgium: https://www.bpost2.be/zipcodes/files/zipcodes_num_nl_new.html https://www.bpost2.be/zipcodes/files/zipcodes_num_nl_new.htm... Fun ones are 0612 (Saint Nicholas) or 1110 (NATO). We got great mileage out of 3 suisses until they went bankrupt. Further random fact: The state and the postal services in Belgium don't always agree on the zip code. We found 1 location in Ghent where a monastery used to be, but it had been torn down and houses were built. The first inherited the address of the monastery, and hence had - on its own - a whole postal code, no street and no house number. But only for the postal services, not for the state (or the other way around, I dont remember). This location actually crashed the server for one geolocation provider. Just to say, we are hackers, we're good at these games. They want a zip code? Find one for your country and feed it to them.
- latexr 5y ago
  I remember a situation when after checkout an employee asked if I’d provide my zip code for statistical purposes. I politely declined and she froze, as if her brain didn’t know how to handle the exception.

worble 5y ago

> Surely "opt-in" destroys any validity the data would have (self selected groups are usually a poor proxy for typical users) Is there actually any data to back this up? Personally I see no reason why an opted-in group would be using an application that differently from the masses - your hot paths are still hot paths, common errors and annoyances will still be triggered. Really the only major thing I see that could be different is that a certain hard to find feature might be used more than it would otherwise. But disregarding that, the argument is not "opt-in" vs "opt-out". Opt-out is not be on the table at all - silently collecting information from users without their consent or any affirmative action on their part is immoral, and damages user trust. So when you take that into account, the options are opt-in data vs no data whatsoever, and opt-in, taking into account any potential bias you think this data might have, is still more infinitely useful then getting nothing whatsoever.

andybak 5y ago
Putting discussions of deanonymization to one side for a second - how is collecting aggregate usage patterns "immoral"? If your issue is about the potential for deanonymization then that's understandable but that's a discussion where we would need to talk about specifics.
- inetknght 5y ago
  > how is collecting aggregate usage patterns "immoral"? What I do alone in my own home should be known only to me. Do I read books? It doesn't matter if I do nor what books. Do I listen to music? It doesn't matter if I do nor what songs. Do I watch tv? It doesn't matter if I do nor what videos. Do I sleep? It doesn't matter if I do nor for how long. Do I play games? It doesn't matter if I do nor which games. Do I use a computer? It doesn't matter if I do nor what software. What I do in my own home is private. Anyone looking at that without my explicit consent is violating my privacy. Usage patterns of what I do in my own home are immoral whether in aggregate or not.
  - andybak 5y ago
    Nobody is talking about what books you read. We're talking about "What percentage of users clicked this button or used this audio filter?".

542458 5y ago

I don’t think anybody is aiming for statistical-research-publication level rigour on the data here. Yeah, it’s not an ideal proxy for genpop, but it’s far better than nothing. The goal here isn’t to be able to say 1.454% of users try feature X, rather it’s to be able to say “only a few users have tried feature X, is it correctly explained in the UI?”.

johannes1234321 5y ago

I agree that it is bad for feature decisions (a filter rarely used can't be removed, maybe the few times it is being used, it is important) However telemetry reporting crashes with context information can be useful to identify bugs. And the combination can be useful "each time a filter is being used it crashes" vs "the filter is used often, but crashes rarely"

Narishma 5y ago

Wouldn't "opt-out" just self select in the other direction?