3 ms·

> Tracking down regressions, crashes, and perf issues without good telemetry about how often it's happening and in what context. If that's what you're aiming a

> Tracking down regressions, crashes, and perf issues without good telemetry about how often it's happening and in what context. If that's what you're aiming at. Collect the data but keep it local. Install some sort of responsiveness/"problem" monitoring. Ask the user to send data relevant to the problem if a problem occurs. IMHO there is no need to systematically collect user data for that. Or get the data from a random sample of users. You don't need data from everyone.

Vinnl 9y ago
> Or get the data from a random sample of users. You don't need data from everyone. To my amateur ear, that actually sounds like a good compromise to lessen the blow somewhat more. You should suggest it to Mozilla :)
- mintplant 9y ago
  That's what's proposed here. I guess no one actually read the post...?
  - dsp1234 9y ago
    There is no mention that once validated, that the RAPPOR-based metrics when fully deployed would be take from a random population. Only that the initial study of the system will be done to a random population. FTA: What we plan to do now is run an opt-out SHIELD study [6] to validate our implementation of RAPPOR. This study will collect the value for users’ home page (eTLD+1) for a randomly selected group of our release population We are hoping to launch this in mid-September." Notably: "this study will collect ... for a randomly selected group" [6] - https://wiki.mozilla.org/Firefox/Shield/Shield_Studies https://wiki.mozilla.org/Firefox/Shield/Shield_Studies
  - stewbrew 9y ago
    I added the second part about the random sample later to the comment when I moved the proposal already out of my short term memory. I hope they use the data from their initial study to test whether the opt-out group actually is different from the group they already get data from.
- Ajedi32 9y ago
  I'm not sure how that would help. If I opt-out of data collection, I don't think I'd be particularly pleased if I get randomly selected to be one of the users in this "random sample" and the stats get sent anyway. And if I opt-in to data collection, why would it matter to me whether the stats I'm sending are a result of me being selected as part of a random sample or not? Might as well just _always_ send those stats; it doesn't matter to me.
AdmiralAsshat 9y ago
This. Firefox prompts for feedback semi-regularly. I seem to recall it even bends over backwards to make it end-user friendly by having "Firefox made me happy / Firefox made me sad" options. It seems like it would not be difficult to tie that screen to a secondary prompt that says, "Can you briefly turn on some additional telemetry for us so that we can try to fix the problem?" Let the users make that choice to temporarily lower their shields so that you can get some useful data out of their machines, in exchange for the implicit pledge that you will use this data for troubleshooting this one explicit issue (i.e. whatever prompted them to click "Firefox made me sad"). That seems like a reasonable compromise to me. I'm happy to send logs if my browser crashes whenever I visit a certain page, and if I know I'm gonna be monitored for that period, I'll isolate my browsing habit to only visit that page. I do not consent, however, to sending everything--even anonymized--on the offchance that Mozilla will see the crash events and use it to flag that domain and maybe fix the issue on that particular page.
ace_of_spades 9y ago
Awesome idea... why not introduce a "reproduce bug" mode which basically monitors all things in detail. If people are annoyed enough to send bug reports there is a good chance they will use it if properly presented. If people are not filing bug reports you don't really have a business there... If you generally need more of this telemetry advertise it and use it yourself... That sounds way more reasonable to me.

Vinnl 9y ago

> Or get the data from a random sample of users. You don't need data from everyone. To my amateur ear, that actually sounds like a good compromise to lessen the blow somewhat more. You should suggest it to Mozilla :)

mintplant 9y ago
That's what's proposed here. I guess no one actually read the post...?
- dsp1234 9y ago
  There is no mention that once validated, that the RAPPOR-based metrics when fully deployed would be take from a random population. Only that the initial study of the system will be done to a random population. FTA: What we plan to do now is run an opt-out SHIELD study [6] to validate our implementation of RAPPOR. This study will collect the value for users’ home page (eTLD+1) for a randomly selected group of our release population We are hoping to launch this in mid-September." Notably: "this study will collect ... for a randomly selected group" [6] - https://wiki.mozilla.org/Firefox/Shield/Shield_Studies https://wiki.mozilla.org/Firefox/Shield/Shield_Studies
- stewbrew 9y ago
  I added the second part about the random sample later to the comment when I moved the proposal already out of my short term memory. I hope they use the data from their initial study to test whether the opt-out group actually is different from the group they already get data from.
Ajedi32 9y ago
I'm not sure how that would help. If I opt-out of data collection, I don't think I'd be particularly pleased if I get randomly selected to be one of the users in this "random sample" and the stats get sent anyway. And if I opt-in to data collection, why would it matter to me whether the stats I'm sending are a result of me being selected as part of a random sample or not? Might as well just _always_ send those stats; it doesn't matter to me.

AdmiralAsshat 9y ago

This. Firefox prompts for feedback semi-regularly. I seem to recall it even bends over backwards to make it end-user friendly by having "Firefox made me happy / Firefox made me sad" options. It seems like it would not be difficult to tie that screen to a secondary prompt that says, "Can you briefly turn on some additional telemetry for us so that we can try to fix the problem?" Let the users make that choice to temporarily lower their shields so that you can get some useful data out of their machines, in exchange for the implicit pledge that you will use this data for troubleshooting this one explicit issue (i.e. whatever prompted them to click "Firefox made me sad"). That seems like a reasonable compromise to me. I'm happy to send logs if my browser crashes whenever I visit a certain page, and if I know I'm gonna be monitored for that period, I'll isolate my browsing habit to only visit that page. I do not consent, however, to sending everything--even anonymized--on the offchance that Mozilla will see the crash events and use it to flag that domain and maybe fix the issue on that particular page.

ace_of_spades 9y ago

Awesome idea... why not introduce a "reproduce bug" mode which basically monitors all things in detail. If people are annoyed enough to send bug reports there is a good chance they will use it if properly presented. If people are not filing bug reports you don't really have a business there... If you generally need more of this telemetry advertise it and use it yourself... That sounds way more reasonable to me.